Maximize RAM use

[BloodySword]

There is no sence for MyDefrag to cache files while the API is not able to use any cache... The only solution would be a VERY BIG copy buffer. So that the HDD is reading a big file block, lets say 128MB, then moves the head to the new track and writes it down there. Is a file is smaller than 128MB, the whole file will be copied. But the Windows API doesn't do this so...

[Falcon4]

How do you know the API doesn't make use of Windows' disk cache? Is that documented?

[BloodySword]

It might makes use of the Disk Cache, but not use of the Windows Cache. If it would make use, defrag would be xtremly faster...

[Falcon4]

It might makes use of the Disk Cache, but not use of the Windows Cache.

They're exactly the same thing...

edit: Unless you're referring to physical on-drive cache, in which case it obviously uses that (which is on the drive itself and is pretty much unavoidable). In that instance, even pre-reading about 1mb of scattered files would still improve performance since they'd be stored in the drive's cache!

But it's still unlikely that it doesn't make use of the Windows disk cache...

[theantagonizer]

I could be incorrect but isn't the reason it is slower in safe mode because it doesnt use the cache? That would mean that during normal booting defrags it does use the cache.

[poutnik]

I could be incorrect but isn't the reason it is slower in safe mode because it doesnt use the cache? That would mean that during normal booting defrags it does use the cache.

It could be reason, but not the only.

I think Win XP or Vista do not have such explicit settings any more,
but I remember Win98 or 95 on FATxx had about 6 settings for advanced disk/file management on FS level.
They affected disk performance and could be switched off for compatibility reason, being all off in Safe mode.

Something similar could happen in Safe mode over NTFS too.

[diagonal]

Only the author of the program can test, whether there will be a package reading of small files, and then their package moving by means API by faster process, than moving one after another.

But it is visible it now solves other problems. Though personally for me it is actual.

Optimising a disk by means of program Vopt, I have an obvious sensation that at first small files are consistently read, and then consistently move to new area.

[amk]

Moreover, looks like Windows locks all read/write operations while perform low-level access

[MessyBlob]

On seeing/hearing the defrag at work, I had the same initial thought most of the above posters: "Could read-ahead speed up the process?"

It crucially depends on where the defrag API sits in the 'command chain' of file operations, and my guess (sorry) is that it would be at a lower level than the Windows file cache, but a higher level than the file system I/O. If someone could confirm this, it should answer the question for everyone who posted here.

What is really needed is a change to the defrag API - out of our hands, really! This airy wishlist would include the ability to be given a 'transactional set' of move instructions that includes several reads and several writes, that commits to the MFT (or journal) at the very last stage. By re-ordering operations, seek efficiencies can be achieved. I expect that shuffling algorithms can be improved too, by being given the ability to plan ahead more.

Background note, alluded to in a previous post: Controllers in some SATA drives can do this (command re-ordering), but drives don't have a big enough cache for this to make much difference to defragging (and even if the cache was big enough, the 'dirty' data would need to be committed far too soon to be effective).

We have to remember what the defrag API is trying to achieve: safe relocation of data. And there's a wider objective in its use: to speed up general operation of the computer's software (something we might forget when tuning: careful not to spend all your time maintaining, and no time using!). The unpopular answer to the problem of 'slow defragging', until the API is drastically improved, is to set it going when you're not using the computer for a few hours (or via the screensaver).

[quanthero]

Defrag API in Windows Vista and 7 deliberately tries to avoid using a lot of cache for moving files. This is most likely done in order to avoid interference of defrag API cache (for moving files) with Superfetch cache.

[BloodySword]

I tend to disable the Superfetch while defragmenting to avoid wrapped around fragmented files. It works well, there are almoust no    occurrence.

[tOM Trottier]

I think it might be worthwhile for Jerome to test out whether prereading files, up to different limits, would be worthwhile. If the windows API does use the cache, there should be improvements quickly shown with bunches of small files since a bunch of  re-reads would not be necessary. There would also be an improvement with hybrid drives.

[ff_mfg]

If someone wants to test it, one doesn't need to be defragmenter developer. All you need is timer utility, some defragmenter which you can feed filelists to (maybe contig? it can't read filelists, but supports wildcards), maybe fragmenter utility (MyFragmenter), and a small batch file. Making a disk image (to revert to) will ensure better testing. Preloading could be done simply by copying to nul.
What you really need is will and time to do it.

P.S. Personally I don't believe it would help. I think defrag API bypasses cache. But what do I know, I might be wrong...

[lugtuava]

It is also in my wishlist. i want to maximize the RAM use of mydefrag so it can perform well in my well in my unit. And i think not all are having the same problem. This is just another scenario