Contentious Issue: Defragmentation DECREASES performance

[int13]

Hi All,

Years back, when I ran my 1st defrag on Win95osr1, my PC, very
perceptibly, slowed down afterwards.

In retrospect, the reason was obvious: with fragments everywhere and
free space scattered throughout, new files (or extensions to existing
files) always had some place "nearby" to be stored.  After Win95
defragmentation, everything was packed neatly at the start of the
disk, so later on - new files etc always got written near the end:
which was the absolute worst place for them to go - the disk heads
spent all their time in the "post defragment" system jumping back and
forth from the FAT16 directories in cylinder 0, to the end of the disk
where all new data gets stored.

Fast-forward to today: we don't use a small number of files anymore,
nor do we use slow old disk drives, and we almost always have
read/write caches both in hardware+software.  There's a pretty good
chance that a defrag won't improve anything anyhow, but I still
believe a defrag will SLOW-DOWN the average PC still.  Why?  Well -
programs and application files are usually larger than a single
cluster, and are almost never read in their entirety.  If you run an
executable program, Windows loads some of the code and parts of the
data, and the rest sits unused on your drive (until you happen to need
to access it - eg - if you choose to "print" from MS-Word, the
printing codepages will get read-in from disk... etc).  When you work
on things, usually only the bits of files relevant to your work get
loaded.  When things get logged, they only ever append to the end of
log file.  None of this activity is improved by defragmenting - it's
actually almost all made worse.

If a fragmentation-management program were written to actually improve
the performance of Windows - it would deliberately RE-FRAGMENT or
PRE-FRAGMENT your hard disk... for example - it would allocate "holes"
at the end of log files to allow for them to expand.  It would monitor
system usage, and move the fragments of files that actually get *used*
all into consecutive sectors, and move the stuff that doesn't get used
out of the way, so it never gets incorrectly "pre-cached" by hardware
or software read-ahead buffers.

The perfect defragmentation tool would have monitored my system usage.
It would know that everyime I read (for example) cluster 12345 (the
start of WINWORD.EXE), that I'm always going to be reading clusters
100, 352423, 1234, 7789, ... etc next (the related DLLs, .INIs,
overlays, .DOT templates, etc etc) - so it should have pre-fragmented
& arranged all these clusters from all these files so they're
consecutive on disk.  My lovely 32mb readahead buffer built into my
latest SATA-II drive will then have automatically pre-loaded pretty
much every cluster that's going to be needed - ms-word will fire up
instantly, getting almost everything it wants directly from the SATA
cache.

So - next time you're hitting the "defrag" button, ponder for a moment
what convinced you to do so, and why there's no evidence you can find
to back up the reasoning :-)

Chris.

p.s. defrag inside vmware images and subsequent duplication (eg: via
     DriveImageXML) *is* a really great way to shrink the heck out of
     your vmware virtual drive files... I shrunk a 30gig image down to
     2.8gigs just now... so there *is* a use for defragmentation
     still.

[Danilovic]

Hi All,

Years back, when I ran my 1st defrag on Win95osr1, my PC, very
perceptibly, slowed down afterwards.

In retrospect, the reason was obvious: with fragments everywhere and
free space scattered throughout, new files (or extensions to existing
files) always had some place "nearby" to be stored.  After Win95
defragmentation, everything was packed neatly at the start of the
disk, so later on - new files etc always got written near the end:
which was the absolute worst place for them to go - the disk heads
spent all their time in the "post defragment" system jumping back and
forth from the FAT16 directories in cylinder 0, to the end of the disk
where all new data gets stored.

With JkDefrag you can reserve a free space at begin of disk for
temporary and new file. Option -f.

If a fragmentation-management program were written to actually improve
the performance of Windows - it would deliberately RE-FRAGMENT or
PRE-FRAGMENT your hard disk... for example - it would allocate "holes"
at the end of log files to allow for them to expand.  It would monitor
system usage, and move the fragments of files that actually get *used*
all into consecutive sectors, and move the stuff that doesn't get used
out of the way, so it never gets incorrectly "pre-cached" by hardware
or software read-ahead buffers.

I think it's in development this feature.

For the other consideration I think (for my esperience) defrag is useful.
IMO.
Danilovic

[jeroen]

usually only the bits of files relevant to your work get loaded.

If only that were true! All Windows versions contain a (hidden) diskcache, which will preload large chunks of files. Even if an application asks for only 1 byte from a data file, then the cache will still load a big piece of that file (more than 1 cluster). Secondly, the vast majority of files are always completely loaded, such as .exe programs, dll's, drivers, images, fonts, and many other files. So I'm sorry, but your used-only-fragments optimization idea will not work.

it would allocate "holes" at the end of log files to allow for them to expand.

Some time ago I have had this idea myself and played with it a little bit. But sadly enough Windows uses a weird algorithm to allocate disk space. As far as I can tell, Windows will not always use free space at the end of a (log)file. Nevertheless, I am planning to implement this in a future version of JkDefrag.