MyDefrag a failure, made fragmentation much worse.

[Roger#73]

I used MyDefrag for the first time on a small FAT32 system drive of 35GB using the "System Disk Monthly" script. The process was slow, taking at least 8-9 hours and afterwards, the drive was much more fragmented than before, with the log showing 9.8GB fragmented and 14.8GB unfragmented. This is about 40%. An analysis by the Windows tool shows 32% fragmented and recommends defragmentation. There are now a huge number of "unmovable" files scattered all over the drive that weren't there before. The drive appears to be completely messed up and the process a total failure. What can I do to get my drive back to some semblance of order? Does anyone know of another tool that will properly defragment a FAT32 drive?

[Darlis]

MyDefrag can not move directories on FAT volumes, see [FAQ]Known problems. If a defragmenter can move directories, it uses it's own method to do so and not the safe Windows Defragmentation API, make sure that you have a proper backup. Upgrade to NTFS if possible.

The huge fragmentation that other programs see comes from the "wrap-around" fragmentation MyDefrag creates, see [FAQ]Why does MyDefrag not perfectly optimize my disk?.

[Roger#73]

Thank you but the huge fragmentation statistics come from MyDefrag's own log file not another program. Surely a fragmentation of 40% of used space is not an acceptable result after a defragmentation? I believe hiberfil.sys can't be defragged but I removed the page file before starting.

MyDefrag.log
Total disk space:      38,336,856,064 bytes (35.7040 GigaBytes)
Volume type:        FAT32

Unfragmented items:    14,896,463,872 bytes            86,512 items
Fragmented items:       9,849,602,048 bytes               943 items
Gaps:                  13,590,757,376 bytes             4,452 gaps
Average gap:                3,047,424 bytes
Median gap:                   163,840 bytes
Biggest gap:              533,037,056 bytes
Average end-begin distance:      294,328.6466 clusters

[ff_mfg]

When processing any disk with SortBy()s, which are part of Monthly scripts, there is a chance of temporarily creating much more fragmentation while a lot of files are being moved out to sort the files. This is especially noticeable if free space is low (if free space is plenty, files moving out do not get fragmented as much). This temporary fragmentation should go away when fragmentation is complete, obviously. However, if at the time of increased fragmentation some unfortunate disk activity happens, which locks those temporarily fragmented files - that's when it can get ugly.

1. Remember about wrap-around fragmentation (already mentioned before, but worth repeating).
2. Directories on FAT - extra unmovables (same). Upgrade to NTFS if possible.
3. Before defragmenting with Monthly stop as much disk activity as possible, that is, quit all background programs, disable antivirus, turn off excessive logging (for example firewall logs on a busy network can generate a lot of writes - small, but frequent), temporarily stop Windows services that can decide to write disk (like indexing, backup, autoupdates).
4. Free up as much free space as possible (delete restore points, browser caches, clear temp directories, etc.)

[Darlis]

943 out of 86,512 items are fragmented, that is about 1% of all items. But these files take over more than half of the used space on the disk, so they must be quite large, and fragmentation does not matter (much) on large files.

The many gaps were probably created because new files were created during the defragmentation process and since MyDefrag can not write to already used space it created wrap-around fragments. These new files (or others) must have been removed later so that all these gaps show up.

Note that if (unmovable) directories are scattered on free space (red dots on the disk map) the gap-count will also go up.

[Roger#73]

There were no programs doing anything in the background that I am aware of that could have written files to the disk during defrag as suggested. Also, no files were deleted from the drive after defragmentation. The only program I forgot to disable was Windows Defender but I don't think that wrote anything to the disk. The drive was relatively clean before I started except I did not remove the old System Restore files. I am surprised MyDefrag is so sensitive to disk activity, as Puran Defrag say "Puran Defrag allows you to use your computer at full capacity even when defragmentation is running." I shut down my Realtime virus scanner, also my LAN and internet connection and I don't have any indexing, backups or autoupdates running. I am fairly sure the computer was doing nothing except running MyDefrag.

The disk has been left with a large number of supposedly unmovable files (not just folders) but there seems no reason why these should be unmovable. I did an analysis with Defraggler and this showed details of fragmentation remaining. Most noticeable were the Outlook Express .dbx files, all fragmented, many in 50 to 75 fragments. I defragmented these and several other files individually with Defraggler and OE is now much more responsive. Some less important files are still  fragmented in up to 100 fragments.

I tried MyDefrag because of the good revues and would like to use it. I don't think I can upgrade to NTFS as the computer is an Acer 1694 XP SP3 and came from the factory with FAT32, which I understand is necessary for the system back up on a separate partition to work. Is it being suggested that I shut down all processes and some services to use MyDefrag as I think this would be too much bother? Is the time of about 8 hours expected on a 35GB drive? Do you suggest I try running MyDefrag again or could I make matters worse? If files have been "locked" fragmented as suggested by ff_mfg, because of file activity during defragmentation, can these files be "unlocked" by running the program again? "Ugly" doesn't sound good.

[Darlis]

The problem in your case is the combination of a lot of unmovable files (the Windows Defragmentation API refuses to move directories on FAT volumes = unmovable file) and the sorting operations of MyDefrag that create the wrap-around fragmentation in the first place. The Defragmentation script will defragment any file, even if it's a wrap-around fragmented (the file has to be movable though). The Daily and Weekly scripts respect the warp-around fragmentation but won't create new one.

So, if you would run the Monthly script again, you will end up with the same fragmentation as in the last run because of the unmovable directories.

There are a lot of backup tools out there, so you're not necessarily stuck with the build-in backup of your laptop. Usually these backups overwrite the whole system partition, no matter with what kind of file system is was formatted. You don't have to convert the backup partition, in fact, you shouldn't even touch it if you want to use it. It's likely that the backup tool expects some files to be at a certain location on the volume and the backup will fail if these files are moved around.

If files have been "locked" fragmented as suggested by ff_mfg, because of file activity during defragmentation, can these files be "unlocked" by running the program again? "Ugly" doesn't sound good.

I think ff_mfg wanted to point out that MyDefrag treats any file that was created or removed during the defragmentation as an unmovable files, causing the "ugly" wrap-around fragmentation (Jeroen is going to change that in a future release). But these files are only unmovable for MyDefrag and only for the currently running script. If you run MyDefrag again, these files will be recognized and treated as normal files.

[Roger#73]

Thank you for the advice. I am fairly sure of two things. There were no files created or removed during the defragmentation process and all the files showing as unmovable are not all folders. Doing an analysis with MyDefrag and hovering the mouse pointer over the unmovable blocks shows the names of the files and many of these are part of a large game called Company of Heroes, some up to 500MB in size. Could the limited amount of free space on the drive (11GB) have caused MyDefrag to designate them unmovable? The Acer computer was not supplied with an XP CD so the restore partition is the only way of restoring it to factory new condition in the event of a catastrophic failure, so I am reluctant to do anything that might render restore unusable. I am wondering if, with a quite full FAT32 drive, MyDefrag is not the right program for me to use.

[Darlis]

11Gb is usually enough space but the problem is probably that MyDefrag cannot find enough free continuous space for large files because of the unmovable directories. Did you try the other scripts I mentioned?

There are some defragmenters that can move directories on FAT volumes but you should be careful using them since they use their own methods of doing so. Consolidating the directories will also result in a performance improvement.

The Acer computer was not supplied with an XP CD so the restore partition is the only way of restoring it to factory new condition in the event of a catastrophic failure, so I am reluctant to do anything that might render restore unusable.

What if the catastrophic failure is a disk failure? The restore partition is of no use in that case.

[Roger#73]

What if the catastrophic failure is a disk failure? The restore partition is of no use in that case.

Point taken, but if I have a disk failure, it will be a new computer and OS for me. Having come through Windows 3.1/95/98, FAT has a certain familiarity. I am puzzled why moving directories is such a risk in XP as I used to do it all the time when defragmenting 95/98 drives without any problems. XP seems to be designed not to work well with FAT32 (formatting large disks, defragmenting problems) but it seems strange that Microsoft would cripple the XP API compared with Win98 and that moving directories could become such a risk. I believe O&O Defrag will move FAT32 directories so I may give it a try.

[BloodySword]

Just update to NTFS, there are no reasons to do not.

There is no reason to use Windows 9x anymore. It could be a good idea to use Windows 3.11 on 64 bit Windows 7 since there is no 16 bit emulation available, but this can be done in DOSBOX or any virtualization software like VirtualBox or vmWare.

Mordern Linux systems can read and even write to NTFS without any flaws.

So give it a try, backup everything and then convert to NTFS.