[Script] Robust System Disk script (good access performance with less shuffling)

[Bill]

Do you prefer folders to be included in zone 1 + 2 and NTFS system files in zone 1+2+3 ( if match the boolean ) ?

Zone 1 contains layout.ini boot files and directories that are in the Windows directory only.
I trust the inclusion of directories in layout.ini. Most Windows sub-directories in layout.ini seem grouped/clustered together.
Movable NTFS system files don't seem to be in this zone.

Zone 2 contains programs hints prefetch files, which doesn't seem to contain directories or NTFS system files.

Zone 3 is a scratch area that contains small and medium files modified recently.
On my system, this zone contains a $TxfLogContainer NTFS system file. If it is really being modified recently (as having been selected by the file boolean) then it's probably better to keep that unstable file outside the stable NTFS system files zone (zone 4). On the other hand, if I include it in zone 4 then it could result in a little less shuffling for zone 3, and the shuffling in zone 4 might be minor, but I haven't really tested that option. I'm open to the suggestion of excluding NTFS system files from zone 3.
I prefer the option that results in the minimum amount of data shuffling when defragging.

[Bill]

I've updated the script to exclude NTFS system files from the scratch area (zone 3).

In other news, I incorrectly assumed that setting 'skipBlockSize' to ZONE222N would be sufficient to compact all files belonging to each zone, but that is not correct. I have updated the script's comments to indicate that both 'skipNumFiles' and 'skipBlockSize' must be set to ZONE220N and ZONE222N respectively on the script's first run to properly order the zones and compact all files belonging to each zone.

After setting 'skipNumFiles' and 'skipBlockSize' to ZONE220N and ZONE222N respectively, the diskmaps below show the result of running the Robust System Disk script on my system drive.

MyDefrag & PerfectDisk:

[Bill]

It seems most Windows boot files that are modified recently don't usually become fragmented even when they are modified. So there isn't any need to keep those files in the scratch area.
I have updated the script to keep all small and medium Windows boot files in zone 1. This may cause more data shuffling for zone 1 (which is already an unstable zone), but boot performance should be a little better (i.e. less disk-head thrashing or seeking), and a little less data shuffling in the scratch area.

Hopefully, this will be the last final version of this script.

[torned]

I try it, and i love it.
Very good performance, computer more responsive.

Thank you.

[Esteban4u]

Hello Bill,

I tried your script and I did get better performance from my system drive. I do have a some questions if you don't mind:

1. How often should this script be run on a system drive (Daily, Weekly or Monthly) ?

2. Will the other bundled scripts that are installed with MyDefrag (Eg. SystemDiskDaily.MyD) undo the optimizations that your script performs on a system drive?

3. Do you recommend running your script on non-system (Data) disks? -If not, do you intend on creating a data disk version of your script? -Or do find the bundled data disk scripts (Eg. DataDiskDaily.MyD) to be sufficient?

[Bill]

I try it, and i love it.
Very good performance, computer more responsive.

Thank you.

You're welcome. Thanks to Jeroen for giving us the power of scripting!

I tried your script and I did get better performance from my system drive. I do have a some questions if you don't mind:

Thanks for the questions. Sorry for not replying sooner, I've been very busy.

1. How often should this script be run on a system drive (Daily, Weekly or Monthly) ?

If files are frequently added and deleted then run it weekly, else run it monthly.

2. Will the other bundled scripts that are installed with MyDefrag (Eg. SystemDiskDaily.MyD) undo the optimizations that your script performs on a system drive?

Yes, zone ordering and file placement are different.

3. Do you recommend running your script on non-system (Data) disks? -If not, do you intend on creating a data disk version of your script? -Or do find the bundled data disk scripts (Eg. DataDiskDaily.MyD) to be sufficient?

I haven't tried running the script on data volumes, but the script should still give good performance because it places small files near the beginning of the volume, which gives quick access to those files.

I use specific scripts for each of my data partitions.
For example, the script below is for my documents & programs partition:

Code:

Title('Data Disk Monthly D')
Description("Documents & Programs")

/* Write the header to the logfile. See the "Settings.MyD" file for the
   definition of the "LogHeader" string. */
WriteLogfile("MyDefrag.log","LogHeader")

/* Select and process the volumes one by one. */
VolumeSelect
  Name("D:")
VolumeActions

  /* Write the "before" statistics to the logfile. See the "Settings.MyD" file
     for the definition of the "LogBefore" string. */
  AppendLogfile("MyDefrag.log","LogBefore")

  /* Zone 1: Place the MFT and some other special NTFS files. */
  FileSelect
    SelectNtfsSystemFiles(yes)
  FileActions
    PlaceNtfsSystemFiles(Ascending, Maximum(16MB - MftSize, RoundUp(MftSize * 0.02,1MB)))
  FileEnd

  /* Zone 2: Directories. */
  FileSelect
    Directory(yes)
  FileActions
    SortByName(Ascending)
  FileEnd

  MakeGap(RoundUp(ZoneBegin,1MB))

  /* Zone 3: Music. */
  FileSelect
    FullPath("D:\Music","*")
  FileActions
MoveToEndOfDisk()
  FileEnd

  /* Zone 4: Movies. */
  FileSelect
    FullPath("D:\Movies","*")
  FileActions
MoveToEndOfDisk()
  FileEnd

  /* Zone 5: Large document files. */
  FileSelect
    FullPath("D:\Documents","*") AND Size(10MB,0)
  FileActions
MoveToEndOfDisk()
  FileEnd
 
  /* Zone 6: Large program files. */
  FileSelect
    FullPath("D:\Programs","*") AND Size(30MB,0)
  FileActions
MoveToEndOfDisk()
  FileEnd

  /* Zone 7: Program files. */
  FileSelect
    FullPath("D:\Programs","*")
  FileActions
MoveToEndOfDisk()
  FileEnd

  /* Zone 8: Large temp files. */
  FileSelect
    FullPath("D:\Temp","*") AND Size(30MB,0)
  FileActions
MoveToEndOfDisk()
  FileEnd

  /* Zone 9: Temp files. */
  FileSelect
    FullPath("D:\Temp","*")
  FileActions
MoveToEndOfDisk()
  FileEnd

  /* Zone 10: All other files. */
  FileSelect
    all
  FileActions
    SortByName(Ascending)
  FileEnd

  /* Write the "after" statistics to the logfile. See the "Settings.MyD" file
     for the definition of the "LogAfter" string. */
  AppendLogfile("MyDefrag.log","LogAfter")

VolumeEnd

/* Write the footer to the logfile. See the "Settings.MyD" file for the
   definition of the "LogFooter" string. */
AppendLogfile("MyDefrag.log","LogFooter")

Another example, the script below is for my videos partition:

Code:

Title('Data Disk Monthly Videos')
Description("Videos")

/* Write the header to the logfile. See the "Settings.MyD" file for the
   definition of the "LogHeader" string. */
WriteLogfile("MyDefrag.log","LogHeader")

/* Select and process the volumes one by one. */
VolumeSelect
  CommandlineVolumes()
VolumeActions

  /* Write the "before" statistics to the logfile. See the "Settings.MyD" file
     for the definition of the "LogBefore" string. */
  AppendLogfile("MyDefrag.log","LogBefore")

  /* Zone 1: Place the MFT and some other special NTFS files. */
  FileSelect
    SelectNtfsSystemFiles(yes)
  FileActions
    PlaceNtfsSystemFiles(Ascending, Maximum(16MB - MftSize, RoundUp(MftSize * 0.02,1MB)))
  FileEnd

  /* Zone 2: Directories. */
  FileSelect
    Directory(yes)
  FileActions
    SortByName(Ascending)
  FileEnd

  MakeGap(RoundUp(ZoneBegin, 1MB))

  /* Zone 3: Large files. */
  FileSelect
    Size(300MB,0)
  FileActions
MoveToEndOfDisk()
  FileEnd

  /* Zone 4: Large files. */
  FileSelect
    Size(200MB,300MB)
  FileActions
MoveToEndOfDisk()
  FileEnd

  /* Zone 5: Large files. */
  FileSelect
    Size(110MB,200MB)
  FileActions
MoveToEndOfDisk()
  FileEnd

  /* Zone 6: Large files. */
  FileSelect
    Size(20MB,110MB)
  FileActions
MoveToEndOfDisk()
  FileEnd
 
  /* Zone 7: Large files. */
  FileSelect
    Size(4MB,20MB)
  FileActions
MoveToEndOfDisk()
  FileEnd
 
  /* Zone 8: All other files. */
  FileSelect
    all
  FileActions
        Defragment()
        FastFill(WithShuffling)
  FileEnd

  /* Write the "after" statistics to the logfile. See the "Settings.MyD" file
     for the definition of the "LogAfter" string. */
  AppendLogfile("MyDefrag.log","LogAfter")

VolumeEnd

/* Write the footer to the logfile. See the "Settings.MyD" file for the
   definition of the "LogFooter" string. */
AppendLogfile("MyDefrag.log","LogFooter")

I haven't had much time to experiment with data disk scripts. Basically I just partition the zones by folder and file size, placing small files near the beginning and large files near the end of the volume.

[Jkr12]

Hi,
are MyD files removed from the post?

[Bill]

Hi,
are MyD files removed from the post?

Copy the code in the first post into a text editor such as Notepad, then save the file as SystemDiskRobust.MyD

[AlexeiK]

bill, can  u add code to defragment this folder C:\Windows\assembly\

its always fragmented
up:defragmented changed to fragmented

[Bill]

Some files in C:\Windows\assembly\ are in zone 1 (layout.ini) and I think the rest should be in zone 7 (small rarely-modified files). Both zones use sort file actions that defragment the files as well.
Are you sure it's not wraparound fragmentation you're seeing?

[AlexeiK]

ok.
will see this files fragments at next defragmenation and report here.

[AlexeiK]

so. i done df-on.
all files got defragmented except
C:\Windows\winsxs\x86_mscorlib_b77a5c561934e089_6.1.7600.16385_none_9c3dd3cb7da78442\mscorlib.dll
C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Build.dll
C:\$MFT
i did 2 times defragmenation and got the same number of fragments of this files.

[poutnik]

Hi Bill,

I wonder, according what criteria do you consider files  < 4 MB as small ?
In fact 4 MB is from some points of view still pretty big.

Considering small/big file access is driven by access time/transfer speed, the grey zone is near 500 kBs.
Considering average file size, typical value for system partition is near 200 kBs. ( can vary a lot, depending on partition stored data.)
Considering median  file size, typical values are 5 - 15 kBs, i.e. a half of files is less than this.

[Bill]

so. i done df-on.
all files got defragmented except
C:\Windows\winsxs\x86_mscorlib_b77a5c561934e089_6.1.7600.16385_none_9c3dd3cb7da78442\mscorlib.dll
C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Build.dll
C:\$MFT
i did 2 times defragmenation and got the same number of fragments of this files.

The two files may be in use by the system (unmovable) or it may be wrap-around fragmentation; in either case I wouldn't worry about it.
How many fragments is the MFT fragmented into? If almost all of the MFT is contiguous then I wouldn't worry too much about it, but the whole MFT should have been defragmented.

Hi Bill,

I wonder, according what criteria do you consider files  < 4 MB as small ? 
In fact 4 MB is from some points of view still pretty big.

Considering access driven by access time versus transfer speed, the grey zone is near 500 kBs.
Considering average file size, typical value for system partition is near 200 kBs, but can very a lot, depending on partition stored data.
Considering median  file size, typical values are 5 - 15 kBs, i.e. a half of files is less than this.

I had made a request for Statistical FileBoolean functions for numeric file properties. Using Windows search I discovered 4MB covers, say, two standard deviations of all files on my system disk (so it covers most files). The distance between the start and end of a 4MB file is small, so random access time is not much affected with many small files placed at the beginning of the disk, also 4MB can still be moved quickly in terms of sequential speed.

[poutnik]

Using Windows search I discovered 4MB covers, say, two standard deviations of all files on my system disk (so it covers most files).
..........
The distance between the start and end of a 4MB file is small, so random access time is not much affected
with many small files placed at the beginning of the disk, also 4MB can still be moved quickly in terms of sequential speed. 

I have guessed you chose the file moving time criteria.
I think it is good for effort (4MB for smalls is good for moving), but not so good for result ( 4MB too big for accessing).
Average access time of files will be possibly affected by scattering very small files in large area.
As far as very most files are very small, and access to very small files is driven by access time....