[Script] Jaspion Data Disk Optimization v3.3

[Jaspion]

Jaspion Data Disk Optimization Scripts v3.3 - 10 October 2012

##################
# Data Disk Optimization #
##################

- Folder-based organization.

- 7 Levels of defragmentation: 6 Levels + 1 Fast Level.
  Fast Levels do not process the MFT and special NTFS files
  to diminish drive wear and defrag times.

- txt CONFIGFILEs to easily set configurations across all Levels.

- Priority Zone: the user may select files to be placed first.

- Last-place Zone: the user may select files to be placed last.
  These files will be processed first and placed at the end
  of the disk. Best used with very large, rarely-used files only.

- Exclusion list: the user may select files to be ignored.

- Larger files are placed further back, thus improving seek times.
  This functionality may be altered or disabled.

- Configurable variables via "Data Disk Variables.MyD".

File layout and respective processing order:
  Note: *  = configurable via corresponding CONFIGFILE.
        ** = configurable via "Data Disk Variables.MyD".

End of the disk:
  Optional last-place files*.
      (These are processed first in order to place very large files
       out of the way where they won't have to be moved again.)
System files position**:
  Directories.
  MFT and some other special NTFS files.
      (The predefined SystemFilesPosition is 25% of the VolumeUsed.
       This parameter may be changed via "Data Disk Variables.MyD".)
Beginning of the disk:
  Optional priority files*.
  Remaining files smaller than a determined size**.
  All remaining files.


Example script:

Code:

##############################
#  Jaspion MyDefrag Scripts  #
##############################

#  Data Disk Optimization v3.3 2012-10-10
#  compatible with MyDefrag v4.3.0 and 4.3.1

Title('Data Disk Optimization - Level 6')
Description("This script performs a complete data disk optimization for
semestral to yearly usage. FILE SORTING: all files.")

# Load the variables from the 'Data Disk Variables.MyD' script
RunScript("Data Disk Variables.MyD")

# Load the exclusion list to prevent certain files from being processed.
!include "CONFIGFILE - Global Exclusion List.txt"!

/* Write the header to the logfile. See the "Settings.MyD" file for the
   definition of the "LogHeader" string. */
WriteLogfile("!ScriptTitle!.log","LogHeader")

/* Select and process the volumes one by one. */
VolumeSelect
  CommandlineVolumes()
VolumeActions

  /* Write the "before" statistics to the logfile. See the "Settings.MyD" file
     for the definition of the "LogBefore" string. */
  AppendLogfile("!ScriptTitle!.log","LogBefore")

  /* Run the variables script again to update volume dependant values. */
  RunScript("Data Disk Variables.MyD")

  /* Zone 1 - Place files at the end of the disk. */
  FileSelect
    !include "CONFIGFILE - Last-place Files.txt"!
  FileActions
    MoveToEndOfDisk()
  FileEnd

  /* Place files at a determined percentage into the data on the disk. */
  MakeGap(RoundUp(SystemFilesPosition,RoundSystemFilesPosition), DoNotVacate)

  /* Zone 2 - Directories. */
  FileSelect
    Directory(yes)
  FileActions
    Defragment(Fast)
    Defragment()
    MoveUpToZone()
    FastFill(WithShuffling)
    AddGap(RoundUp(ZoneEnd, 32M))
  FileEnd

  /* Zone 3 - Place the MFT and some other special NTFS files. */
  FileSelect
    SelectNtfsSystemFiles(yes)
  FileActions
    PlaceNtfsSystemFiles(Ascending,MftSize * 0.1)
  FileEnd

  /* Place files at the beginning of the disk. */
  MakeGap(0)

  /* Zone 4 - Priority files. */
  FileSelect
     !include "CONFIGFILE - Data Disk Priority Files.txt"!
  FileActions
    SortByName(Ascending)
  FileEnd

  /* Zone 5 - Remaining files smaller than a determined size. */
  FileSelect
    Size(0,LargeFileSize)
  FileActions
    SortByName(Ascending)
  FileEnd

  /* Zone 6 - All remaining files. */
  FileSelect
    all
  FileActions
    SortByName(Ascending)
  FileEnd

  /* Write the "after" statistics to the logfile. See the "Settings.MyD" file
     for the definition of the "LogAfter" string. */
  AppendLogfile("!ScriptTitle!.log","LogAfter")

VolumeEnd

/* Write the footer to the logfile. See the "Settings.MyD" file for the
   definition of the "LogFooter" string. */
AppendLogfile("!ScriptTitle!.log","LogFooter")

Please let me know of any problems or suggestions.

Happy optimizing! 

Download link (includes Jaspion System Disk Optimization Scripts) here: http://www.mydefrag.com/forum/index.php?topic=5843.0

[Darlis]

I don't think that you've got the ChunkSize option right: Defragment(ChunkSize(1024)) will leave only fragments (if any) that are larger than 1GB, so any subsequent defragment action with a smaller chunk size is useless.
Also, having an additional defragment action does not result in faster defrags: MyDefrag can't do in-place defragmentation, so whenever a file needs to be defragmented the whole file will be moved.

[Jaspion]

I don't think that you've got the ChunkSize option right: Defragment(ChunkSize(1024)) will leave only fragments (if any) that are larger than 1GB, so any subsequent defragment action with a smaller chunk size is useless.
Also, having an additional defragment action does not result in faster defrags: MyDefrag can't do in-place defragmentation, so whenever a file needs to be defragmented the whole file will be moved.

Hi Darlis! Thanks for your help again. Since it's normal to set things like this:

Defragment(Fast)
Defragment()

I thought the defragment command would just try the next command always, including with the ChunkSize option. I imagined first very big files would be defragmented (if possible) in 1GB chunks, then if no 1GB spaces (or files that big) are available, follow to do that with 512MB chunks, and so on.

So this function doesn't work like that? I can have only:

Defragment(Fast)
Defragment()

OR

Defragment(ChunkSize(X))
Defragment()

But not:

Defragment(ChunkSize(X))
Defragment(Fast)
Defragment()

OR

Defragment(ChunkSize(Y))
Defragment(ChunkSize(X))
Defragment(Fast)
Defragment()

Please explain.

[Darlis]

I thought the defragment command would just try the next command always, including with the ChunkSize option. I imagined first very big files would be defragmented (if possible) in 1GB chunks, then if no 1GB spaces (or files that big) are available, follow to do that with 512MB chunks, and so on.

The first part is true but for the second one you have to know that MyDefrag consolidates all fragments from the first fragment on that fulfills the Chunksize option. So, if the volume is that full (or has too much fragmented space) so that the fragments can't be combined to one fragment, a smaller chunk size won't help.

But not:

Defragment(ChunkSize(X))
Defragment(Fast)
Defragment()

Technically you can use this but if the disk has enough free space, using the Chunksize beforehand does more work than necessary.
Example with a chunk size of 1GB and a file that has these fragments: 2GB-0.5GB-1GB
First, Defragment(ChunkSize(1024)) moves 1.5GB data, so the result will be 2GB-1.5GB.
Then Defragment(Fast) (or Defragment()) moves 3.5GB data, so the result will be 3.5GB (defragmented file).
So, you version moves 1.5GB more than the regular Defragment() combo.

If you want less file movement you should probably go with only Defragment(ChunkSize(X)).

[Jaspion]

I thought the defragment command would just try the next command always, including with the ChunkSize option. I imagined first very big files would be defragmented (if possible) in 1GB chunks, then if no 1GB spaces (or files that big) are available, follow to do that with 512MB chunks, and so on.

The first part is true but for the second one you have to know that MyDefrag consolidates all fragments from the first fragment on that fulfills the Chunksize option. So, if the volume is that full (or has too much fragmented space) so that the fragments can't be combined to one fragment, a smaller chunk size won't help.

But not:

Defragment(ChunkSize(X))
Defragment(Fast)
Defragment()

Technically you can use this but if the disk has enough free space, using the Chunksize beforehand does more work than necessary.
Example with a chunk size of 1GB and a file that has these fragments: 2GB-0.5GB-1GB
First, Defragment(ChunkSize(1024)) moves 1.5GB data, so the result will be 2GB-1.5GB.
Then Defragment(Fast) (or Defragment()) moves 3.5GB data, so the result will be 3.5GB (defragmented file).
So, you version moves 1.5GB more than the regular Defragment() combo.

If you want less file movement you should probably go with only Defragment(ChunkSize(X)).

I see! Thank you! So just to be clear, good options are:

Defragment(Fast)
Defragment()

OR

Defragment()

OR

Defragment(Fast)

OR

Defragment(ChunkSize(X))

I've been using the second option -- just the Defragment() command. Is it really better to combine Defragment(Fast) with Defragment()?

[Darlis]

I've been using the second option -- just the Defragment() command. Is it really better to combine Defragment(Fast) with Defragment()?

Check out what the Fast option does. Since Jeroen uses this combination in the default scripts it seems to be the preferable combination.

[Jaspion]

I've been using the second option -- just the Defragment() command. Is it really better to combine Defragment(Fast) with Defragment()?

Check out what the Fast option does. Since Jeroen uses this combination in the default scripts it seems to be the preferable combination.

I know, it moves files if it can find a big enough gap for the entire file. But since Defragment() will be used later, wouldn't it be better to have only Defragment() so this command can also shuffle files that would have been already moved by Defragment(Fast)?

[Darlis]

Like I said, Jeroen uses this combination. I don't know the exact difference between this combo and a single Defragment().

[jeroen]

But since Defragment() will be used later, wouldn't it be better to have only Defragment() so this command can also shuffle files that would have been already moved by Defragment(Fast)?

Files that have been processed by Defragment(Fast) and that have been successfully defragmented, will not be processed again by Defragment().

I don't know the exact difference between this combo and a single Defragment().

The combo will finish sooner, that's all. The end result is the same. The Defragment() algorithm is thorough, but slow, so it's quicker to use Defragment(Fast) first, for the easy fragmentation.

[Jaspion]

Files that have been processed by Defragment(Fast) and that have been successfully defragmented, will not be processed again by Defragment().

The combo will finish sooner, that's all. The end result is the same. The Defragment() algorithm is thorough, but slow, so it's quicker to use Defragment(Fast) first, for the easy fragmentation.

Thanks Jeroen!

Changes according to Jeroen's and Darlis' invaluable inputs will be implemented soon and released in a new version!

I am currently working on my System Disk Scripts as well, v3.0RC will come soon.

[Jaspion]

Hi Jeroen!

I was about to release v1.1 of my Data Disk Scripts but I just had to ask you something. I need to confirm two things about how MyDefrag works. Darlis explained:

I thought the defragment command would just try the next command always, including with the ChunkSize option. I imagined first very big files would be defragmented (if possible) in 1GB chunks, then if no 1GB spaces (or files that big) are available, follow to do that with 512MB chunks, and so on.

The first part is true but for the second one you have to know that MyDefrag consolidates all fragments from the first fragment on that fulfills the Chunksize option. So, if the volume is that full (or has too much fragmented space) so that the fragments can't be combined to one fragment, a smaller chunk size won't help.

So let me see if I understand this: ChunkSize only works if there is a big enough gap for the entire fire?

But (can)not (use):

Defragment(ChunkSize(X))
Defragment(Fast)
Defragment()

Technically you can use this but if the disk has enough free space, using the Chunksize beforehand does more work than necessary.
Example with a chunk size of 1GB and a file that has these fragments: 2GB-0.5GB-1GB
First, Defragment(ChunkSize(1024)) moves 1.5GB data, so the result will be 2GB-1.5GB.
Then Defragment(Fast) (or Defragment()) moves 3.5GB data, so the result will be 3.5GB (defragmented file).
So, you version moves 1.5GB more than the regular Defragment() combo.

If you want less file movement you should probably go with only Defragment(ChunkSize(X)).

So I want to confirm these before posting my new scripts. I'm confused because:

Files that have been processed by Defragment(Fast) and that have been successfully defragmented, will not be processed again by Defragment().

Is this also true of ChunkSize? That is, files that have been processed by ChunkSize(Y) will be left alone by Defragment() or Defragment(Fast)? Can I have more than one ChunkSize command? How will MyDefrag behave in these cases?

Thanks for the help!

[jeroen]

ChunkSize only works if there is a big enough gap for the entire fire?

Yes. Same as with Defragment(Fast).

Files that have been processed by ChunkSize(Y) will be left alone by Defragment() or Defragment(Fast)?

No. Defragment() and Defragment(Fast) will process files that are fragmented, and skip unfragmented files. If a file is successfully defragmented by Defragment(Fast) then it will not be processed again by Defragment(), because it is not fragmented any more. ChunkSize defragment does not fully defragment a file - that's the whole point of it, it's a partial defragment. It's designed for large files such as logfiles and databases, that grow and fragment at their end. So after a ChunkSize defragment a file will still be fragmented, albeit LESS fragmented, and therefore Defragment() or Defragment(Fast) will process the file again.

Can I have more than one ChunkSize command? How will MyDefrag behave in these cases?

Yes, you can have more than one Defragment(ChunkSize) command. For every command MyDefrag will look at all the files and process the files that meet the criteria. For ChunkSize a file must have at least 2 fragments, and at least 1 fragment that is smaller than the specified ChunkSize. MyDefrag will then defragment the file starting at that smaller fragment, exactly as Darlis described in the 2GB-0.5GB-1GB example. Multiple ChunkSize defragments will each look at all the files and will process files that meet the criteria, perhaps processing a file again that was already processed by a previous ChunkSize defragment.

[Jaspion]

Thank you so much Jeroen!! It is clear now!

Darlis too thank you for your help!

[Jaspion]

Version 2.0 released!

Check first post for info & download link.

[Jaspion]

Version 3.2 released!

More info on first post.