[Script] Optimized System Disk Scripts

[homeless_sapient]

I have tested Ultimate defrag and Mydefrag. It seems that Ultimate Defrag has better boot times, but it is not as thorough like Mydefrag.

Incredible, because UltimateDefrag has an inexplicable error when handling "Respect Layout.ini" option:
• When I select the "Respect Layout.ini" option a lot of files from my Layout.ini are not placed in the required place!
• After a brief surveillance I realized that the missing files are all hard links.

For example the first 8 entries of Layout.ini looks like this:

C:\WINDOWS\SYSTEM32\NTOSKRNL.EXE
C:\WINDOWS\SYSTEM32\PSHED.DLL
C:\WINDOWS\SYSTEM32\KDCOM.DLL
C:\WINDOWS\SYSTEM32\CLFS.SYS
C:\WINDOWS\SYSTEM32\CI.DLL
C:\WINDOWS\SYSTEM32\HAL.DLL
C:\WINDOWS\SYSTEM32\CONFIG\SYSTEM
C:\WINDOWS\SYSTEM32\CONFIG\SOFTWARE

But after the optimization process the Cluster Viewer looks like this:

C:\$Boot
C:\Windows\System32\config\SYSTEM
C:\Windows\System32\config\SOFTWARE

The ntoskrnl.exe, pshed.dll, kdcom.dll, clfs.sys, ci.dll and hal.dll are all missing from the first sectors, and they are all HARD LINKS.

MyDefrag and O&O Defrag don"t miss these files, and the Cluster Viewer looks like this:

C:\$Boot
C:\Windows\winsxs\amd64_microsoft-windows-os-kernel_31bf3856ad364e35_6.1.7601.17640_none_ca31f809cade8847\ntoskrnl.exe
C:\Windows\winsxs\amd64_microsoft-windows-pshed_31bf3856ad364e35_6.1.7600.16385_none_b7e7d4f746c595bb\PSHED.DLL
C:\Windows\winsxs\amd64_microsoft-windows-b..gertransport-serial_31bf3856ad364e35_6.1.7601.17556_none_6fb25371c3691bc8\kdcom.dll
C:\Windows\winsxs\amd64_microsoft-windows-commonlog_31bf3856ad364e35_6.1.7600.16385_none_da778c54413d0c9c\clfs.sys
C:\Windows\winsxs\amd64_microsoft-windows-codeintegrity_31bf3856ad364e35_6.1.7601.17514_none_fe9df6ad1b5f6e87\ci.dll
C:\Windows\winsxs\amd64_microsoft-windows-hal_31bf3856ad364e35_6.1.7601.17514_none_094ef8137049c196\hal.dll
C:\Windows\System32\config\SYSTEM
C:\Windows\System32\config\SOFTWARE

So the "Respect Layout.ini" option is a deficient function in Distrix UltimateDefrag.

[erudite]

MyDefrag does take a while so this is really great.  Thanks.

[Rohk]

Updated the scripts:

• Added an additional RunScript() inside the VolumeActions to update volume-dependent values

Aaah.. 

I was just looking for why does the runscript before VolumeActions pick up wrong volume's VolumeSize variables!

Your scripts are full of knowledge! Thank you 1000x times for creating them and putting all the hard work into them!   

[peptobismal]

Darlis... this is what I was looking for. 

I liked that... not ALL Program Files are lumped together seemed like the most used .exe's were placed pretty close to beginning. All font files were pretty much lumped together at a reasonably close position on the hard drive. Still, finishing up Monthly, will see how well it actually did. 

*** I saw in another post that it is best to disable Windows default Defragmenter? Is that right and how come?

Okay, on my second run through of the script... Let it go through fully the first time and realized there were a lot of programs in AppData and hiberfile.sys that I never use. Luckily, the extra 4gigs emptied made it run through zone 5 really quick... my question though... do you not think that zone 5 could be split into 2 zones? Having all that Windows folder stuff so closely together with the Program Files seems unnecessary?

AMD Catalyst - Seems to take a rather unusually long time to boot anyway to fix the autoruns? All I have set are the default, WiFi, Action Center, Volume Control, Security Center, Realtek HD Manager, and Catalyst.

Kind of glad I didn't delete the original System Disk scripts, no offense.

My Issues:
After defragging... All USB devices (except keyboard/mouse) must now be unplugged and plugged back in to find.
Lots of drivers are not taken into consideration... I was scared of that seeing all my winsxs folders going to the end of program files.
ALL autoruns besides Security center take a drastically long time to startup, especially AMD Catalyst.
No matter what you do as far as running this... it seems like the first zone keeps getting bigger and bigger?!
The first few restarts seem to take a little bit longer to get to login, which does get better the more restarts.

Questions:
-> I noticed that you have basically just flip-flopped zones 1 & 2 vs. default monthly system disk.
-> Your Zone 3 leaves no gap between boot optimization and program hints zone.
-> I had seen you reply to someone's script and say that putting the $MFT and stuff at the beginning of the disk is a step backwards in performance, your monthly script... puts them so much closer to the beginning of the disk than default?! Which seeing you have all system files set to a 2GB limit on the hard drive it feels like this was not in thoughts of any large disks?

So, I'm failing to see the benefits in your script vs. the defaults?

[Darlis]

*** I saw in another post that it is best to disable Windows default Defragmenter? Is that right and how come?

Different defragmentes have different optimization strategies. The XP defragmenter moves the boot files around the volume every 3 days and MyDefrag moves the back again. The Win7 defragmenter is compatible with most scripts, though.

do you not think that zone 5 could be split into 2 zones? Having all that Windows folder stuff so closely together with the Program Files seems unnecessary?

Why is that unnecessary?

Seems to take a rather unusually long time to boot

Prefetch works differently in Windows 7. When you move a file that is index by Prefetch, the index has to be recreated, usually when that file is needed again. Windows has fully optimized itself again after 5 reboots.

After defragging... All USB devices (except keyboard/mouse) must now be unplugged and plugged back in to find.

It's unlikely that this is a problem caused by my scripts or MyDefrag. Did you defrag them too or does this happen after defragmenting any volume?

Lots of drivers are not taken into consideration... I was scared of that seeing all my winsxs folders going to the end of program files.

Are you sure that these drivers are actually used? Windows keeps a lot of backups.

No matter what you do as far as running this... it seems like the first zone keeps getting bigger and bigger?!

The directories zone? Or do you mean zone 3 with the boot files?

-> I noticed that you have basically just flip-flopped zones 1 & 2 vs. default monthly system disk.

The directories zone is usually more stable in size than the NFTS system files zone. If the latter one changes in size, the first one would have to be shifted around (moving a lot of small files 'expensive')

-> Your Zone 3 leaves no gap between boot optimization and program hints zone.

That's indeed a bug. The VolumeSize * 0.. has gone missing somehow.  I'll fix that later.

-> I had seen you reply to someone's script and say that putting the $MFT and stuff at the beginning of the disk is a step backwards in performance, your monthly script... puts them so much closer to the beginning of the disk than default?!

Most people like like to see short boot times. And since these files are at the beginning of the volume (along with a lot of other program files) I've come to the conclusion that this is a good place. You can change the location of the first zones if you like, that's what the Variables script is for.

So, I'm failing to see the benefits in your script vs. the defaults?

Look what I've written in the first post. I've optimized them for faster optimizing without reducing performance. You can see the difference in my benchmarks.[/quote].

[peptobismal]

do you not think that zone 5 could be split into 2 zones? Having all that Windows folder stuff so closely together with the Program Files seems unnecessary?

Why is that unnecessary?

So ALL winsxs folder should be okay at the end of the drive? Nothing in that folder needs to be in a more accessible place at all? The only thing I see is that you instead of using VolumeFree use VolumeSize quite a bit, putting all the System Files at like 1MB? I mean I got a 500Gig, but that's not all my System Partition... C: Drive is only like 156MB, it just seems like everything at the beginning of the drive is sooo crammed and that organization isn't taking place too efficiently as far as drive accessibility?

After defragging... All USB devices (except keyboard/mouse) must now be unplugged and plugged back in to find.

It's unlikely that this is a problem caused by my scripts or MyDefrag. Did you defrag them too or does this happen after defragmenting any volume?

Well, my thoughts being that after using the default scripts, I boot up fine... USB's are ALL recognized, including memory card reader using the internal USB port. After using your script, I have to unplug USB keys and the memory card reader is not on; however, during BIOS load is fine. So, yes drivers somewhere are going where Windows can't find them.

Lots of drivers are not taken into consideration... I was scared of that seeing all my winsxs folders going to the end of program files.

Are you sure that these drivers are actually used? Windows keeps a lot of backups.

Pretty sure... AMD Catalyst took about 4 times longer than usual to start, USB drives unrecognized? Drivers are going somewhere that Windows is like what's going on here...?

No matter what you do as far as running this... it seems like the first zone keeps getting bigger and bigger?!

The directories zone? Or do you mean zone 3 with the boot files?

-> Your Zone 3 leaves no gap between boot optimization and program hints zone.

That's indeed a bug. The VolumeSize * 0.. has gone missing somehow.  I'll fix that later.

Yeah, I think these both are hand-in-hand issues.

-> I noticed that you have basically just flip-flopped zones 1 & 2 vs. default monthly system disk.

The directories zone is usually more stable in size than the NFTS system files zone. If the latter one changes in size, the first one would have to be shifted around (moving a lot of small files 'expensive')

Understood.

-> I had seen you reply to someone's script and say that putting the $MFT and stuff at the beginning of the disk is a step backwards in performance, your monthly script... puts them so much closer to the beginning of the disk than default?!

Most people like like to see short boot times. And since these files are at the beginning of the volume (along with a lot of other program files) I've come to the conclusion that this is a good place. You can change the location of the first zones if you like, that's what the Variables script is for.

Another thought is instead of right after the BootOptimization or where ever the Fonts are organized in... would it be beneficial for a monthly script since it should be the most lengthy anyways to FileSelect not(FileName("*.ttf") or FileName("*.font") or FileName("*.fnt"))?

So, we could maybe get those after the $MFT and all the ntfsFiles? They seem to be all bunched right at the beginning before these files... seems like leaving a bigger gap, and putting them afterwards with a gap before ProgramHints might be more optimized, rather than going through all these little font files to get to the necessities?

So, I'm failing to see the benefits in your script vs. the defaults?

Look what I've written in the first post. I've optimized them for faster optimizing without reducing performance. You can see the difference in my benchmarks

So basically this is for the speediness of the defrag and not necessarily a lengthy Windows FileSystem optimization? I just feel like there are a lot of files in the Windows directory that could have been placed more properly on the drive. Seeing Acrobat Reader and miscellaneous files (etc.) going right after $MFT can't bee the most proper placements... I dunno, I can't seem to type a proper Google search to find an informative lengthy Windows 7 FileSystem structuring/engineering article.

Not meaning to get on your bad side by any means, it just felt after looking at some things that they could be improved upon, but I would like to know for sure booting orders, driver orders (AMD & nVidia w/ programs necessary), and everything prior to writing a script... do you have any suggestions on reading material persay?

Also, based on your benchmarks this could be... not the best option for 7 as comparative to XP, correct?

Code:

  /* Zone 3: files used when booting, and a gap. */
  FileSelect
    ImportListFromBootOptimize()
  FileActions
    SortByImportSequence(Ascending)
    AddGap(RoundUp(ZoneEnd + maxGapSize, roundGapSize))
  FileEnd

----------------------
Change to end of Zone 5:

  FileActions
    SortByName(Ascending)
    AddGap(RoundUp(ZoneEnd + maxGapSize, roundGapSize))
  FileEnd

Code:

/* The minimum gap size between zones */
SetVariable(minGapSize, 32MB)

/* The maximum gap size between zones */
SetVariable(maxGapSize, 150MB)

/* The size to round a gap between zones to.
 * This is used to stabilize the position of a zone. */
SetVariable(roundGapSize, VolumeFree * 0.01)

So, far with these small changes... it looks like nothing that has been .dll or in the Windows folder has really gone to the end of the drive at all... Just things unused that I can tell, 70% done with Zone 5, but yeah not many .dll's or .sys files have gone to the end of the drive... lots of winsxs files have been bumped up to the lower end such as drivers, etc... but yeah... ALL of this folder and the majority was at the end of the drive. We'll see how it does after a boot up?

Possibly just a gapping issue in Windows 7 that caused everything and I mean EVERYTHING in winsxs folder to go to the end of the drive and it felt like the basic drivers were not close enough during boot?

Or maybe rather your values of VolumeSize * .01 on a 1TB roughly 10MB, 500Gigs roughly 5MB? 156 roughly 1.5MB, correct? So it can get pretty darn crammed...

Boot results:
- 1st/2nd - Everything boots fine... Catalyst long wait time to load.
- 3rd - USB key/Memory Card reader are unrecognized, Catalyst still long load, un-plug/plug-in USB key Windows recognizes asks to explore or readyboost
- 4th - USB key/Memory Card reader work again.
- 5th - Unrecognized... unplug/plug in still workaround, but memory card means to open the case.

In Windows 7, there isn’t a ‘dllcache’ folder and nor can you find the ‘i386′ folder, where the system caches (like in XP) all it’s source modules. It is this WinSxS folder which stores the shared components of side-by-side applications. These files can be multiple versions of the same assembly or application. Every side-by-side assembly has a unique identity. One of the attributes of the assembly identity is its version. “Side-by-side assemblies are used by the operating system as fundamental units of naming, binding, versioning, deployment, and configuration. The Winsxs folder includes all manifests, optional components and 3rd party Win32 files”.

So maybe in the zone that sorts the Program Files and winsxs and all the "regular files" it would be best to sortby LastAccessed? Just thoughts, because after realizing this is deemed XP optimized... well, I just know that something gets in a whacked order when using this on Windows 7, because this is the only times I have these problems at all, honestly.

Which sorting by Ascending may not technically affect XP negatively in anyway as "dllcache" and "i386" folders would be relatively close to the beginning of this zone.

It appears that certain "Registry HardDrive Performance Tweaks" were having a negative impact on the way my system was defragging, after setting LastAccess, 8Dot3, All Prefetch values, and many other FileSystem related registry values, it appears to be corrected. I am 100% sure of this as I realized that $Top should have NOT been moving with $MFT. So, my apologies for everything I have said!

And a word to anyone else looking for "optimizing" Windows "HardDrive" performance tweaks are mainly geared towards SSDs NOT HDDs! Be careful even with SSDs Windows has pretty much turned off Prefetch, SuperFetch, etc. for your optimal performance! HDDs are defaultly optimized in the registry, basic routine maintenance is all that should be upheld for optimal system and hard drive performance!

[Darlis]

So ALL winsxs folder should be okay at the end of the drive? Nothing in that folder needs to be in a more accessible place at all?

I trust Prefetch in this case. If some files are not indexed then they're unlikely to be used (regularly).

putting all the System Files at like 1MB? I mean I got a 500Gig, but that's not all my System Partition... C: Drive is only like 156MB, it just seems like everything at the beginning of the drive is sooo cram

Where do I do that?

Well, my thoughts being that after using the default scripts, I boot up fine... After using your script, I have to unplug USB keys and the memory card reader is not on

So you use the default scripts, restart and USB works. You use my scripts and do not restart and USB does not work? But it does work after a restart?
The test you haven't performed is: does USB still work directly after using the default scripts?

I still highly doubt that it has something to do with file movement. Could it be an energy saving feature perhaps?

Pretty sure... AMD Catalyst took about 4 times longer than usual to start

Like I said before: Prefetch has to rebuild it's index after a file has been moved. That usually happens the first time the program is started again. The next starts should be faster.

Another thought is instead of right after the BootOptimization or where ever the Fonts are organized in... would it be beneficial for a monthly script since it should be the most lengthy anyways to FileSelect not(FileName("*.ttf") or FileName("*.font") or FileName("*.fnt"))?

Used fonts are usually indexed by Preftech. But it seems to be a good idea to create an extra zone for them. I'll take a look at it.

So basically this is for the speediness of the defrag and not necessarily a lengthy Windows FileSystem optimization? I just feel like there are a lot of files in the Windows directory that could have been placed more properly on the drive. Seeing Acrobat Reader and miscellaneous files (etc.) going right after $MFT can't bee the most proper placements... I dunno, I can't seem to type a proper Google search to find an informative lengthy Windows 7 FileSystem structuring/engineering article.

You'll hardly find any official documentation about proper file placement. The whole optimization stuff is all because of the way mechanical HDDs work. Format an SSD with NFTS (or any other file system) and you can basically scrap your defragmenter.

The file placement itself is more or less random. Of course you can safely say that Adobe files are likely near the beginning of the disk but weather they end up before or after the MFT (or how close) highly depends on all other installed programs and how you've set up the script.

Not meaning to get on your bad side by any means, it just felt after looking at some things that they could be improved upon, but I would like to know for sure booting orders, driver orders (AMD & nVidia w/ programs necessary), and everything prior to writing a script... do you have any suggestions on reading material persay?

I'm always open for improvements. This article about Super/Prefetch may be interesting to you.

It's also the magic of Prefetch that determines which files are important and which are not. The AMD/nVidia folders mostly contains the installation files which you'll likely never need again, the driver files are copied to the Windows directory. So why place it at the fast end of the volume?

Also, based on your benchmarks this could be... not the best option for 7 as comparative to XP, correct?

Since Prefetch has been improved a lot in Windows 7 you likely won't get that much of a performance improvement compared to XP.

Or maybe rather your values of VolumeSize * .01 on a 1TB roughly 10MB, 500Gigs roughly 5MB? 156 roughly 1.5MB, correct? So it can get pretty darn crammed...

1% of 1TB is 10GB not 10MB.

Boot results:
- 1st/2nd - Everything boots fine... Catalyst long wait time to load.
- 3rd - USB key/Memory Card reader are unrecognized, Catalyst still long load, un-plug/plug-in USB key Windows recognizes asks to explore or readyboost
- 4th - USB key/Memory Card reader work again.
- 5th - Unrecognized... unplug/plug in still workaround, but memory card means to open the case.

This looks more like a hardware/software problem to me, not something that has been caused by my scripts or MyDefrag.

So maybe in the zone that sorts the Program Files and winsxs and all the "regular files" it would be best to sortby LastAccessed?

LastAccess is disabled by default since Vista. Also a lot of other programs like to change this value (virus scanners for example), so this is not very reliable.

It appears that certain "Registry HardDrive Performance Tweaks" were having a negative impact on the way my system was defragging, after setting LastAccess, 8Dot3, All Prefetch values, and many other FileSystem related registry values, it appears to be corrected. I am 100% sure of this as I realized that $Top should have NOT been moving with $MFT. So, my apologies for everything I have said!

You shouldn't temper with these values unless you have a very special system setup. The default values are usually the best ones to use. Did you ever measured a performance improvement after using such tools?
It is also possible that these tools have caused the problems with your USB. Especially TuneUp for example is known to be able to trash a system.
Btw. $Tops is one of the NTFS files that is supposed to be moved together with the MFT.

[peptobismal]

Hmmm... well, I had noticed when I did a search for $Tops on Google one of the main results was this FAQ saying that MyDefrag is unable to move it but Windows can clean it? So I set it to clean. 

But no I never allow any program to automatically "improve" performance. That's the only reason I knew exactly what had been changed is that I had the registry files I created.

[peptobismal]

One thing... I am using the Daily script right now... and the thing that I notice is that it feels like it does TOO much? Like the whole Zone 4 was a solid block, but it kept moving things from that and down to the now big block that is the boot and the beginning of my drive? Right now it is cramming at least half of the last zones back down? So, I don't get why... it seems to be just as "thorough" as the monthly? I would assume most of the solid blocks would stay solid? And not just cram or move down more and more into solid sections of the hard drive, right?

[Kasuha]

Default scripts contain some usage-related logic - if a file is long unused, it goes to spacehogs zone. It also follows windows prefetch recommendations. All in all, if you have a program which you use only rarely, it will move towards the end of the disk after about a month, then it will jump back to the beginning after as little as single use.

[Darlis]

Kasuha is right. Zone 3 and 4 take their information from the Prefetch files. The difference between these zones: The first zone selects it's files from the layout.ini, a list that has been generated by Prefetch for optimal file placement. It is generated about every 3 days. Zone 4 gets it's information directly from the Prefetch files which are updated every time a program starts.
That explains why files from zone 4 start to migrate to zone 3.

[Darlis]

I've updated the Scripts:

- Fixed the too small gap size by adding VolumeSize * 0.01 (also, gaps won't exceed a maximum of 1GB)
- Removed zone 4
- Fonts will get sorted in the Weekly and Monthly script

[RaizonDe]

Just tried this script out .. Noticeable performance boost. 

but the $MFT still remains fragmented and placed roughly around the backend of the disk :O .

Some game files and a mp4 files are unmovable too.

Are these a weird outcome ?

 

[Darlis]

but the $MFT still remains fragmented and placed roughly around the backend of the disk :O .

The MFT should be placed at 2GB unless you've changed the Variables script. Note that the beginning of the volume is displayed at the bottom of the diskmap, not at the top. If you've used the Monthly script then the MFT can fragment.

Some game files and a mp4 files are unmovable too.

It's likely that some other process had a lock on these files (virus scanner maybe?) or you don't have enough free continuous space to move the file to.

[Rohk]

If you've used the Monthly script then the MFT can fragment.

Why this may hapen only with the Monthly script? I don't see the difference between the file actions on monthly,weekly and daily scripts for the ntfs system files zone.

Also why do you use the reversed order for the mft and directories compared to the standard system disk scripts, is it for more stability?

I have also noticed that sometimes the ntfs system files zone is sorted even if the last run was few minutes ago. Does the "PlaceNtfsSystemFiles()" create wrap around fragmentation like the normal SortBy file actions?

Ps. Nice to see an update!