Low impact server script

[dtfinch]

I inherited some servers (running Exchange and SQL Server) that were badly fragmented. Databases in thousands of fragments. Backup files in hundreds of thousands of fragments. Attempts to defrag the servers would lead to immediate user complaints about slowness and timeouts. Early attempts even did more harm than good, as mydefrag would vacate a large database file to make room for itself, then fail halfway, leaving the file much more fragmented than when it began.

I tried to make a script to run periodically that would keep our servers from reaching that level of fragmentation again, without causing noticeable slowdowns or lockups while it's running.

Code:

# Tailored for database servers. In our case Exchange 2003 and SQL Server 2005.

# For any moves that may affect open database files, it slows down to 50% and reduces chunk move size to 32mb.
#   32mb because defrag seemed to move files at only about 4mb per second in my tests.
#   A lockup of more than 8 seconds (32/4) seems like way too long, but chunk sizes below 32mb seem way too small.

# Not very concerned with file order and position on the disk.
# Just keeping big files mostly defragmented, without database lockups, and keeping smaller files out of the way.

WhenFinished(exit)
OtherInstances(exit)

Title('Light Server Defrag')
Description("Low impact defrag for database servers")

SlowDown(90)
MaxRunTime(4 hours)

ExcludeFiles((Fragmented(no) and Size(128MB,0)) or AverageFragmentSize(256MB,0)) #ignore large, mostly unfragmented files

SetVariable(CRLF,"
")

WriteLogfile("MyDefragServer.log","LogHeader")


# System disk
VolumeSelect
Name("c:")
VolumeActions
AppendLogfile("MyDefragServer.log","LogBefore")

# Less concerned about vacating on the C: drive, since we shouldn't have any major databases here.

MakeGap(RoundUp(VolumeUsed * 0.3,VolumeSize * 0.05), DoNotVacate)

FileSelect
SelectNtfsSystemFiles(yes)
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in ntfs zone.!CRLF!")

PlaceNtfsSystemFiles(Ascending SkipBlock(4,16MB), MftSize * 0.1)
FileEnd

FileSelect
Directory(yes)
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in directory zone.!CRLF!")

SortByName(Ascending SkipBlock(8,128KB))
FileEnd

MakeGap(0, DoNotVacate) #the donotvacate is probably unnecessary, since 0 is never forward

FileSelect
ImportListFromBootOptimize() and (FileName("*.exe") or FileName("*.dll") or FileName("*.sys"))
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in boot zone.!CRLF!")

SortByImportSequence(Ascending SkipBlock(4,16MB))
FileEnd

MakeGap(0, DoNotVacate)

FileSelect
# Database files. Don't really belong on C:, but sometimes they're there
FileName("*.mdf") or FileName("*.ldf") or FileName("*.edb") or FileName("*.stm")
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in database zone.!CRLF!")

# Smaller chunk size to prevent lockups, at cost of some MFT bloat
SlowDown(50)
FileMoveChunkSize(32MB)

Defragment(ChunkSize(128))
FileEnd

MakeGap(0, DoNotVacate) #reset zone cursor.

FileSelect
# Defrag large non-database files but don't bother relocating
Size(64MB,0)
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in big zone.!CRLF!")

SlowDown(90)
FileMoveChunkSize(1024MB) # default

Defragment(ChunkSize(128))
FileEnd

MakeGap(0, DoNotVacate) #reset zone cursor

FileSelect
All
# Everything left is small, non-database files to move out of the way
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in remaining zone.!CRLF!")

SlowDown(90)
FileMoveChunkSize(1024MB) # default

Defragment(Fast)
FastFill() #FastFill only acceptable in last zone. Otherwise it can lock/vacate database files.
FileEnd

AppendLogfile("MyDefragServer.log","LogAfter") 
VolumeEnd


# Data disks
VolumeSelect
Removable(no)
and Writable(yes)
and Mounted(yes)
VolumeActions
AppendLogfile("MyDefragServer.log","LogBefore")

# Skip the system and boot optimization, since they vacate, and vacating part of a very large file could trigger
# a full move, which often seems to get halfway and then fail, leaving the file in worse shape than before.

MakeGap(0, DoNotVacate)

FileSelect
# Select all database files. Even if not fragmented, we don't want them caught up in the FastFill later.
FileName("*.mdf") or FileName("*.ldf") or FileName("*.edb") or FileName("*.stm")
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in database zone.!CRLF!")

# Smaller chunk size to prevent lockups, at cost of some MFT bloat
SlowDown(50)
FileMoveChunkSize(32MB)

Defragment(ChunkSize(128)) # ChunkSize does not take a suffix. Always megabytes.
FileEnd

MakeGap(0, DoNotVacate) #reset zone cursor

FileSelect
# Defrag large non-database files but don't bother relocating
Size(64MB,0)
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in big zone.!CRLF!")

SlowDown(90)
FileMoveChunkSize(1024MB) # default

Defragment(ChunkSize(128))
FileEnd

MakeGap(0, DoNotVacate) #reset zone cursor

FileSelect
All
# Everything left is small, non-database files to move out of the way
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in remaining zone.!CRLF!")

SlowDown(90)
FileMoveChunkSize(1024MB) # default

Defragment(Fast)
FastFill() # FastFill only in last All zone. Otherwise it can vacate files.
FileEnd


AppendLogfile("MyDefragServer.log","LogAfter")
VolumeEnd

AppendLogfile("MyDefragServer.log","LogFooter")

[dtfinch]

I revised it a bit, making it smaller and hopefully a little better. I figure that by moving the database and big file zones to the top of the script, I can avoid the risk them being vacated by the ntfs and boot optimizations, and safely handle both system and data disks with one volume section. I also split the final fastfill zone into two zones, to keep small, static files closer to the beginning of the disk, and added some gaps.

Code:

# Tailored for database servers. In our case Exchange 2003 and SQL Server 2005.

# For any moves that may affect open database files, it slows down to 50% and reduces chunk move size to 32mb.
#   32mb because defrag seemed to move files at only about 4mb per second in my tests.
#   A lockup of more than 8 seconds (32/4) seems like way too long, but chunk sizes below 32mb seem way too small.

# Not very concerned with file order and position on the disk.
# Just keeping big files mostly defragmented, without database lockups, and keeping smaller files out of the way.

WhenFinished(exit)
OtherInstances(exit)

Title('Light Server Defrag')
Description("Low impact defrag for database servers")

SlowDown(90)
MaxRunTime(4 hours)

ExcludeFiles((Fragmented(no) and Size(128MB,0)) or AverageFragmentSize(256MB,0)) #ignore large, mostly unfragmented files

SetVariable(CRLF,"
")

WriteLogfile("MyDefragServer.log","LogHeader")


# System disk
VolumeSelect
Removable(no)
and Writable(yes)
and Mounted(yes)
VolumeActions
AppendLogfile("MyDefragServer.log","LogBefore")

MakeGap(VolumeSize * 0.05, DoNotVacate)

FileSelect
# Database files. Don't really belong on C:, but sometimes they're there
FileName("*.mdf") or FileName("*.ldf") or FileName("*.edb") or FileName("*.stm")
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in database zone.!CRLF!")

# Smaller chunk size to prevent lockups, at cost of some MFT bloat
SlowDown(50)
FileMoveChunkSize(32MB)

Defragment(ChunkSize(128))

SlowDown(90)
FileMoveChunkSize(1024MB) # reset to default

FileEnd

MakeGap(VolumeSize * 0.05, DoNotVacate) #reset zone cursor.

FileSelect
# Defrag large non-database files but don't bother relocating
Size(64MB,0) and SelectNtfsSystemFiles(no)
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in big zone.!CRLF!")

Defragment(ChunkSize(128))
FileEnd

#I'm hoping this will not vacate the previous zones, but just work around them
MakeGap(RoundUp(VolumeUsed * 0.3,VolumeSize * 0.05), DoNotVacate)

FileSelect
SelectNtfsSystemFiles(yes)
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in ntfs zone.!CRLF!")

PlaceNtfsSystemFiles(Ascending SkipBlock(4,16MB), MftSize * 0.1)
FileEnd

FileSelect
Directory(yes)
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in directory zone.!CRLF!")

SortByName(Ascending SkipBlock(8,128KB))
FileEnd

MakeGap(0, DoNotVacate) #the donotvacate is probably unnecessary, since 0 is never forward

FileSelect
ImportListFromBootOptimize() and (FileName("*.exe") or FileName("*.dll") or FileName("*.sys"))
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in boot zone.!CRLF!")

SortByImportSequence(Ascending SkipBlock(4,16MB))

AddGap(0.002*VolumeSize, DoNotVacate)
FileEnd


FileSelect
# Pack small, rarely changing files towards beginning of disk
LastChange(,30 days ago) and CreationDate(,30 days ago) and Size(0,1MB)
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in small, static zone.!CRLF!")

Defragment(Fast)
FastFill() #This will vacate a bit, but probably not too much

AddGap(0.002*VolumeSize, DoNotVacate)
FileEnd

FileSelect
All
# Pack the rest on top
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in medium sized or dynamic zone.!CRLF!")

Defragment(Fast)
FastFill() #FastFill only acceptable in last zone. Otherwise it can lock/vacate database files.
FileEnd

AppendLogfile("MyDefragServer.log","LogAfter") 
VolumeEnd

AppendLogfile("MyDefragServer.log","LogFooter")

[Darlis]

The problem with running servers is that the database files can be write-locked at any time. By using a very small chunk size you can fragment files very easily.
I would suggest to run a thorough optimization script when the servers are down (like overnight or on weekends) once. After that only use Defragment(Chunksize(128)) on the database files. This will leave the first part where it is, only moving the last part(s).

If the Windows auto-optimization is enabled you should disable it. You don't know when it will start (according to Murphy it will only run when you need the performance ).

#I'm hoping this will not vacate the previous zones

No, MyDefrag won't touch any files that have been selected in previous zones.

[matthelm]

If you have enough room on the drives, you could also dedicate "zones" on the disk for each data base.

I use this command:

  MakeGap(VolumeSize * 0.003, DoNotVacate)  // Change the 0.003 to what you need for each zone.

to move the start of a zone.  This keeps the files in one area, giving a slightly better chance of not moving files that don't need to be moved.  I do my movie files this way.  If I add a movie to one zone, only that one needs to be defrag.  Of course you won't get max speed for all files, but it sounds like that isn't your problem.

I'd also suggest more drive if you can, and spread the databases over more drives, but at a guess you've already looked at that part.

I think the "data" part of my defrag script now has 18 zones, but that is for 2 drives.  It can also get hard to keep track of your "zones" on more than 1 drive.   

[dtfinch]

Watching it in action, spending a long time moving small files to the start of the disk, and noticing that my date filter will get a bunch of them vacated whenever I install updates, I feel like I went overboard with the last couple zones, and for long-running database servers I don't really care if all the ntfs system files, directories, or boot prefetch are grouped and sorted. This one should accomplish the original goals much quicker. It does try a little to keep ntfs system files and directories mostly together, but shouldn't spend much time on it.

Code:

# Tailored for database servers. In our case Exchange 2003 and SQL Server 2005.

# For any moves that may affect open database files, it slows down to 50% and reduces chunk move size to 32mb.
#   32mb because defrag seemed to move files at only about 4mb per second in my tests.
#   A lockup of more than 8 seconds (32/4) seems like way too long, but chunk sizes below 32mb seem way too small.

# Not very concerned with file order and position on the disk.
# Just keeping big files mostly defragmented, without database lockups, and keeping smaller files out of the way.

WhenFinished(exit)
OtherInstances(exit)

Title('Light Server Defrag')
Description("Low impact defrag for database servers")

SlowDown(90)
MaxRunTime(4 hours)

ExcludeFiles((Fragmented(no) and Size(128MB,0)) or AverageFragmentSize(256MB,0)) #ignore large, mostly unfragmented files

SetVariable(CRLF,"
")

WriteLogfile("MyDefragServer.log","LogHeader")


# System disk
VolumeSelect
Removable(no)
and Writable(yes)
and Mounted(yes)
VolumeActions
AppendLogfile("MyDefragServer.log","LogBefore")

MakeGap(VolumeSize * 0.1, DoNotVacate)

FileSelect
# Database files. Don't really belong on C:, but sometimes they're there
FileName("*.mdf") or FileName("*.ldf") or FileName("*.edb") or FileName("*.stm")
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in database zone.!CRLF!")

# Smaller chunk size to prevent lockups, at cost of some MFT bloat
SlowDown(50)
FileMoveChunkSize(32MB)

Defragment(ChunkSize(128))

SlowDown(90)
FileMoveChunkSize(1024MB) # reset to default

FileEnd

MakeGap(VolumeSize * 0.1, DoNotVacate) #avoid putting big files in the beginning of the the disk, not that it's likely to happen otherwise

FileSelect
# Defrag large non-database files but don't bother relocating
Size(128MB,0) and SelectNtfsSystemFiles(no) and Directory(no)
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in big zone.!CRLF!")

Defragment(ChunkSize(128))
FileEnd


#try to clump ntfs metadata together, though some will fall below the zone untouched. Don't care about order.
MakeGap(VolumeSize * 0.25, DoNotVacate)
FileSelect
SelectNtfsSystemFiles(yes) or Directory(yes)
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in ntfs zone.!CRLF!")

Defragment(ChunkSize(128))
FastFill()
FileEnd


MakeGap(0, DoNotVacate)

FileSelect
All
# Pack the rest on top
FileActions
AppendLogfile("MyDefragServer.log","!ZONE220N! items (!ZONE222N! bytes) in remaining zone.!CRLF!")

Defragment(Fast)
FastFill() #FastFill only acceptable in last zone. Otherwise it can lock/vacate database files.
FileEnd

AppendLogfile("MyDefragServer.log","LogAfter") 
VolumeEnd

AppendLogfile("MyDefragServer.log","LogFooter")