Defraging Full Large Volume

[zcolton]

Here's the situation:
I have a win server that runs a video streaming server. It has multiple volumes ranging in size from 850 GB to 1.7 TB. The content is stored in a sing directory on the volumes. As content is ingested into the server, the files become very fragmented. So the problem I was facing was "How can I defrag and sort the files by size on a full volume." So i wrote a little vbscript to take care of that. The procedure does require a temp storage area to move files to that has enough free space to hold 55% of the total space on the volume to be defraged. Here is the script:

Code:

' Defrag full volume, order files from smallest to largest
' Single content folder

' Set user defined variables
strSourceDrive = "Q:"
strSourcePath = "\Content\"
strTempPath = "I:\"
strJkDefrag = "C:\JKDefrag\JkDefrag.exe"

Set objFSO = CreateObject("Scripting.FileSystemObject")
Set DataList = CreateObject("ADODB.Recordset")
DataList.Fields.Append "strFileName", 200, 255
DataList.Fields.Append "strFileSize", 20
DataList.Open

' Move files to temp location until 55 percent free space on source starting with the smallest files
Set objSourceLocation = objFSO.GetFolder(strSourceDrive & strSourcePath)
For Each objFile In objSourceLocation.Files
 DataList.AddNew
 DataList("strFileName") = objFile.Name
 DataList("strFileSize") = int(objFile.Size)
 DataList.Update
Next
Set objSourceLocation = Nothing
DataList.Sort = "strFileSize ASC,strFileName"
DataList.MoveFirst
Set objSourceDrive = objFSO.GetDrive(objFSO.GetDriveName(strSourceDrive))
Do Until int((objSourceDrive.FreeSpace/objSourceDrive.TotalSize)*100) > 54
 strFileName = DataList.Fields.Item("strFileName")
 objFSO.MoveFile strSourceDrive & strSourcePath & strFileName, strTempPath
 DataList.MoveNext
Loop
Set objSourceDrive = Nothing
DataList.Close
Set DataList = Nothing

' Force all remaining files to front of drive
Set WshShell = WScript.CreateObject("WScript.Shell")
Return = WshShell.Run(strJkDefrag & " -u DisableDefaults -f 0 -a 5 -q " & strSourceDrive, 1, true)

' Defrag and send remaining files to end of drive starting with the largest file
Set objSourceLocation = objFSO.GetFolder(strSourceDrive & strSourcePath)
Set DataList = CreateObject("ADODB.Recordset")
DataList.Fields.Append "strFileName", 200, 255
DataList.Fields.Append "strFileSize", 20
DataList.Open
For Each objFile In objSourceLocation.Files
 DataList.AddNew
 DataList("strFileName") = objFile.Name
 DataList("strFileSize") = int(objFile.Size)
 DataList.Update
Next
Set objSourceLocation = Nothing
DataList.Sort = "strFileSize DESC,strFileName"
DataList.MoveFirst
Do Until DataList.EOF
 strFileName = DataList.Fields.Item("strFileName")
 Return = WshShell.Run(strJkDefrag & " -u DisableDefaults -f 0 -a 6 -q " & strSourceDrive & strSourcePath & strFileName, 1, true)
 DataList.MoveNext
Loop
DataList.Close
Set DataList = Nothing

' Retrieve files from temp area, one at a time, largest first, defrag and send to end of drive
Set objTempLocation = objFSO.GetFolder(strTempPath)
Set DataList = CreateObject("ADODB.Recordset")
DataList.Fields.Append "strFileName", 200, 255
DataList.Fields.Append "strFileSize", 20
DataList.Open
For Each objFile In objTempLocation.Files
 DataList.AddNew
 DataList("strFileName") = objFile.Name
 DataList("strFileSize") = int(objFile.Size)
 DataList.Update
Next
Set objTempLocation = Nothing
DataList.Sort = "strFileSize DESC,strFileName"
DataList.MoveFirst
Do Until DataList.EOF
 strFileName = DataList.Fields.Item("strFileName")
 objFSO.MoveFile strTempPath & strFileName, strSourceDrive & strSourcePath
 Return = WshShell.Run(strJkDefrag & " -u DisableDefaults -f 0 -a 5 -q " & strSourceDrive & strSourcePath & strFileName, 1, true)
 Return = WshShell.Run(strJkDefrag & " -u DisableDefaults -f 0 -a 6 -q " & strSourceDrive & strSourcePath & strFileName, 1, true)
 DataList.MoveNext
Loop
DataList.Close
Set DataList = Nothing
Set WshShell = Nothing
Set objFSO = Nothing

I welcome any and all comments. I hope this examble would be useful to others out there.

[jeroen]

Thank you very much for your contribution, I appreciate it! But I do not quite understand what your script is supposed to do. To "defrag and sort the files by size on a full volume" all you have to do is call JkDefrag with the "-a 8" option. What does your script do differently?

[zcolton]

Doesn't jkdefrag need available continuous free space to defrag a file? In my situation, the volume is completely full. I'm working with files that can easily be 6 to 10 GB.

[jeroen]

Doesn't jkdefrag need available continuous free space to defrag a file? In my situation, the volume is completely full. I'm working with files that can easily be 6 to 10 GB.

To completely defragment a file JkDefrag needs a gap big enough to hold that file. If there is no gap big enough then JkDefrag will reduce the number of fragments by using the largest gaps available. For huge files such as yours it is not very important to completely defragment all files, a few fragments are acceptable and will not cause any significant performance loss.

[zcolton]

To completely defragment a file JkDefrag needs a gap big enough to hold that file.

The proccess the script runs through makes sure that there is enough space for every file to be defragmented. The files on these volumes are never modified. When the volume is full, I run this script, and I never need to defrag the volume again.

[Henry]

I got a 11 GB volume, 2 4GB files and a few small files

For some reason JKDefrag -a 7 / 8 just doesn't do all of its routines : First time ran : fragmented the files into more than 13 fragments, and then started writing clusters.  Stopped when it reached a file cluster, it didn't move it away even though I saw black space.  Second time ran: Redid the whole thing, except that it moved a bit more data.

I think the VBScript command might be a workaround, the temp files are stored on another volume while defragging.

[jeroen]

fragmented the files into more than 13 fragments

The "-a 7" and "-a 8" create fragments because they wrap big files around unmovable files. This is by design, it is not a bug. For more information see the JkDefrag homepage. Try "-a 2" if you want to fully defragment your files.

I think the VBScript command might be a workaround, the temp files are stored on another volume while defragging.

Be my guest, but I think that is potentially dangerous, in case of a crash or for files that are locked. Also, if the other volume happens to be on the same disk then it will be very slow.