Display of work queue measurement

[elfring]

Your tool is slowly working on my biggest partition.

Code:

Commandline argument '-a' accepted, optimizemode = 1
JkDefrag v3.26
Date: 2007/09/23
NtfsDisableLastAccessUpdate is inactive, using LastAccessTime for SpaceHogs.
Starting JkDefrag for 'E:'
Phase 1: Analyze
Finished.
- Total disk space: 186828976128 bytes (174.00 gigabytes), 45612543 clusters
- Bytes per cluster: 4096 bytes
- Number of files: 777006
- Number of directories: 95918
- Total size of analyzed items: 179372638208 bytes, 43792148 clusters
- Number of fragmented items: 81, 0.01% of all items
- Total size of fragmented items: 3963031552 bytes, 967537 clusters, 2.21% of all items, 2.12% of disk
- Free disk space: 5190758400 bytes, 1267275 clusters, 2.78% of disk
- Number of gaps: 2513
- Number of small gaps: 690, 27.46% of all gaps
- Size of small gaps: 16875520 bytes, 4120 clusters, 0.33% of free disk space
- Number of big gaps: 1823 (72.54% of all gaps)
- Size of big gaps: 5173882880 bytes, 1263155 clusters, 99.67% of free disk space
- Average gap size: 504.29 clusters
- Biggest gap: 1066680320 bytes, 260420 clusters, 20.55% of free disk space

I am interested to see more details about the optimisation progress. I've got doubts about the performance. Is the Input/Output activity of around 9 MB that is shown by Process Explorer for the current defragmentation run fast enough?

[WHRoeder]

Commandline argument '-a' accepted, optimizemode = 1

I am interested to see more details about the optimisation progress.

-a 1 doesn't do any optimization, all it does is read the list of files on the disk to see what needs to be done.
Try -a 3

I've got doubts about the performance

As far as performance is concerned, all defraggers use the same API and should have similar results

[cf]

If you want to see the length of your disk queue or the disk IO load just run "perfmon.msc" and select the appropriate performance indicators.
I personally like the "time (%)" indicator for physical disks which shows disk load in percent (like CPU load).

[elfring]

-a 1 doesn't do any optimization, all it does is read the list of files on the disk to see what needs to be done.

I know - I've just provided some facts about my hardware allocation. The tool's processor usage goes up to 50 % on my system. The "Virtual Size" for the program "JkDefrag.exe" is 551956 K (Private Bytes: 331.9 MB) at the moment while the defragmentation is performed without further command line parameters.

As far as performance is concerned, all defraggers use the same API and should have similar results.

I guess that a profiler can show a few opportunities for process optimisation.
For example, I see the status display "Moving 31 clusters from 2964525 to 516785" for several minutes.

[elfring]

I personally like the "time (%)" indicator for physical disks which shows disk load in percent (like CPU load).

This value went up to 5 - 10 a few times, it is mostly under 3.

[jeroen]

I am interested to see more details about the optimisation progress.

See the "-d" commandline argument. Your log shows that your disk has a huge number of small files, so I am guessing that the bottleneck will be the CPU, not the disk I/O.

[elfring]

It is unlikely that the drive "Samsung SP2504C" and my processor "AMD Athlon 64 X2" can not efficiently cope with the defragmentation process.
I need the software "Berkeley Open Infrastructure for Network Computing" to keep it busy. :wink:

[elfring]

I'd like to give you a little statistics update from my software test.

Code:

22:32:38 Analyzing volume 'E:\'
22:32:38 Starting JkDefrag for 'E:\'
22:32:39 Phase 1: Analyze
22:34:57 Phase 2: Fixup
22:37:20 Zone 1: Fast Optimize
22:37:21 Zone 2: Fast Optimize
22:39:05 Zone 3: Fast Optimize
22:43:48 Finished.
22:43:48 - Total disk space: 186828976128 bytes (174.00 gigabytes), 45612543 clusters
22:43:48 - Bytes per cluster: 4096 bytes
22:43:48 - Number of files: 777037
22:43:48 - Number of directories: 95936
22:43:48 - Total size of analyzed items: 179376021504 bytes, 43792974 clusters
22:43:48 - Number of fragmented items: 1, 0.00% of all items
22:43:48 - Total size of fragmented items: 3932311552 bytes, 960037 clusters, 2.19% of all items, 2.10% of disk
22:43:48 - Free disk space: 4298379264 bytes, 1049409 clusters, 2.30% of disk
22:43:48 - Number of gaps: 34
22:43:48 - Number of small gaps: 10, 29.41% of all gaps
22:43:48 - Size of small gaps: 196608 bytes, 48 clusters, 0.00% of free disk space
22:43:48 - Number of big gaps: 24 (70.59% of all gaps)
22:43:48 - Size of big gaps: 4298182656 bytes, 1049361 clusters, 100.00% of free disk space
22:43:48 - Average gap size: 30864.97 clusters
22:43:48 - Biggest gap: 1868288000 bytes, 456125 clusters, 43.46% of free disk space
22:43:48 These items could not be moved:
22:43:48   Fragments       Bytes  Clusters Name
22:43:48           1    67108864     16384 E:\$LogFile
22:43:48          23      109208        27 E:\$MFT::$BITMAP
22:43:48           1   894615552    218412 E:\$MFT
22:43:48           1       17408         5 E:\Programme\T-DSL Speed Manager\Pcandis5.sys
22:43:48           1        4096         1 E:\$MFTMirr
22:43:48           1        4144         2 E:\.::$SECURITY_DESCRIPTOR
22:43:48           2     5701568      1392 E:\$Bitmap
22:43:48   --------- ----------- --------- -----
22:43:48          30   967560840    236223 Total
22:43:48 These items are still fragmented:
22:43:48   Fragments       Bytes  Clusters Name
22:43:48          23      109208        27 E:\$MFT::$BITMAP
22:43:48           2     5701568      1392 E:\$Bitmap
22:43:48           6  3932309504    960037 E:\geladen\openSUSE\openSUSE-10.2-GM-DVD-x86_64.iso
22:43:48   --------- ----------- --------- -----
22:43:48          31  3938120280    961456 Total

[elfring]

I saw in the tool "Process Explorer" that the disk usage decreased in speed while the memory allocation quickly grew up to 331.9 MB at the beginning of the optimisation process. Does it show the usual effects on analysis of big partitions?
Is there a measurable correlation between Input/Output activity and the conversion of the Master File Table and volume bitmap into internal data structures to feed the Windows Defragmentation API with file rearrangements?

[jeroen]

I'm sorry, but I don't understand your question. After the Analysis phase the memory usage should not grow any further.

[elfring]

I try to get an indication for the performance of the internal file tree/bitmap processing and the corresponding defragmentation speed on the partition.
The protocol is not displayed in a clear format in the main application window so far. (The detailed log output can only be seen with a delay.)
It becomes interesting when no progress can be noticed for a while.

[jeroen]

I try to get an indication for the performance of the internal file tree/bitmap processing and the corresponding defragmentation speed on the partition.

I have never done any profiling measurements, if that is what you mean. My impression is that the bottleneck is the Microsoft defragmentation API, not JkDefrag.

The protocol is not displayed in a clear format in the main application window so far. (The detailed log output can only be seen with a delay.) It becomes interesting when no progress can be noticed for a while.

I don't think detailed messages about the "protocol" should be displayed on the main application window. When you see no progress then the Windows defragmentation api is busy with a file. The api has no callbacks and there is no way for JkDefrag to know how far a filemove has progressed.

[elfring]

I guess that it helps a little bit to see from the status display in the GUI when a lengthy move operation was started.

[jeroen]

I guess that it helps a little bit to see from the status display in the GUI when a lengthy move operation was started.

See the "Moving NNNN clusters" message.

[elfring]

Would you like to add a timestamp to the message in the GUI?