Esprit's mods

[Esprit]

I don't know where is the bottleneck but I had a 700MB movie compressed in XviD which was on 9427 fragments (!). And there wasn't any troubleshooting seeing it with any video player I used to.

Ordinary film takes 2 hours which means here a data rate about 100 kB/s but your disk can handle 50 MB/s. So fragments are not a problem but try to copy film and you'll see the data rate is only few MB/s, which is far away from 50 MB/s.

In the other hand, I had a 3GB VMWare image (.vmdk) on 12590 fragments, and it become impossible to run VMWare (after more than two hours torturing my HD I had no choice than physicaly shuting off my computer as my keyboard, my mouse and any programs became unusable)

I suggest to keep fragments over 20 MB so you should have no more than 150 fragments. Another problem could be distance of neighborhood fragments which should be minimal.

[jeroen]

I don't know where is the bottleneck but....

I think your case proves the point which I was trying to make, "the formula cannot be applied universally to all files. It's all right for some files, but not for other files."

[Esprit]

I proudly present new modification of JkDefrag v3.34. This is pointed to fragments. As I mentioned before fragments are not needed to be larger than 5000 clusters. So I programmed version which defragments only fragments that are smaller than 5000 clusters. If next fragment is very big (>10000 clusters) then it won't merge current fragment with it but it steals some clusters from big fragment. This prevents moving of very big clusters. Now the link

http://homel.vsb.cz/~sru048/JkDefrag%20mod%20by%20Esprit/JkDefrag%20v3.34.E.f.2.exe

I tested this release so it should work but definitely it won't hurt your files due to use of WinAPI. There is a little bug: files that are defragmented are still colored yellow. All fragments are bigger than 5000 but decision about fragmentation is based on number of fragments. Don't worry about that.

There is great improvement for large file (video) disk comparing to JkDefrag. Classic defragmentors will lower the number of fragments when some gap is available. So after first run specially after fast optimize big gap is created at the end. Second run will cause moving large fragments to defragment all files. But my mod will be much faster at second run because all previously defragmented files will have fragments bigger than 5000 cl. so now they won't be processed.

I created function ReorganizeFragments which deletes virtual fragments from list of fragments and also treats all neighborhood fragments as one. This greatly simplify some implementations and lowers memory usage (see little lowering during defrag.). To lower memory usage at start all functions have to be rewritten so I didn't take risk that something could go wrong (still this could be implemented to version 4).

Next time I'll try to change FixUp phase because it still defragments files that have big fragments. It should move only fragments. Also I'll try to make moving of clusters in parallel which should greatly improve moving speed in some cases.

[jeroen]

I proudly present new modification of JkDefrag v3.34.

Thank you very much for your contribution! In v4 of JkDefrag I will make something similar to what you have done, but as an option, not as the default. I think most people will expect a defragmenter to defragment all files, not just small files.

I created function ReorganizeFragments which deletes virtual fragments from list of fragments and also treats all neighborhood fragments as one.

JkDefrag has 2 methods of moving a file, fast and slow. If the fast method fails it will try the slow method. The slow method moves every fragment one at a time. By joining fragments in the list the slow method is rendered useless. The result is that JkDefrag cannot move many files.

[Esprit]

I created function ReorganizeFragments which deletes virtual fragments from list of fragments and also treats all neighborhood fragments as one.

JkDefrag has 2 methods of moving a file, fast and slow. If the fast method fails it will try the slow method. The slow method moves every fragment one at a time. By joining fragments in the list the slow method is rendered useless. The result is that JkDefrag cannot move many files.

So I'll try to remove only virtual fragments and we'll see if it works. This could lower memory usage which will help defragment disk servers where are a lot of files and little RAM. Another possibility is to store compacted fragments and just before slow method the GetFragments would be called.

[jeroen]

So I'll try to remove only virtual fragments and we'll see if it works.

The first versions of JkDefrag used a "compressed" fragment list such as you are proposing, but later on I found that I needed the full information. The virtual fragments are used for calculations in many places in JkDefrag. They are mainly used by compressed files.

[Esprit]

This release was removed due to possible problem with virtual fragments.

  Release of another version. This time it fixes coloring of files and lowers memory usage (in my case to 87%). Next release would fix fixup phase which defragments defragmented files which is quite slow.
  Function ReorganizeFragments where renamed to CompactFragments and new function RemoveVirtualFragments was created. This lower memory usage. I hope that RemoveVirtualFragments would be accepted to original version of JkDefrag and the rest of the code won't work with virtual fragments any more. Then the code will be smaller and better readable.
  I also overloaded function IsFragmented which is crucial in this approach. It tests fragmentation only for whole file and is much simpler than original function. Maybe it would be better to create overloaded function in original version due to better reading and understandig the code as well as little speedup.

[jeroen]

I hope that RemoveVirtualFragments would be accepted to original version of JkDefrag and the rest of the code won't work with virtual fragments any more.

Thank you very much for your contribution, I appreciate it. Your idea of storing a compacted fragment list in memory, and fetch the full list from Windows when it is needed, is a good idea and I have put it on the list for v4 of JkDefrag.

[RJARRRPCGP]

after more than two hours torturing my HD I had no choice than physicaly shuting off my computer as my keyboard, my mouse and any programs became unusable)

LOL that reminds me of a 486 back when we still had Windows 3.1 and when Smartdrv.exe wasn't loaded.