Um, don't save a list of unmovable files...

[Falcon4]

Seriously, why is a list of unmovable files maintained anyway? All that does is make it so MyDefrag never even attempts a file that it encountered an error moving on the previous runs, typically because the file was deleted while MyDefrag was running. So you end up with a long (I've seen 2MB) list of files that MyDefrag is now programmed to ignore.

Erroring out on a file is an invisible process compared to the rest of the heavy duty, CPU intensive processing it does. It would be much more beneficial to just let MyDefrag mark a file as unmovable each time, than to maintain a list of files it thinks are always going to be locked...

At the very least, can it be possible to have a script command that deletes (or better: switches off the processing/reading/updating of) the mydefrag.dat file?

Just my two cents. I keep thinking of this every time I see a red spot on the disk map (although I typically run it minimized since the GUI is so CPU intensive). It's still only a minor suggestion since MyDefrag already works quite well otherwise

[jonib]

MyDefrag keeps a list of unmovable files so it can test if they are unmovable the next time, before doing any other processing, so if it finds a file on the list that is movable it will be removed from that list, and processed normally.

I just did a test, there ware a bunch of kaspersky files in the unmovable list, so I changed some setting that ware preventing MyDefrag to move them and now they are not in the unmovable list anymore.

Knowing what files are unmovable right at the start makes the zone calculations more effective.

jonib

[jeroen]

Jonib is right, upon loading the list of unmovable files MyDefrag will check each and every file to see if it is indeed unmovable. The listed files are not assumed to be permanently unmovable. Having beforhand knowledge about unmovable files is a big timesaver, it makes many operations a lot more efficient and in some cases can dramatically reduce the amount of data movements.

[usch]

Having beforhand knowledge about unmovable files is a big timesaver

I'm not sure how much time it really saves. I ran the following test on one of my PCs.

Start scenario: Drive is already defragmented, except for a bunch of unmovable files. Ideally MyDefrag should analyze the drive, find that there is nothing left to do, and exit. MyDefrag Debug level is set to -d 4 on the command line.

Test 1: No MyDefrag.dat present
Time passed between "Analyzing the commandline" and "Finished executing the script": 31 minutes.

Test 2: Restart with the same parameters, using the MyDefrag.dat file created in Test 1
Time passed between "Loading list of unmovable files" and "Finished loading list of unmovable items": 27 minutes.
Time passed between "Analyzing the commandline" and "Finished executing the script": 35 minutes.

So the defragmentation by itself was in fact faster, but since it took 27 minutes to load the list alone (trying to open 1800 files to see if they were still unmoveable), the overall running time was actually 4 minutes slower than without the predefined unmovable list.

[jeroen]

Which MyDefrag version? There was a change in v4.1.1 that made loading the list of unmovable files a lot faster. Also, it is not very common to have 1800 unmovable files. Are you running MyDefrag in a userid that has full administrator privileges?

[usch]

Which MyDefrag version? There was a change in v4.1.1 that made loading the list of unmovable files a lot faster.

I did the test with 4.1; if you say that there was a significant change I will try it again with 4.1.1.

Also, it is not very common to have 1800 unmovable files. Are you running MyDefrag in a userid that has full administrator privileges?

An EFS encrypted user profile. And yes, I did that from an administrator account. Half of the problem will probably go away with the announced update when MyDefrag can move encrypted files again, but 900× $EFS:$LOGGED_UTILITY_STREAM will remain unmovable.

[usch]

Ok, I re-ran the test now with v4.1.1. Here's the result:

Test 1: No MyDefrag.dat present
Time passed between "Analyzing the commandline" and "Finished executing the script": 28 minutes.

Test 2: Restart with the same parameters, using the MyDefrag.dat file created in Test 1
Time passed between "Loading list of unmovable files" and "Finished loading list of unmovable items": 27 minutes.
Time passed between "Analyzing the commandline" and "Finished executing the script": 29 minutes.

So there has been an increase in speed (analyzing the remainder of the disk after MyDefrag.dat was loaded was almost 4 times faster), but loading the list took exactly the same time as before.

But I think we can leave it at that for now, and wait how the next version performs.

[cg]

An EFS encrypted user profile. And yes, I did that from an administrator account. Half of the problem will probably go away with the announced update when MyDefrag can move encrypted files again, but 900× $EFS:$LOGGED_UTILITY_STREAM will remain unmovable.

In another post, jeroen said the next version will use a different method for opening files.  When that happens the EFS issue should go away.

[usch]

Another test run now with v4.1.2.

Test 1: No MyDefrag.dat present
Time passed between "Analyzing the commandline" and "Finished executing the script": 3:42 minutes.

Test 2: Restart with the same parameters, using the MyDefrag.dat file created in Test 1
Time passed between "Loading list of unmovable files" and "Finished loading list of unmovable items": 1:30 minutes.
Time passed between "Analyzing the commandline" and "Finished executing the script": 4:09 minutes.

The overall time went down from more than 30 minutes in v4.1.0 to about 4 minutes now. Not bad. Apparently the program was not only unable to move the encrypted files, but downright choked on them. The MyDefrag.dat file now contains "only" some 700 unmovable files, most of them $EFS:$LOGGED_UTILITY_STREAM items, but these seem to be handled reasonably fast.

Note that the run using the existing MyDefrag.dat file was still slower than a fresh start. However, when I repeated the test with debug logging turned off the second run was a bit faster than the first, so it might depend on the circumstances.

[jeroen]

The MyDefrag.dat file now contains "only" some 700 unmovable files, most of them $EFS:$LOGGED_UTILITY_STREAM items, but these seem to be handled reasonably fast.

Yes. I have it on my wishlist to do something about that, perhaps in the next version.

with debug logging turned off

Writing the debug logfile can seriously slow the program down, depending on the debug level.