FastFillUp?

[Kasuha]

The problem with FastFill command is, it moves small files towards the start of the zone. If a zone is only maintained with that command and not occasionally sorted, the lack of small files slowly progresses towards the beginning of the zone, either leaving growing amount of gaps at the end of the zone (when using FastFill()) or increasing the amount of file shuffling over time (when using FastFill(WithShuffling)). In my particular case, now the shuffling in my "small spacehogs" zone starts as early as at about 60% of the zone and the amount of shuffling is rather heavy.

My suggestion is an attempt to at least partially reverse the process of drifting small files towards the zone start. FastFillUp command would work exactly like FastFill command, just upwards, i.e. filling any gaps from the end of the zone with files from the start of the zone. Note that the file order in which they would be selected for filling would need to be reversed too so that it prefers files lying closer to the beginning of the disk. The sequence I have on mind is:

FastFill()
FastFillUp(WithShuffling)
FastFill(WithShuffling)

The first command would "stuff" the zone, possibly leaving gaps in the end.
The second command would move some files backwards (but in general less and different files than which were moved by the first command), filling up the gaps left from first command and at a price of some shuffling creating just one relatively large gap at the start of the zone.
The third command would fill the one remaining gap with hopefully minimal file moves. This step can eventually be removed, leaving the gap in the beginning as part of the pre-zone gap.

I believe this sequence would in the long run require less file moves to keep the zone "stuffed" than just FastFill(WithShuffling).

[jeroen]

Thanks for sharing your idea, I appreciate it. Yes, FastFill will slowly over time migrate the small files to the beginning of a zone. This is because small files have a higher chance of fitting into a new gap than large files. But I think this will also happen with your proposed FastFillUp? Except that small files will tend to migrate to the end of the zone. The majority of files are small files, and especially frequently accessed files such as configuration files are small. I think it is better to have them migrate to the beginning of a zone, where the disk is faster, than to the end.

[Kasuha]

Jeroen, you're saying that small files would start migrating towards the end of the zone - can you provide an explanation why should it happen if used the way I described ... or have you just ridiculed FastFillUp by taking it out of context of my suggestion?

Most of existing commands can be ridiculed this way, there is only tiny fraction of all possible FileActions sequences which make sense, the rest either does nothing, performs unnecessary file moves or has undesirable effects. FastFillUp is meant to be used some particular way in cooperation with other FileActions so I'd like you to criticize this intented usage rather than generic standalone command properties.

No, I don't think small files would start piling up at the end of the zone if FastFillUp was used the way I propose. I believe they would actually continue piling up at the start of the zone, just the effect would not be as pronounced. I also believe the amount of file moves to fill up the zone would not increase over time as it does now.

[jeroen]

Jeroen, you're saying that small files would start migrating towards the end of the zone - can you provide an explanation why should it happen

The FastFill command will fill a gap with files from above the gap, by moving them down. Only files that are smaller than the gap are used to fill the gap. Statistically speaking, given a lot of MyDefrag runs in the course of several months, the big files in a zone have a very low chance of being moved down by FastFill, because they have a very low chance of fitting into a gap. Small files have a higher chance of being moved, because they have a higher chance of fitting into a gap. This is why FastFill has a tendency to migrate small files to the beginning of a zone, because they have a higher chance of being moved. The same effect will happen with your FastFillUp, only the small files will migrate upwards to the end of the zone.

... or have you just ridiculed FastFillUp by taking it out of context of my suggestion?

I am sorry if my answer came across like that, it was not my intention to ridicule your idea in any way.

[Kasuha]

I am perfectly aware of what FastFill(...) does with a zone and I am perfectly aware of the fact that if FastFillUp(...) was used alone it would have very similar results in the opposite direction, with yet another consequence that it would not stuff into the zone any of the files lying above the zone. So I believe it is clear that there is no point in using it alone.

And you still seem to have not noticed the sequence I have proposed for practical usage:

FastFill()
FastFillUp(WithShuffling)
FastFill(WithShuffling)

So my question here is, what do you think about this approach as a whole. I don't think it is going to migrate small files in either direction except maybe to the center of the zone. But I don't believe that second command would completely 'undo' the first and the third would 'undo' the second, sizes and counts of gaps after each of them would vary and decrease.

[jeroen]

You are right, I did not consider the sequence of commands. No, I don't think that would be useful. The first FastFill() will still tend to migrate small files to the beginning of the zone, the next commands do not change that.

[Kasuha]

Well, maybe, but that was not the point. The point was to implement some counter-measures for this migration having negative effect on further FastFill() processing.

I explained it all in my first post here, obviously you did not read it.