This is a little long, but I think it can reduce memory usage to about 15% of it's current level, so bear with me. (I was encouraged by Jeroen's implementation of my last memory optimization idea, so blame him.
High memory usage has been a recurring theme for JKDefrag and MyDefrag. I believe that the numbers are something like 600 bytes per file for JKDefrag and 300 bytes per file for MyDefrag. This is just the initial cost of scanning the MFT. Sorting options use even more. I had a flash of insight on this today that I wanted to share before I forgot about it.
The in-memory structures are used to support two very different tasks:
Category 1: Some of the data (names, dates, attributes, full paths, etc.) is needed to support determining a target position for a file: a zone and sometimes a sorted position within the zone. Once a target position has been determined for a file, this information is no longer needed. This can all be disposed of before file placement begins. For files in unsorted zones, this can be discarded almost immediately.
Category 2: Data needed to support the file placement process. I am not certain of the exact data structure, but let's assume we just have one record per file segment. The required data would be something like:
file id (8 bytes)
starting internal file position (8 bytes)
starting lcn (8 bytes)
length (8 bytes)
target zone (2 byte)
target sorted position within zone (4 bytes)
pointer to next segment (8 bytes)
This category 2 data comes to approximately 46 bytes per segment, or about 15% of the current memory requirement.
I propose that it work this way (warning, psuedocode ahead):
preload the volume booleans
for each MFT item
if we can determine the zone without using a full path then
store the target zone
if (this item is a directory) or (the target zone is either undetermined or requires sorting) then
create a category 1 data structure for this item
next MFT item
for each MFT item with a undetermined zone
calculate the zone using full path information from the category 1 data and store it in the category 2 data
if (this record is not a directory) and (the target zone is unsorted)
dispose of the category 1 data for this item
next MFT item
for each sorted zone, perform the sort and store the target position in the category 2 structure
dispose of all the category 1 data structures
perform the file placement using the category 2 data.