problem with defragmenting 2 TB raid 5

[lightningchaser]

I'm having problems with defragmenting a 2TB raid 5 diskset (70% filled, 2.000.000 files,1700 folders). Jkdefrag seems to hang with 100% cpu and not doing anything. If I write extended debug information a see a lot of malloc() errors.

I'm using windows 2003 server (32bit). Memory usage of the Jkdefrag rises during analysis fase to 2 GB. Defragmenting starts but the program hangs with 100% cpu. I suppose the MFT is too big?

Does it help to defragment first only certain file types? or are there any other tricks to defragment this raidset?

I used the latest JKdefrag version.

[Esprit]

The main problem is memory load. Win32 cannot assign more than 2 GB to 1 process so memory allocation fails (malloc). The problem is total number of files and folders on disk. Defraging of subset won't help. I don't know why there is a deadlock but there are some possibilities.

• Lower number of files and folders.
• Use my modified build with lower memory load which is not stable and not supported but due to use Win32 API it is safe, see http://www.kessels.com/forum/index.php?topic=911.15.
• Move to 64-bit system.
• Don't use JkDefrag.

I started to work on version which consumes much less memory but now I don't have much time to finish it. Official build keeps in memory every fragment but I'll try to treat fragments that are next to each other as a single fragment. Version mentioned above just doesn't use short filenames that are needed rarely.

[lightningchaser]

I tried your version but i still uses 2GB of memory... the log file tells me it started defragmenting but will it actually do something?

[jeroen]

If I write extended debug information a see a lot of malloc() errors.

Esprit is right, JkDefrag is simply running out of memory to store information about all the files on disk. I have it on my wishlist to test for this eventuality and gracefully stop with a nice error message, but the current version will continue and try to process the files about which it has information. There is no commandline option or anything like that to lower the memory consumption. I am hoping version 4 will use less memory, on which I am working right now.

[Esprit]

Each file and folder takes approximatelly few hundreds of bytes so your 2 M files should consume about 1 GB. It looks like your files are heavily fragmented. Try contig.exe (from SysInternals) to lower number of fragments and then try JkDefrag again. Also compression of files would cause higher memory load in JkDefrag (due to virtual fragments).