Intro Download and install Frequently Asked Questions Tips and tricks

Homepage







© J.C. Kessels 2009
MyDefrag Forum
May 22, 2013, 01:35:44 am *
Welcome, Guest. Please login or register.

Login with username, password and session length
News:
 
   Home   Help Search Login Register  
Pages: 1 2 [3]
  Print  
Author Topic: Why defragging MFT is a must  (Read 61028 times)
jeroen
Administrator
JkDefrag Hero
*****
Posts: 7155



View Profile WWW
« Reply #30 on: September 23, 2008, 05:44:36 am »

All I know for sure is that Vista can move the MFT. Perhaps also XP, I haven't tried JkDefrag v4 on XP yet.
Logged
PoP
JkDefrag Senior
****
Posts: 32


View Profile
« Reply #31 on: October 03, 2008, 03:08:04 pm »

Well, I work on three computers. One of them shows the same problem as quoted at the beginning of this topic : exactly the same sequence of screenshots : after defragging the disk is clean, then after some time a bunch of files have been written right in the middle of my HDD partition.
What's the difference between the latter computer and the formers ?
I have left on the boot optimization function/prefetching on the latter and turned it off on the formers.
It turns out that the boot optimization is independent of the native defragger which I never uses. If the optimisation is on Windows moves and arranges the files used during boot time to some free place, rather far from the beginning of the disk. This is performed when the system is idle. The next time JKDefrag operates it moves back the files to Zone 2, and so on ...

I suspect this is the problem for Eric2.

JKDefrag does not support boot and application optimization. May be V4 will ?
I looked at registry changes before and after the optimizer has run, i.e. after defragging and a few days later. It seems to me that there is at least a quick and dirty workaround to the problem. I may be wrong but when the optimization function has run, it writes to this key :
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Dfrg\BootOptimizeFunction]
"Enable"="Y"
"LcnStartLocation"="5148587"
"LcnEndLocation"="5245978"
"OptimizeComplete"="Yes"
"OptimizeError"=" "

I suspect that  LcnStartLocation and LcnEndLocation are the start and end clusters of the optimal layout. If I am right it would be sufficient for JKDefrag to avoid moving files that are in this area.

Logged
jeroen
Administrator
JkDefrag Hero
*****
Posts: 7155



View Profile WWW
« Reply #32 on: October 03, 2008, 06:34:54 pm »

JKDefrag does not support boot and application optimization. May be V4 will ?
Yes, it will. I have it running over here in the pre-release version.
 
Quote
I suspect that  LcnStartLocation and LcnEndLocation are the start and end clusters of the optimal layout. If I am right it would be sufficient for JKDefrag to avoid moving files that are in this area.
That's what the documentation says, yes, and it would be great if I could do that. But the values can be completely off, and even wrong, for example a LcnEndLocation that is smaller than LcnStartLocation. I cannot trust these values.
Logged
Eric2
JkDefrag Hero
*****
Posts: 98


View Profile
« Reply #33 on: October 06, 2008, 05:58:32 pm »

All I know for sure is that Vista can move the MFT. Perhaps also XP, I haven't tried JkDefrag v4 on XP yet.

... but I thought that you always said that you processed (in v3 and earlier) the $MFT file like any other file of the system and that if it was not defragmented under XP in those versions that was simply because the windows API did not want to do it and not because you skipped that file in particular...
Sorry about my confusion but do you mean then that this was not actually the case and that in v4 you added that possibility ? ... which has then to be tested under XP ?


Logged
Eric2
JkDefrag Hero
*****
Posts: 98


View Profile
« Reply #34 on: October 06, 2008, 06:23:00 pm »

If the optimisation is on Windows moves and arranges the files used during boot time to some free place, rather far from the beginning of the disk. This is performed when the system is idle. The next time JKDefrag operates it moves back the files to Zone 2, and so on ...

I suspect this is the problem for Eric2.

Thanks for sharing your analysis and experimentation on this strange behaviour.
For the PCs where optimisation is off  have you noted after few days of usage whether new files (or new fragments of updated files)  have been written or not  far away from the Jkdefrag zones on the disk as well ?
...because in my case it happens also for new files (typically internet or dowloaded new files)... and on a PC where I had a system partition that was nearly full and fragmented  it continues writing them at the end of the partition though I have done some clean-up and run a JkDefrag -a3 -f1.... actually it does rarely write the new files or new fragments of updated files in the free zones created by the JkDefrag run... unfortunately. And I think that this is because the $MFT and/or the windows NTFS driver wronly believes that the first free available zone to write into  is still far away.  Thus my wish to clear and defrag the $MFT...
I should try with diskeeper trial version but have not done it yet...
 
Logged
jeroen
Administrator
JkDefrag Hero
*****
Posts: 7155



View Profile WWW
« Reply #35 on: October 07, 2008, 09:47:06 am »

... but I thought that you always said
Yes, JkDefrag version 3 asks the Windows defragmentation API to move the MFT, and the API refuses it. I have found that it only refuses to move the first 16 inodes in the MFT. The rest of the MFT can be moved (on Vista, maybe also XP).
Logged
Eric2
JkDefrag Hero
*****
Posts: 98


View Profile
« Reply #36 on: October 07, 2008, 04:52:48 pm »

Thanks Jeroen. That would be great to move at least some parts of the $MFT. I did not know that you could ask the API to do it. Would that be using the API with a different parameter than the whole file ?
Have you described already somewhere how you are using the windows API (command name, parameters..)  to defrag files? Do you think that running the API for some parts of the MFT is something we can try ourselves under XP through a simple command line?
Logged
jeroen
Administrator
JkDefrag Hero
*****
Posts: 7155



View Profile WWW
« Reply #37 on: October 07, 2008, 08:20:56 pm »

Have you described already somewhere how you are using the windows API (command name, parameters..)  to defrag files?
Look for "FSCTL_MOVE_FILE" with Google to find the Microsoft manual page. The JkDefrag library is documented inside the source, no formal documentation. The combined JkDefrag sources are a working example on how to use the library.

Quote
Do you think that running the API for some parts of the MFT is something we can try ourselves under XP through a simple command line?
If you are a programmer then it should be possible with a days work. Microsoft used to have a small demonstration program available, with sources, but I don't think it's available anymore. There are some defrag utilities floating around on the internet, perhaps one of them is flexible enough to move only a section of a file. I don't know, you will have to look.
Logged
Eric2
JkDefrag Hero
*****
Posts: 98


View Profile
« Reply #38 on: October 08, 2008, 06:53:08 pm »

Yes, JkDefrag version 3 asks the Windows defragmentation API to move the MFT, and the API refuses it. I have found that it only refuses to move the first 16 inodes in the MFT. The rest of the MFT can be moved (on Vista, maybe also XP).

So do I understand well ? (otherwise please correct me): 
- Jkdefrag 3 asks the Windows defragmentation API to move the entire MFT file and the API refuses to do it on XP, but on Vista the API manages to move it except for the first 16 inodes of the MFT.
- Jkdefrag 4 asks the Windows defragmentation API to move the entire MFT file as well and on Vista the API manages to move it as well except for the first 16 inodes of the MFT. You don't know under XP because you haven't tested Jkdegfrag4 under XP as you said. (but we should not expect a difference of behaviour between Jkdefrag 3 and 4 under XP regarding the MFT since they must use the same windows API, should we ?)


« Last Edit: October 08, 2008, 06:55:31 pm by Eric2 » Logged
jeroen
Administrator
JkDefrag Hero
*****
Posts: 7155



View Profile WWW
« Reply #39 on: October 08, 2008, 08:44:14 pm »

please correct me
JkDefrag v3 asks the API to move the entire MFT. The API always refuses this. JkDefrag v4 asks the API to move only the movable part of the MFT (everything after the first 16 inodes), and this succeeds on Vista.

Quote
(but we should not expect a difference of behaviour between Jkdefrag 3 and 4 under XP regarding the MFT since they must use the same windows API, should we ?)
No, it's entirely possible that JkDefrag v4 will be able to move the MFT on XP. I don't know yet because I haven't tested it yet.
Logged
Eric2
JkDefrag Hero
*****
Posts: 98


View Profile
« Reply #40 on: October 08, 2008, 09:56:17 pm »

Many thanks Jeroen, that's clear to me now.
So you found a way (for v4) to ask the API to move only the part of the MFT after the first 16 inodes.
Quote
it's entirely possible that JkDefrag v4 will be able to move the MFT on XP. I don't know yet because I haven't tested it yet.
If we can somehow help you testing that out pls don't hesitate to ask, I have few partitions with a fragmented MFT where I would be very happy to test that for you.
Logged
blume
Newbie
*
Posts: 4


View Profile WWW
« Reply #41 on: October 09, 2008, 12:38:14 pm »

after ages yesterday I found this:
http://technet.microsoft.com/en-us/library/bb457112.aspx
Quote
...
The MFT

The master file table (MFT) is defragmented along with other files on the volume. Because the first fragment of the MFT cannot be moved, the MFT is typically contained within two fragments when sufficient space is available on the volume. In this case, the MFT is considered defragmented. If the MFT is contained within three or more fragments, Disk Defragmenter looks for free space where the MFT might fit. If sufficient free space exists, the MFT is moved as a whole (minus the first fragment). If space is not available, the MFT is not defragmented.
...
works well on xp and vista...
nice they actually mentioned it somewhere
so its ok if the $MFT is in 2 fragments
« Last Edit: October 09, 2008, 06:09:25 pm by blume » Logged
Eric2
JkDefrag Hero
*****
Posts: 98


View Profile
« Reply #42 on: October 11, 2008, 03:07:31 pm »

Thanks Blume for the link, this is interesting to know.

Yesterday a friend of mine showed me his PC. It was nearly 3 years old, running Windows XP SP2 on a disk of 40 GB , only one partition. He was complaining that the PC was slower than 2 years ago. As I had my USB key with  CCleaner (portable vesion)  and JKDefrag 3.34 on it I proposed to have a look. I found at the first Jkdefrag analysis that his disk may be a perfect example for another experience to prove how windows xp continues writing at the end of the disk even after a good JKDefrag defragmentation. Even worse the Windows Boot File Optimisation process moved back all the "optimised" files at the end of the disk!
I took notes and screenshots of all the steps in order to share them with you. Here they are...



1st analysis



-> CCleaner to clean the disk + deletion of all files in C:\windows\temp\
2nd Analysis: this is not obvious to see but there were few hundred files deleted.

 

-> JkDefrag.exe -a 3  -l "JkDefrag_a3.log"  C:\*

 

-> Leave the PC up and running for the next 7 hours .. without doing anything
3rd Analysis:  note the yellow stars in the dark part ! HDview shows they are all newly small files created in C:\Windows\Temp\,  MSIxxxxxx.LOG from 16KB to few hundred KB  size.

 

-> Create a flat file of 500 MB :   fsutil file createnew c:\windows\temp\tempMB500.txt  524288000
4th Analysis: we can obviously see where the 500MB file has been created

 

->  Create a flat file of 100 MB :   fsutil file createnew c:\windows\temp\tempMB100.txt  104857600
5th Analysis: we can obviously see where the 100MB file has been created

 

->  Create a flat file of 30 MB :   fsutil file createnew c:\windows\temp\tempMB30.txt  31457280
6th Analysis: we can obviously see where the 30MB file has been created

 

->  Create a flat file of 10 MB :   fsutil file createnew c:\windows\temp\tempMB10.txt  10485760
7th Analysis: we can obviously see where the 10MB file has been created

 

Though it is obvious the location of the newly created files is confirmed with HDview.
-> Run :  Rundll32.exe advapi32.dll,ProcessIdleTasks
-> This has the effect to force the Windows Boot files optimisation (assuming the correct parameters are set in the registry for it and that the right services have been started, the C:\windows\prefetch\layout.ini was also updated during that process)





7th Analysis: we can obviously see where Windows moved the optimisation files !   Cheesy  did I say “optimisation” ? …  Smiley

 

Conclusion:   
On this PC, which was heavely fragmented and which had not been  cleaned up  for a long time, a JKdefrag –a3  occurred and that moved all the files down properly.  Classic case for many PCs which had never been cleaned up and defragmented.
Then it appeared that the newly created files, which were small log files or specific bigger files, were created anywhere on the disk but only in the previous area that older files previously occupied, never above it. And for sure they were not created in the fastest available location on the disk as we saw.
The Boot File Optimization moved the files at the top of the old area… probably at their same old location prior to the Jkdefrag –a3 ,  obviously not at the fastest location again (as it is near the end of the disk!  Undecided)

The $MFT was still one fragment on this disk.
19:46:22   Fragments       Bytes  Clusters Name
19:46:22           1    94306304    184192 C:\$MFT
I note that after a brand new installation of Windows XP, following a reformat of the hard drive,  this behaviour never happens , at least not immediately … 

In such case the $MFT was not fragmented so there was no reason to defragment it I have to say… but surely that MFT needs to be cleaned up … Is that what one calls defragmenting the free space?...I don’t know but the cleanest behaviour would be to see the new files created at the first fastest available area on the disk.

To prevent at least the good files to be moved away at the end of the disk again , after re-running JKDefrag -a3 again I disabled the BootOptimisation in the registry:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Dfrg\BootOptimizeFunction
-> Enable = N


 
Logged
Pages: 1 2 [3]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.5 | SMF © 2006-2008, Simple Machines LLC Valid XHTML 1.0! Valid CSS!