Intro Download and install Frequently Asked Questions Tips and tricks

Homepage







© J.C. Kessels 2009
MyDefrag Forum
July 29, 2014, 01:33:19 am *
Welcome, Guest. Please login or register.

Login with username, password and session length
News:
 
   Home   Help Search Login Register  
Pages: [1] 2
  Print  
Author Topic: MFT, folders and $Logfile  (Read 6863 times)
poutnik
JkDefrag Hero
*****
Posts: 1112


View Profile
« on: March 26, 2009, 08:01:58 am »

I am not expert in filesystem details.

But AFAIK  NTFS  uses $Logfile for its log for disk changes
(not the same as USN journal log on application level ),

until there are reflected in other NTFS metadata structures.

So I suppose there is very frequent write access.
Does exist related need or advantage to keep $Logfile and MFT/folders close each other ?
« Last Edit: March 27, 2009, 07:34:49 am by poutnik » Logged

It can be fast, good or easy. You can pick just 2 of them....
Treating Spacehog zone by the same effort as Boot zone is like cleaning a garden by the same effort as a living room.
jeroen
Administrator
JkDefrag Hero
*****
Posts: 7205



View Profile WWW
« Reply #1 on: March 27, 2009, 12:52:40 pm »

The purpose of the $Logfile as I understand it is to be able to roll-back changes in case of a crash. So, if something has changed in the metadata but not logged in the $Logfile then it can be rolled-back. This is only possible if the $Logfile is updated AFTER metadata updates, the other way around would be useless. The file is very likely delay-write, when the disk is not otherwise busy. So I think there is only a small advantage to having the MFT and the $Logfile close together.
.... It's an interesting question though, and I would like to know for sure.
Logged
jimbo
JkDefrag Hero
*****
Posts: 84


View Profile
« Reply #2 on: March 27, 2009, 03:18:45 pm »

The usual sequence for transaction logging in a system such as NTFS is
[ul]
  • Write details of transaction to logfile
  • Make change to files
  • Write transaction complete marker to logfile
[/ul]

since disk is cached, there is no guarantee that these writes will happen immediately, but they are constrained such that they must be committed to disk in that order.

If several write transactions are taking place at once, it is possible for them to be interleaved, so the first step for each can be merged and committed to the logfile, then the second step for each written to disk, then the logfile updated.

The NTFS logfile is an "infinite rolling log" so it simply cycles through the allocated space. It is therfore possible for it to become full, and if that happens, all writes to the filesystem hang until the open ones are flushed to disk and the logfile cleaned up.


During filesystem recovery, for example after a crash, NTFS performs three passes.
First is an analysis pass, where it determines the state.
Next is a redo pass, where it tries to re-apply all the open transactions in the log, marking them as done if they complete successfully.
Finally is the undo pass, where it will roll back any changes that are still open, guaranteeing that the file system structure is consistent and stable.
Logged
poutnik
JkDefrag Hero
*****
Posts: 1112


View Profile
« Reply #3 on: March 27, 2009, 03:45:59 pm »

Note that on my system partition filemap the $Logfile is placed perfectly in the middle,
as if it was important to have its location optimized.
« Last Edit: March 27, 2009, 03:47:39 pm by poutnik » Logged

It can be fast, good or easy. You can pick just 2 of them....
Treating Spacehog zone by the same effort as Boot zone is like cleaning a garden by the same effort as a living room.
HolyMan
JkDefrag Hero
*****
Posts: 166


View Profile
« Reply #4 on: March 27, 2009, 05:21:42 pm »

Note that on my system partition filemap the $Logfile is placed perfectly in the middle,
as if it was important to have its location optimized.
On mine it is not. So you may be observing a coincidence.
On my disk the $Bitmap file is in the middle.
But if the size of the $Logfile is fixed and ultra heavy access, then I would sure like it at the very beginning of the volume. Is there a way to move it or recreate it on a more convenient spot?
Logged
poutnik
JkDefrag Hero
*****
Posts: 1112


View Profile
« Reply #5 on: March 27, 2009, 07:14:16 pm »

But if the size of the $Logfile is fixed and ultra heavy access, then I would sure like it at the very beginning of the volume. Is there a way to move it or recreate it on a more convenient spot?
I very doubt it is movable.
Not even sure, if Perfectdisk boot optimize moves it, but my memory is not so good.

Maybe it is not at very beginning for the same reason,
as it is not the most recommended place for MFT: Not to travel too far.

Not being at the start means little less speed, but also access to and from the both sides.
Logged

It can be fast, good or easy. You can pick just 2 of them....
Treating Spacehog zone by the same effort as Boot zone is like cleaning a garden by the same effort as a living room.
poutnik
JkDefrag Hero
*****
Posts: 1112


View Profile
« Reply #6 on: March 28, 2009, 09:13:56 am »

On mine it is not. So you may be observing a coincidence.
On my disk the $Bitmap file is in the middle.

I have there $Bitmap, then $Logfile, followed immediately by MFT reserved zone.
Logged

It can be fast, good or easy. You can pick just 2 of them....
Treating Spacehog zone by the same effort as Boot zone is like cleaning a garden by the same effort as a living room.
Bibby
JkDefrag Senior
****
Posts: 24


View Profile
« Reply #7 on: March 28, 2009, 10:14:28 am »

I have seen both $bitmap and $logfile being moved by MyDefrag... at least it shows it is doing this in the gui.
Logged
HolyMan
JkDefrag Hero
*****
Posts: 166


View Profile
« Reply #8 on: March 28, 2009, 03:23:43 pm »

I have seen both $bitmap and $logfile being moved by MyDefrag... at least it shows it is doing this in the gui.
Was that on the bootvolume?
Logged
poutnik
JkDefrag Hero
*****
Posts: 1112


View Profile
« Reply #9 on: March 28, 2009, 03:35:38 pm »

I have seen both $bitmap and $logfile being moved by MyDefrag... at least it shows it is doing this in the gui.
These files ( and others, of course)  I have in Mydefrag.dat file,
containing files known as unmovables. The same file are there for any other partition.

C:\$LogFile
C:\$MFT::$BITMAP
C:\$Bitmap
Logged

It can be fast, good or easy. You can pick just 2 of them....
Treating Spacehog zone by the same effort as Boot zone is like cleaning a garden by the same effort as a living room.
grahamcopley
JkDefrag Senior
****
Posts: 48


View Profile
« Reply #10 on: April 29, 2009, 05:20:46 pm »

Am I right in thinking that, most of the time, the disk drive is moving backwards and forwards between 'system files' (i.e. Pagefile.sys, $MFT, $MFT::$BITMAP, $MFTMirr, $LogFile, etc).

If so, then to optimize the disk, then shouldn't the Pagefile, $MFT, Boot Files, Directories and Frequently Accessed Files be placed as-close-as-possible/in-between the unmovable 'system files'.

This might mean moving the files to between 1/4 and 1/2 into the disk, but give 'reduced disk seek times'. Wouldn't this give a better performance improvement than that of 'lower rotational delays', gained by placing them at the start of the disk.

Do some files need to be placed nearer than others to specific files?
Logged
jeroen
Administrator
JkDefrag Hero
*****
Posts: 7205



View Profile WWW
« Reply #11 on: April 29, 2009, 09:30:44 pm »

Am I right in thinking that, most of the time, the disk drive is moving backwards and forwards between 'system files' (i.e. Pagefile.sys, $MFT, $MFT::$BITMAP, $MFTMirr, $LogFile, etc).
These files are heavily cached, and only the $MFT is used frequently. The pagefile for example is only used if your computer has insufficient memory (not quite true, but close enough). The others are likewise not used very much. Please note that most of these files are unmovable on a running system, they can only be moved if you boot from a cdrom.

Quote
This might mean moving the files to between 1/4 and 1/2 into the disk
The best place for the $MFT and the directories is the sweet spot of the harddisk heads, the average middle of all head movements. This is not the middle of the data on the disk, because disks are faster at the beginning than at the end. An accurate determination of the sweet spot requires measuring, and moving these system files will alter the sweet spot. A good approximation is about 1/3 into the data.

Quote
Do some files need to be placed nearer than others to specific files?
I'm not sure what you are asking. MyDefrag scripting makes it possible to organize your disk any way you want.
Logged
poutnik
JkDefrag Hero
*****
Posts: 1112


View Profile
« Reply #12 on: April 29, 2009, 10:40:28 pm »

Am I right in thinking that, most of the time, the disk drive is moving backwards and forwards between 'system files' (i.e. Pagefile.sys, $MFT, $MFT::$BITMAP, $MFTMirr, $LogFile, etc).
These files are heavily cached, and only the $MFT is used frequently. The pagefile for example is only used if your computer has insufficient memory (not quite true, but close enough). The others are likewise not used very much.

I am afraid $Logfile as NTFS circular transaction log is used very often.
It is cached, but writes must be done.
Every disk write must be marked in log as started and finished transaction,
because NTFS is a transaction oriented filesystem.
Logged

It can be fast, good or easy. You can pick just 2 of them....
Treating Spacehog zone by the same effort as Boot zone is like cleaning a garden by the same effort as a living room.
poutnik
JkDefrag Hero
*****
Posts: 1112


View Profile
« Reply #13 on: April 29, 2009, 10:52:24 pm »

The best place for the $MFT and the directories is the sweet spot of the harddisk heads, the average middle of all head movements. This is not the middle of the data on the disk, because disks are faster at the beginning than at the end. An accurate determination of the sweet spot requires measuring, and moving these system files will alter the sweet spot. A good approximation is about 1/3 into the data.

I remember this 1/3 from some MS recommendation.
It also depends, how frequently accessed files are distributed.

E.g. All files accessed in last month are placed in first 20-22% of data in my system partition.
So sweet spot would be here somewhere 10-15% of data range.
Logged

It can be fast, good or easy. You can pick just 2 of them....
Treating Spacehog zone by the same effort as Boot zone is like cleaning a garden by the same effort as a living room.
poutnik
JkDefrag Hero
*****
Posts: 1112


View Profile
« Reply #14 on: April 30, 2009, 05:39:42 am »

I am afraid $Logfile as NTFS circular transaction log is used very often.
It is cached, but writes must be done.
Every disk write must be marked in log as started and finished transaction,
because NTFS is a transaction oriented filesystem.

Interesting educational PDF in EN about logfile is here:
http://twiki.di.uniroma1.it/pub/Sistemioperativi3/OnLine/NTFSLog.pdf
Logged

It can be fast, good or easy. You can pick just 2 of them....
Treating Spacehog zone by the same effort as Boot zone is like cleaning a garden by the same effort as a living room.
Pages: [1] 2
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.5 | SMF © 2006-2008, Simple Machines LLC Valid XHTML 1.0! Valid CSS!