usn journal vs deletejournal on vista

[lh]

I am having an issue with vista sp1 and
fsutil usn deletejournal /N C:

I wish to delete it and start over with it.  However it gives me an 'error:  access is denied.'  when I run the command.  Has anyone else seen this?

[boco]

UAC enabled? 

You have to start a console with Admin (elevated) rights. Right click on the entry for the DOS console in your start menu, then select 'Run as Administrator'.

[lh]

I usually use a admin level console to do all of this.  I had also gave disabling UAC a try a few weeks ago.  That did not seem to work  .  Even tried it from a recovery console and safe mode. 

It is acting like something is running that has it.  Turned off sysindexer which is the big one.  Do not have live messenger installed (another big one to grab this file).  I was also thinking it could be permissions.  But I am a bit shy of messing with those as I could realy mess up the install which I do not want to do (least until I have a good backup).  Have tried looking at the access permissions rights but cacls/icacls also gives the same error.

It is kind of a pain, as currently it is spread across 70 or so fragments in zone 2.

Also make sure you turn sysindexer OFF.  Wow this bad boy destroyed any sort of free space compaction I had done with -a 7.  It just slowly gets worse and worse.  As it is a forward building file.  So it always is claiming 'small' free chunks where ever it can find it.  The indexer files are usually in the 300-400 fragment range.  It is a slick utility.  But performs badly as it likes to rebuild its index files a lot and they get/are fragmented all over the drive.  Then it tends to make other files fragment.  Never mind how much it slows the computer down.

[boco]

I guess Vista itself makes use of USN journal, so if this is the drive where Vista resides, deleting is a no-go.

ATM Sysindexer behaves on my drives, so I leave it enabled. But thanks to your hint, I know where to look in case.

[fr@nkie]

Hi,

I have the exact same problem LH reported.

After instalation of SP1, my USN increased more than 70%, now with 2,7 GB and 75 fragments. This is awful!!!

Not being able to delete the USN Journal is not only bad for disk fragmentation, but also for backup management (now a days I have to use double layer DVD to backup my drive). Besides, if the USN gets corrupt or dirty, and this appened in the past, you have to do a Windows Clean Install to fix Windows. 

I have uninstalled Live Messenger, Search disabled, DFS disabled, Indexing not installed (Vista default). UAC and Defender also disabled.

It seems that Vista is realy using the USN, like BOCO said, I haven't been able to delete it either in Normal or Safe Mode (with just 4 services running: RPC, Plug and Play, DCOM, User Profile Service). Again, to confirm BOCO, I CAN delete and rebuild the USN on my D:\ drive (without OS), but CANNOT delete it in the C:\ drive (with OS).

I belive this is a Vista bug, so I post a thread at Microsoft, but still waiting for the replly.

If they're able to solve the problem, I'll be glad to share with you.

Cheers

[lh]

Cool.  Its not really a big problem for me.  It is more anoying that I cant figure out what HAS this file.  I also tried what you did.  I turned off every service I could.  I even took the extreem step of killing off every task I could.

There also seems to be SOMETHING that causes it to be rebuilt.  As now instead of 70ish fragments I have 30ish, but the size is about the same.  All I have been doing is using media player too.

But if you get something out of the other forum please post here!  Or at least point at the forum you are using.  Thanks.

[fitch]

Hi guys,

I haven't attempted to defrag Vista yet but sounds like you have the same problem as trying to get rid of the USNJRNL on XP.  Have you tried booting with BartPE and then deleting the journal from there?  Just got rid of mine that way (119 fragments) on winXP based system.  Now finally I can get all my files contiguous.

Good luck

[fr@nkie]

I have never had this problem with XP.

But the UBCD4Win seems a better option than Bart PE, which is not updated since early 2006 and it does not  support Vista. You can use the UBCD4Win on a Vista host providing your source is XP. The resulting ISO is slower and bigger than Bart, but also provides you additional tools and seems easy to build.

Cheers

[lh]

I was going to give BartPE a try.  However it is XP vs Vista.  The BartPE build process doesnt work with Vista at least from what I have seen on the forums.  There is a way to do it with Vista and the some MS tookits but I have not downloaded the kit yet to try it.  I will give the UBCD4Win a look though. 

The HP laptop that has this issue also has a recovery console that boots from a different partition.  I tried removing it from there.  I think I got the same error (this was a couple of months ago, ive been trying to do this awhile).

I would also like to understand what is causing it in the first place.  So I can either limit the issue or remove it.  At one point Vista had rebuilt it and made it contig anyway!  So there is something weird going on with this file.

[cquinn]

I know this won't help much,  but I just tried deleting the journal on my laptop running Vista Ultimate SP1, and it seems to have processed without any errors.   

[fr@nkie]

Thanks CQUINN,

I believe LH and I have a common denominator, which is a HP Laptop with a Recovery Partition instead of a Vista DVD. Maybe this is it. I'm using Premium 32. This sounds like a HP problem.

[lh]

I went and tried it on a clean install of vista.  It worked like a champ.  So as fr@nkie said it is something specific to HPs image of Vista (home premium 32).  Possibly the copy of Norton that was installed caused it in the first place.

From the research I have done on the Journal it is actually kind of slick.  It tracks when a file has changed in some way.  The defrag utility here could actually use it by caching off the last USN it saw then fiddling only with the files that have changed in some way.  Such as movements etc.  It could also use it to decide what files go towards the front of a zone as it has a LOT more info in it than just timestamp.  It is however a CHANGE journal.  So lastaccess time is still usefull for files that are accessed a lot but not changed.  I am not sure yet but even running the defrag may be causing the file to get bigger.  Need to do more research.

It is pretty binary either on or off.  Then there are caps about how big the file gets.  I had removed a BUNCH of permissions from a large number of files (20k).  So that may have caused it to grow up to the max size (2 gig I think is the default).  Other programs can fiddle with the size so that is probably why you see people saying it has taken over their drive.

Here is part of the API for it.
http://msdn.microsoft.com/en-us/library/aa365481(VS.85).aspx

I was hoping to find something about the error but nothing so far.  I am really starting to think it is a permission thing or some leftover bit of software from the image.  Both of my XP copies at home had it off by default so I couldnt use those to test.

[fr@nkie]

Hi LH,

So you where able to delete it with a clean install?

Can you post your services config here so I can check my own? We disabled a lot of things, maybe missing some service dependency, let me try to reverse the procedure.

Type services.msc, go to View> Add/Remove Columns, under Displayed Columns remove Description and Log On As; type OK. Then go to Action> Export List..., and paste it in here.

2GB is the limit? So I'm over the threshold, and growing!

I also had Norton IS, but I got rid of it with the Norton Removal Tool. Nevertheless Vista kept the drivers, which I removed manually from devmgmt without error, and some items in the Registry, which I could find with the search function, typing "Symantec". As for left overs, maybe, but I can't see anything unusual with Autoruns, Process Explorer or Hijackthis; maybe Vista is keeping old DLL and SYS clones in the WINSXS folder for PC history, and that is creating conflicts. But I didn't change any permissions (why should?, everything is running OK except this command), so I don't think that can be the "cause". Besides, why can I remove the Journal in my D:\ drive? I just did it again, as speaking, this was no begginer luck!

Were you using Acronis in your previous instalation?

I've also searched in the HP database and forums, but no sucess. Looks like their clients are not USN-aware.

If you want to it try in XP, simply type fsutil usn createjournal m=1000 a=100 c:, and then delete it. Works like a charm!

[lh]

I didnt blow the install away on the laptop.  The other vista was another MSDN virtual machine I have.  It deleted before and after sp1.  I also turned on windows defender then turned it off like what I had happen on the laptop.  It also worked.  So its not that .  I am tempted to try a norton install on it.

I dont have easy access to the box at the moment.  But I tried disabling as many tasks as I could a few weeks ago.  I went into the services control panel and stopped every single one that it would let me.  Makes shutting down a bit difficult .  But it was down to the bare min 5 or so services.  I did not disable them as I wanted to have a box that booted up again.  I did this about a month ago.  Right now the only thing that I have disabled is the sysindexer and windows defender.  So it is pretty close to original install.  This tells me its probably not a service.  I also used proc explorer from sysinternals and went on a exe killing frenzy at the same time.

That Winsxs folder is another nightmare of a dir.  TONS of linked files.  Interesting fix for DLL hell.  But more of a way of keeping exe's in sync with older dlls.  Not sure if that is a good thing or bad.  .manifests have their uses.

I didnt think of drivers.  Definatly something to try there.

This weekend I want to try a knoppix boot and see if I can see what the permisions are on those files.

It could have been turned on from anything in the extra junk HP installed.  Office, Norton, etc...  So even though the program is gone its still busy logging away as it a combination of 'is the file there, what mode is it in, and a flag in the boot record of the disk'.

Im still leaning towards permissions .  It could have come from the service that initated the journal or from the original image itself.  As access denied is really one of 2 things either permision or someone has the file locked.  Given the fact it did the same thing from the recovery console makes me think permision.  Though I do get the same access denied error from the clean install vista with icacls.

You can see the sizes for the journal by doing 'fsutil usn queryjournal c:'  Though I think mine will probably do the same thing and just keep growing.  I even tried filling the drive to see if I could cause it to purge itself.  My math was also wrong on the limit.  It is about 38meg by default.

I also never used Acronis.

[fr@nkie]

Guess I agree, I was missing your point.

Last night I booted in normal mode without any third-party processes and services, and I search Process Explorer for NTFS metadata (Image #1).

The USN is really not in use, but it's there (Image #2 - this is Hdview). Neither Vista or any third-party processes are using it.

Further on, I looked at the permissions of the NTFS metadata I could query ($Extend$RmMetadata), and I confirm that any member of the Administrator group has no permissions to delete, execute, change permissions or modify its state (Image #3).

The latest image (Image #4) shows what I, as a member of the Administrator Group, am able to do: query but not delete (=LH).


So, how can we change the permissions of a super-hidden file like the USN Journal? If, at least, where visible from the Windows API...