Intro Download and install Frequently Asked Questions Tips and tricks

Homepage







© J.C. Kessels 2009
MyDefrag Forum
May 26, 2013, 02:45:53 am *
Welcome, Guest. Please login or register.

Login with username, password and session length
News:
 
   Home   Help Search Login Register  
Pages: [1] 2
  Print  
Author Topic: Comments or macro usage  (Read 4106 times)
TeDDy
JkDefrag Hero
*****
Posts: 158


View Profile WWW
« on: March 05, 2009, 02:03:17 pm »

Definition in help is:
Scripts can contain the following comments:
Anything between "/*" and "*/"

Built-in scripts have comments or keywords wich can be filled by the command line arguments.
   Slowdown(/*SLOWDOWN*/) default=100
   Debug(/*DEBUG*/)
   WhenFinished(/*WHENFINISHED*/)
   SetBeginOfZone(/*FREESPACE*/)      will execute defaults if not specified by cl. arguments?

Simply putting this macros will modify the program behaviour?
If the only rule in the script selects:

-  FileSelect    all    /*SPACEHOGS*/
or
-  FileSelect    all

Files are selected all OR, all AND, NOT....SPACEHOGS ?
files selected by this ONLY rule go to begin or spacehog zone ?

  FileSelect
    FileName("$mft")
    Directory(yes)
can be used to put them in absolute position on drive,
can they be sorted (directories)?

processing $mft optimise also the expansion zone and metadata ?

Pagefile.sys, Hiberfile.sys and other usually locked files folders can be totally moved/defragged if windows booted from another drive?

On ntfs drives folders can be moved by the API and on Fat32 not ?!? Vista or w7 can move them on fat32 ? The API has evolved somehow ? No backward compatibility ? Under older OS (like 98) folders are defragmented/moved. Maybe a  patch to the api, some underground , not documented functions can do it on fat32 ?

Raxco Perfect disk, somehow unmounts drives to defrag folders/system under windows.
With MyDefrag drive is mounted when defrag mft and folders. As I can see it is not moved. Perhaps if not fragmented, don't touch it ?

If it can't be moved, what is the purpose to select it?




Logged
WindHydra
JkDefrag Hero
*****
Posts: 69


View Profile
« Reply #1 on: March 05, 2009, 04:36:23 pm »

You used too many ?s it's confusing.

It seems that /*SPACEHOGS*/ excludes the spacehog files (files >50MB, or with certain extensions, or not accessed in a month).

On NTFS things such as MFT, folders, and other file system related stuff are files, so they can be defragged.  MyDefrag didnt' use "FileSelect FileName("$mft") Directory(yes)" because it wants to put $MFT in the very beginning (thus using sort) but doesn't want to sort directories.  Many directories are quite small, takes only 1 cluster.

Moving the $MFT file might cause the MFT reserved zone to be moved at next reboot.  The reserved zone can't be defragged.
Logged
jeroen
Administrator
JkDefrag Hero
*****
Posts: 7155



View Profile WWW
« Reply #2 on: March 05, 2009, 06:33:25 pm »

processing $mft optimise also the expansion zone and metadata ?
The $MFT reserved area is not a file, but a setting inside Windows. The ReclaimMFTzone volumeaction will move files out of this area. I don't know which metadata you are referring to, but most NTFS streams are treated by MyDefrag just like ordinary files.

Quote
Pagefile.sys, Hiberfile.sys and other usually locked files folders can be totally moved/defragged if windows booted from another drive?
I'm sorry but I do not have a list. It depends on (the version of) the Microsoft defragmentation API, what services are running, on which userid MyDefrag is running, and more.

Quote
On ntfs drives folders can be moved by the API and on Fat32 not ?!?
Correct. MyDefrag will send the commands to the API, but if the API has refused to move 20 directories in a row then MyDefrag will treat all directories as "unmovable".

Quote
Vista or w7 can move them on fat32 ?
I don't know, I have not tested JkDefrag or MyDefrag on Windows 7.

Quote
No backward compatibility ?
The Microsoft defragmentation library is part of Windows, embedded deep inside filesystem drivers. It is not included in (compiled into) MyDefrag. The Application Programming Interface (API) of the defragmentation library is the same on all Windows versions, and that is why there is only 1 version of MyDefrag.

Sorry, I don't understand your other questions.
Logged
TeDDy
JkDefrag Hero
*****
Posts: 158


View Profile WWW
« Reply #3 on: March 06, 2009, 09:08:25 pm »

q1
On ntfs fresh formatted drives, no files on them, diskmap shows 2 compact unmovable files/zones. First big file/zone is

MFT+reserved area. The second is metadata(as PerfectDisk says). Mydefrag process both with ReclaimMFTzone?

q2
 - 1 hdd, 2 active partitions each with his OS. If booting from first OS can defragment ALL files from the other

partition with windows files(pagefile, hiberfile, dat, etc).
 - Or boot from BartPe to defrag the windows drive.

q3,4,5
Quote
The Application Programming Interface (API) of the defragmentation library is the same on all Windows

versions
Same on all, same on NT's, or same on w2k/XP/2k3/etc ?

 If same on all, on w95/98/ME folders CAN be moved on fat32.
 Back compatibility mean: if defraggers can move folders on w98's API/library/system, the ones included in w2k should

be able to move them the same. I suspect some undocumented functions can do the job.
 Why? Because Raxco's PerfectDisk is able to move folders on fat32 drives in offline mode under windows by closing all

open handles to the drive.
 So, the API can do it, in special conditions, somehow, by locking the drive until offline defragmentation ends.

 My feeling is the API refuses to move folders because some deep conditions are not satisfied by previous actions or

executed functions at driver/filesystem level.
 I'll try to trace the calls on built-in defrag and SpeedDisk on w98 and w2k built-in defrag/Raxco/Mydefrag. Lot of

time to spent. Who knows? Some diferences founded this way can make the light.

The other questions were about the diferences between a script made by you and the same script without /*SPACEHOGS*/.
The script with /*SPACEHOGS*/, make excludes in files selected previously.
The macro is to pass to program the -u additions for spacehogs from the commandline.
Without -u params in the commandline changes are made by the "unused" /*SPACEHOGS*/.
I've noticed the modified behaviour when runned un a slow computer, the macro forces previous selected files to exclude

the spacehogs without the exclusion to be present in the previous file selection.
 The simple presence of the macro in script body seems to do the exclusion otherwise necesary in previous selections.

Am I correct?

If this is correct, it is a big step to solve "my problem" spreaded on this forum.
A macro-keyword able to exclude masks by default from previous selections(like /*SPACEHOGS*/ does) and named

/*ENDOFDRIVE*/ can select files and exclude them from previous selections (including spacehogs) by default.

With a new /*ENDOFDRIVE*/ macro, stronger than /*SPACEHOGS*/, files can be selected to go to the end, eliminating the

hasle to build complex "* and not..." for n-1 rules.

Pushing things even far, the built-in list of spacehogs can migrate to a text file, in the mydefrag folder, with

spacehogs on separate lines or same line with separators.
 This file, spacehogs.txt can have a sister named endofdrive.txt wich can contain another list of masks to place to the

end of drive.
 If those files are present in mydefrag folder, and their contents is not nul, script compiler can add/process by

default the spacehog+endofdrive selections and substract mentioned files from previous rules.
 So anyone who creates a script and make 1 selectionrule, can have by default 2 more rules by those 2 files content.
 People who don't need to optimize with /*ENDOFDRIVE*/ and /*SPACEHOGS*/ can delete the macros or empty/delete the

files.

_________
 Just to complete the idea I will put next the answer included in reply to the other topic

http://www.kessels.com/forum/index.php?topic=1559.msg9588#msg9588
___________

By enforcing the behaviour of endofdrive over spacehogs and both to have specific position on drive (fastzone,

spacehogs, freespace, endofdrive) you can avoid complex excludes in rules for the fast zone.

Another advantage can be less file moves when making place in the fast zone.

The general act of MyDefrag is to solve selected files for a zone and then solve next selection/zone.

When process zone 1 and find a file who don't fit here, it moves it to first gap. Am I right?

I assume he don't know where to put it because the next rules are not parsed yet. Right?

If a script has 4 rules, some files are moved by 1,2,3 and in the end by 4.

 If the program read all the rules in script, when vacate a file from z1 it can place it directly to his zone (z3 for

example). If the space in z3 fot this file is also taken by a file (belonging to z2 for ex) to suspend first move and

vacate new file to z2.
 This is recursive but somehow quick convergent. For sure this result in less file movement.

That's why I say: after drive analysis, all declared zones for placement must be known. To know how to better vacate

files.

The reverse method is to read all the rules and to process first files who go to the end, calculate free space, process

spacehogs and after that sort remaining files for the fast zones of drive.

Let me explain.

Most of drives have tens-hundreds filetypes who
- need low speed,
- are big
- many
- never modified

In this category I put archives, music, movies, pictures, txt, pdf, inf... etc
This files can take from 20 to 80% of drive space and every ata drive has at the end enough speed(> 4Mib/sec) to allow

them to be read without problems. Those files can be also acessed 1/day but still no need to mix them with files

acessed thousand times/day to the fast zones of the drive.
 The sortbyacess rule can't count the total hits. Just the time elapsed from last acess.
 Two files acessed in the last 1 day can have few hits (mp3 file) or milions of hits (dat file).
 This is where I wish to make the selection possible.
 By acess time they can go to zone1, luckyly to spacehogs. By my wish some 1day files can go to end and stay here

whatever.

 If the first action is to move those files to the end, then the spacehogs and exclude them (with as simple as possible

selection type) from other possible selections will free the space in the fast zones (this allow easy fill with

necesary files), reduce some useless movements to vacate and also minimise the needed ram for processing (by keeping

just remaining files to process for fast zones) .

 Like renovation of a new house and firstly put carpets, wall painting, place furnitures and after that throwing the

garbage and old stuff to the box on the keep room.
 My fixation is to get/clean the garbage and put it in a box far away, move the old stuff to keep in a separate room

and when I have full place to move, beautify the rooms without the need to move unused boxes from here to there to put

a chair and again to put table or to put my wife to hold it until I finish the placement.

 I admit, this is somehow the viceversa (partial) of what mydefrag do at this moment.

 In my opinion the simplest method after analise of drive is to:
 - read a file with endofdrive(EOD) and spacehogs(SH) masks
 - select-cut the EOD files from the list and place them
 - calculate the free space before the EOD files to get the last sector of SH zone(+ 10÷30% to compensate unmovables)
 - select-cut the SH files from the list and place them

< now the length of the list get smaller without EOD&SH files and has only the "fast files" >

 - place files the "fast files" by the rules in the script


 But, as a middle solution, when placing files begining with first sectors of the drive, when a file who need to be

moved to a later zone matching the next rules, a good idea is to suspend active zone and vacate this file to his zone

and then resume the active zone.
 
 At least do that with EOD & SH files!

_________
about your reply at http://www.kessels.com/forum/index.php?topic=1559.msg9588#msg9588
___________

q1
 My time is zero compared with the time you put to make this program better or answer to our questions.
 Making what we think is right motivate the time spent to do it.
 As long my ideas are not rejected, I can explain and rafinate them.
 I think they are good enough as they come from long observations of the SpeedDisk's behaviour.

q2
 Declared before just to be known by next rules.
 I explained how rules can be tricked for SortByLastAccess without precise selection for EOD&SH files.
 To avoid "mistakes" the EOD & SH rules must be stronger than other selections, and to free the fast zone they should

have bigger priority.
 The current paradigm, without some fix from you, will force the repetition of "and not(*lot of masks*)" to be present

in each of (n-1)rules. Before, I explained how I think the fix can be done (in script declared once or in external

file).

q3
 The scripting has keywords. 'Select' is one of them. 'Select1', 'Select2' can be labels for later selections.
 Or to make selections inline with + - ?
 Just an idea!

 My favourite programming language is unable to read/write unicode files, will make only commandline, no gui dos like

programs and for sure can't solve the problems for mydefrag or usual users (it will restrict the flexibility).
 I can make just an program who read some ini files to produce another script to be used by mydefrag.
 This is the longer way.
 The short way is for Mydefrag to have built-in gui for make the config for each and save it in the root of each drive;
 Or to have a simplified way to make rules in the script as I try so hard to explain.
 
 True, my extra program can read hundred of selections and repeat them in the generated myd script. Bad idea because the lines can be much longer the mydefrag can read perhaps.
 Or
 The script wich has /*ENDOFDRIVE*/ to read the lines from 'eod.txt' and similar for /*SPACEHOGS*/ with 'SH.txt'. Better I say because this reuse the masks in every generated script by one or two macro/keywords in script body.

q4
 Predefined/prioritary zones are very good. They separate the files in fast/slow zones and by deleting the macro/keyword from script this zones will not be used.

q5
 Slow/Fast Optimize scripts has 5 zones not counting freespace zones. Pretty close to 8 wich has the endofdrive zone.

 Spacehogs go just before the big free space. Endofdrive files go after the big free space, at the very end of drive.

q6
 True if little free space. But zones can be processed in any order as mydefrag really can write anywhere in the free space on drive.
 If first rule places files at some offset of the drive, future rules will not be able to place files before this offset point. Only after or in strange positions!
Logged
jeroen
Administrator
JkDefrag Hero
*****
Posts: 7155



View Profile WWW
« Reply #4 on: March 11, 2009, 02:00:34 am »

I'm sorry Teddy, but I do not have the time to read or answer your posting, it is too long.
Logged
TeDDy
JkDefrag Hero
*****
Posts: 158


View Profile WWW
« Reply #5 on: March 11, 2009, 10:44:04 pm »

No problem.

Take a long breath.

Maybe you find this useful! (I hope)

Basically, this is the full list/explanation.

Sorry, but shorter posts can't get the big picture.

From here, short explanations can be done by request, if you ask some.

Please take a look here http://www.defraggler.com

It moves files to end before regular defrag.
Logged
WindHydra
JkDefrag Hero
*****
Posts: 69


View Profile
« Reply #6 on: March 12, 2009, 01:34:19 am »

Didn't finish your post, but are you saying mydefrag should analyze all the files before moving them so there will be less file movement?  Sounds like a good idea, but it might be too much trouble to optimize file movements at the very beginning, especially when many files are fragmented and there's not enough disk space.  I think PerfectDisk does something like this, its analysis phase is very long and it claims it can defrag in one pass.

It might be a good idea to calculate the start of the next zone though, so at least files aren't moved into the designated free space before the next zone.  Or does mydefrag already does this?
Logged
TeDDy
JkDefrag Hero
*****
Posts: 158


View Profile WWW
« Reply #7 on: March 12, 2009, 01:57:50 pm »

 That is the idea: read all rules from script, analize the drive and establish the aproximate begin and end for each rule.
 This way the vacated files go directly to their defined zone.
 My idea is to apply this only to SPACEHOGS and ENDOFDRIVE rules/zones because the files are just fastfill-ed not sorted. In this 2 zones, files can have 2-3 fragments without the need to defragment.

 The feature can be added in 2 ways:

 - select endofdrive files and move to end, leave free space zone, select spacehogs and move them before the free space, optimize remaining files for performance to the begin of drive as usual.

 - read endofdrive and spacehogs masks before other rules and AUTO exclude them from rules for the fast zones AND process them as the last 2 actions of script.

 In my opinion first way is better because endofdrive and spacehogs zones need less optimisations, can have gaps and fragments and stay in place from defrag to defrag.
 After that the files from fast zones can be optimised easier because of extra space created at the beginning.

 The program understands "SPACEHOGS" macro. Also it has a built-in list for it.

 Adding "ENDOFDRIVE" macro can be easy, I think.

 Also, when script contains "SPACEHOGS" and/or "ENDOFDRIVE" macros, MyDefrag can understand to read the contents of 2 text files with masks founded in his folder.
 This way, the script will not contain huge masks for them.

 This time, the post is shorter, based on previous loong explanations.

 Can I have your opinion?
Logged
WindHydra
JkDefrag Hero
*****
Posts: 69


View Profile
« Reply #8 on: March 12, 2009, 04:11:02 pm »

I think I get it now.  You are saying to move the endofdrive files to the very end of the disk, and spacehogs to a spacehog zone just below the endofdrive zone.  But aren't both endofdrive and spacehog files slow? Why seperate them into 2 zones?

Mydefrag optimizes files for speed, so you should use the optimize strategy on the system drive only and keep the large files on a seperate partition/disk.  Personally I use Fast Optimize on C drive and a custom script with 2 zones (MFT+directory in zone 1 and everything else in zone 2).  Moving files to the every end will just give up the speed for those files.
Logged
TeDDy
JkDefrag Hero
*****
Posts: 158


View Profile WWW
« Reply #9 on: March 12, 2009, 06:08:18 pm »

 You are the first who get closer to the idea I try to spread here !
 It is not my idea because Symantec was used it in SpeedDisk long time ago with a big success.
 The same idea is used already by Defraggler (but less flexible)

Just to clarify you:
 I divide the drive in 2 zones:

 - the fast zone(FZ), between 20-40% of drive, at the begining (outer traks). This should have the MFT, folders, layout.ini and the most acessed files sorted decreasing. In this zone the read/write speed is in 70-100% the drive can have. First tracks can be more than 10 times faster compared to last tracks !

 - the slow zone(SZ) (remaining of the partition) with 3 subzones in it:
     a) - SPACEHOGS
     b) - biggest freespace
     c) - ENDOFDRIVE files/folders

 a) files very frequently modified, temporary, logs...you got the idea. The files in this zone are deleted, modified, added every day or minute. They need some speed for read/write and free space to expand near them (the b subzone is for that). Optimizing them will generate short movements in the same zone.

 b) free space, the biggest free space zone on drive to hold expansions of the modified spacehogs

 c) files who don't need big speed for read (< 3Mb/sec), are not modified, are big. Those files can be acessed daily or even more frequently but the speed is not important. Such files are: archives(rar,zip,cab,iso,nrg, etc ), images(jpg,gif etc ), movies, music(mp3,wav,ogg etc ), readonly documents(pdf etc), huge number of other files(inf,cat,txt,nfo,hlp,chm, etc )

 For the "c" zone I was write almost of my posts on this forum.
 The number of masks for a & c is very big and in the today's way of scripting they must be excluded in each selection for the FZ with the inconvenients:
       - the files excluded are vacated somewhere just to free the space, and later to be moved by other rule
       - the same file can be vacated many times before the rule who really selects it. A movie can be recursively vacated(each time more close to the end) until the last rule selects it and put to end (*).

 The big number of types to process for subzones a & c can be handled more convenient from 2 external files:
   - each line a selection, 2 files, or
   - comma separated in one line (1 file, 2 long lines).

(*) to avoid repetead vacations for some files selected to exclude in first rules(fast zone), the files selected to be put in the slow zone must be moved, firstly the c's, next the a's and after this the "fast zone" can be optimized (with a & c excluded).
« Last Edit: March 12, 2009, 08:28:27 pm by teddy » Logged
WindHydra
JkDefrag Hero
*****
Posts: 69


View Profile
« Reply #10 on: March 12, 2009, 08:24:12 pm »

Your optimization strategy is like Fast Optimize, the only difference being an EndOfDrive zone at the very end of drive. And you want to be able to do the EndOfDrive zone before all the other zone.  It might be useful if you can choose which zone mydefrag processes first.

Even if this function is not implemented, you can just make 2 scripts.  One for only the endofdrive zone, and the other for the MFT/directory, Fast files, and space hog zones.  But this can't be done yet because the SetBeginOfZone can't take 2 arguments right now.  You can do SetBeginOfZone(PlusPercentageOfDisk(100),DoNotVacate) to move selected files to the end when the problem is fixed.
Logged
TeDDy
JkDefrag Hero
*****
Posts: 158


View Profile WWW
« Reply #11 on: March 12, 2009, 08:50:41 pm »

Right !

The big diference is the huge number of files wich go to end to make place in the fast zone of drive. This files, once they go to end, they stay here.

I think this is more important than spacehogs.

The line between them is the freespace and the purpose/behaviour of files in them.

Spacehogs are volatiles, very modified.
EndOfDrive very static.

Already running 2 pass defrag. The pain is the big list to include/exclude.
By default, endofdrive can be processed with more priority as spacehogs is and both before other placements.

And this ONLY if 2 macro/keywords are present in the script !

At the moment, mydefrag is not capable to process the first rule who put files to end and then the second rule who put files before the previous finished zone.

I mean it should work with zones who are 6-5-4-0-1-2-3 processed, not just one after the other 1-2-3-4-5-6.

« Last Edit: March 12, 2009, 08:52:59 pm by teddy » Logged
poutnik
JkDefrag Hero
*****
Posts: 1105


View Profile
« Reply #12 on: March 12, 2009, 10:17:53 pm »

Teddy,

it seems like you have very different definition of spacehogs, comparing to Jkdefrag/Mydefrag.

2/3 of my spacehogs were not accessed in last 30 days,
and very most of them are not frequently modified.

Not to confuse people, you should use different terminology for it.
Logged

It can be fast, good or easy. You can pick just 2 of them....
Treating Spacehog zone by the same effort as Boot zone is like cleaning a garden by the same effort as a living room.
TeDDy
JkDefrag Hero
*****
Posts: 158


View Profile WWW
« Reply #13 on: March 13, 2009, 12:36:38 am »

 Spacehogs and endofdrive are files/folders declared by mask. The user can use any valid mask on them.

 I've used the spacehogs word ONLY because the zone has the same position on drive in raport with optimised files for the fast zone and has common files/folders with it.
 With other words, I've splitted the Jeroen defined spacehogs in 2 zones separated by the big freespace.
 The spacehogs "redefined by me" wil be the very modified files wich can expand in the very next free space.
 You are right, I should name it "VOLATILE" and define his place/order to mach to spacehogs.

 My mistake and excuses for the created chaos !

 The endofdrive files "introduced by me" will be the very static percent of files from the old spacehogs zone, never modified or used at low speed. This files should never mix with those used to boot the computer.

Am I right?

Try to understand the "new definition" I make for them and the place on drive I think they should have.

It's bad what I said?

I try to find a way to optimise my files this way in the very simple manner possible.

Until now, I use this optimisation method only by rebooting to w98se and defrag my fat32 drives with SpeedDisk. This is the best one in my opinion.

No other defrag software is able to do this under w2k/XP.

If mydefrag, in the future will be able to do almost same good job is a mistake ?
I think not.

If I am wrong, please accept my apologies.

I try to show a way to achieve a better placement, already invented/used/verified and 5 star clasified.

 Of course, jkdefrag/mydefrak make things a bit diferent, but I supposed Jeroen and other users never see/used SpeedDisk, or never tweaked it, or never take a look on how it works.
 Also it was opensource, now is free and has a friendly forum.
 Some of the ideas are implemented in next versions.
 Many of them are simple and clear.
 Well, mine is not so simple to understand as I can see. That's why I made those loong posts.

 In your case, 2/3 of the spacehogs are moved around when fastfill the zone. But files can be more than 30days older. I deal with files from 1996 on my drives and 99.99% of them feel good to the end of drive.
 If they are just mp3, listened daily should I move them when defrag the spacehog zone ?
 Do they need more read speed that has the last track on drive ?
 Are they modified to get fragmented ?
 Absolutely not, never.

Please correct me if I am mistaking.
Logged
WindHydra
JkDefrag Hero
*****
Posts: 69


View Profile
« Reply #14 on: March 13, 2009, 02:27:22 am »

Can you show me an example of your EndOfDrive selection mask?  I think you are making things more complicated than it should.  Also, don't feel like you need to explain EVERYTHING.  It makes your posts way too long and confusing.

When the SetBeginOfZone function works correctly (hopefully soon), you can just use it to move all the endofdrive files to the end.  And since those files are rarely modified, you rarely need to run this script.  Afterwards, just use the FastOptimize script and replace the space hog zone selection (default is "all") with not("endofdrivefiles") to select the remaining files that are not at the end of drive.
« Last Edit: March 13, 2009, 02:40:43 am by WindHydra » Logged
Pages: [1] 2
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.5 | SMF © 2006-2008, Simple Machines LLC Valid XHTML 1.0! Valid CSS!