Version 4 scripting language

[schitzn]

Theres been little updates on version 4's progress.

I've got some questions and feedback.

Firstly, I say the following as a fellow programmer, with concern for how implementation may affect the performance of JKDefrag, please do not take this as criticism, rather just a second perspective on your concept of creating a scripting language.

 ShowTitle('Defragmenting and optimizing regular files')
  FileSelect
    Size(0,50000000) and
    not(Name("*.zip")) and
    not(Name("*.arj")) and
    not(Name("?:\pagefile"))
  FileActions
    Defragment
    FastFill
  FileEnd

For each "not(Name("individualfile))"  statement in your scripting language, Im imagining the preprocessing prior to making the decision to move a file would be a significant CPU hog, based on alone the time to compare two strings, and furthermore with flags like "?" "*" .  What happens when this list becomes a large list of individual files specified for a zone, Every file on the volume has to suffer the same pre-processing to determine if it is to be moved.

I don't do a lot of C++ programming, but I know some of the language, as I code primarily in VB and ASM. In VB, the * ? flag support isn't built in support for native String functions, and I assume it to be the same for C++.

 ShowTitle('Defragmenting the pagefile')
  FileSelect
    Name("?:\pagefile")
  FileActions
    FastFill
  FileEnd

The zone seems to be allocated by a PlaceFreeSpace(1.0,1.0) before and after , which I assume is a % of total capacity.  My concern is that on a daily defrag under this routine, if the before zone resizes, the pagefile zone will also need to shift, and this being a single chunk to move, the benefit to move the file outweighs the time/disk wear.


Again, these are just concerns I would have if I was to attempt this programming feat myself, I know personally a 2nd perspective can bring issues to light that otherwise may go unconsidered

I have some possible suggestions, but I would like your feedback on my highlights before I blab on.

[jeroen]

Theres been little updates on version 4's progress.

Yes. The new NTFS scanner that was introduced in v3.17 was a prerequisite for v4 and had to be finished first.

Im imagining the preprocessing prior to making the decision to move a file would be a significant CPU hog

Yes. It will depend on the number of masks, ofcourse. The language is designed so that the selections can be done first, building a list of files that match, and then the list is handed to the defragger/optimizer.

The zone seems to be allocated by a PlaceFreeSpace(1.0,1.0) before and after

No. The size of the zone is determined automatically. The parameters to PlaceFreeSpace are a minimum/maximum. The free space will only be adjusted if it is smaller than the minimum or larger than the maximum.

[Cap1]

Here is an example of how a JkDefrag script would look like using this grammar:

Code:

# JkDefrag example script v4.0

Slowdown(90.0)
DebugLevel(6)
LogFile("xx.log")
AllowMultipleInstances(no)

VolumeSelect
  Mounted(yes) and
  Removable(no) and
  Writable(yes) and
  Size(2,60000000000)
VolumeActions
  ShowTitle('Checking the volume')
  CheckVolume

  ShowTitle('Reclaiming MFT zones')
  ReclaimMFTzone

  ShowTitle('Defragmenting the MFT')
  FileSelect
    Name("?:\$mft")
  FileActions
    FastFill
  FileEnd

  ShowTitle('Defragmenting and optimizing directories')
  FileSelect
    Directory(yes)
  FileActions
    SortByName(ascending)
  FileEnd

  PlaceFreeSpace(1.0,1.0)

  ShowTitle('Defragmenting and optimizing regular files')
  FileSelect
    Size(0,50000000) and
    not(Name("*.zip")) and
    not(Name("*.arj")) and
    not(Name("?:\pagefile"))
  FileActions
    Defragment
    FastFill
  FileEnd

  PlaceFreeSpace(1.0,1.0)

  ShowTitle('Defragmenting the pagefile')
  FileSelect
    Name("?:\pagefile")
  FileActions
    FastFill
  FileEnd

  ShowTitle('Defragmenting and optimizing the SpaceHogs')
  FileSelect
    All
  FileActions
    Defragment
    FastFill
  FileEnd

  ShowInfo(Summary)

  FileSelect
    Unmovable(yes) and Largest(25)
  FileActions
    ShowInfo(Summary)
  FileEnd

  FileSelect
    Fragments(1,0) and Largest(25)
  FileActions
    ShowInfo(Summary)
  FileEnd

  FileSelect
    Largest(25)
  FileActions
    ShowInfo(Summary)
  FileEnd

  ShowTitle('Finished')

VolumeEnd

Thats looks rly good .

Would it be possible to defrag a complete Specific Directory, incl. Subdirs and files, in a block at the beginning of zone 2? My idea improving loading times of Games that have files that are bigger as 50mb( ! ) and contains *.zip files ( BF2, Eve-Online).

Thanks alot for this tool . If scripting is possible, to code  for own needs, it would be a killer application

Best Regards

CaPoNe

[jeroen]

Would it be possible to defrag a complete Specific Directory, incl. Subdirs and files, in a block at the beginning of zone 2?

The scripting is designed to specify exactly what files go into a zone, and how they should be optimized. You can define as many zone's as you want. It replaces the 3 zone's that JkDefrag has now.

[elfring]

I dare to tune in to this discussion a bit late. I hope that another point of view is still useful. :idea:

I am thinking about implementing a scripting language in version 4 of JkDefrag, because I am running into difficulties in implementing new features with the current commandline structure. I need something more flexible. As far as I know there are no other defraggers that offer a scripting language, so it would make JkDefrag totally unique.

So I am back at the question at which language I should use for the JkDefrag script. I suppose I could use Lua. But then every programmer would have to learn Lua (including myself), which is a far steeper learning curve than a simple, small, dedicated language with a small set of commands. And it seems a bit silly to me to force programmers to learn Lua just to make a defrag script. Inserting Lua into other programming languages might even be dangerous, it might open up cross-scripting vulnerabilities.

That's why I am thinking of a small and dedicated scripting language. A model would be the SQL language, designed for a specific task, relatively small and easy to learn, and can be used from within virtually all programming languages.

I imagine that a powerful C++ class library will be needed for its implementation.
How do you think about to reuse the tool "Simplified Wrapper and Interface Generator"? How much can it save efforts for your imaginations and requirements by automatic API generation?

[jeroen]

I imagine that a powerful C++ class library will be needed for its implementation.

The project so far has zero C++ code. I don't like OOP and try to avoid it wherever possible.

How do you think about to reuse the tool "Simplified Wrapper and Interface Generator"?

Thanks for your suggestion, I appreciate it! It looks very interesting. However, the JkDefrag "api" will consist of just a single function, so it would be overkill to use SWIG.

[Myriades]

Hi jeroen
correct me if i'm wrong, i think u missed the moveup function in fileaction.

and so, u could add a function like that
moveto(lcnstart | auto , lcnend | auto) only 1 auto authorized
this with an aim of creating diffrent zone starting or ending where we want

Cu, Laurent

[jeroen]

correct me if i'm wrong, i think u missed the moveup function in fileaction.

Very observant of you, my compliments! I left it out to simplify the program, and because I think it's not needed any more. It will be possible to define zones that are not defragmented and/or optimized.

and so, u could add a function like that
moveto(lcnstart | auto , lcnend | auto)

See "SetBeginOfZone". But it will not be allowed to place a zone before a zone that has already been placed, zone's must be defined in the script in the same sequence as they are placed on disk.

[elfring]

The project so far has zero C++ code. I don't like OOP and try to avoid it wherever possible.

If you want to be so consequent, you should use the file extension ".c" instead of ".cpp".
I find it strange that you do not want to benefit from advantages by object-oriented design. I suggest also to try aspect-orientation to save some software development efforts.

However, the JkDefrag "api" will consist of just a single function, so it would be overkill to use SWIG.

I guess that one function will not be enough for a complete application programming interface because of various user requirements. There are a lot of design factors to consider for interesting functionality.

Another promising approach for file defragmentation selection and optimisation is provided by rule engines.

[Myriades]

Hi jeroen

Are u sure u want to leave it out? :cry:

Sometimes, i think it will be necessary to move all the data to the end of the drive. this to restore the original size of the reserved space of the mft -> the mft will not frag if there is enough reserved space  :wink:

CU, laurent

[TAC109]

I too would like to put in a request to keep the moveup function in version 4 of JkDefrag. It is one of the unique features in JkDefrag that differentiates your program from all the others. And I have found it very useful at times!

[jeroen]

Are u sure u want to leave it out? :cry:

Well, let's see if we can find a good reason why it should be included.

this to restore the original size of the reserved space of the mft

But the MFT reserved area is way to big. Default is 12.5% of the disk. My laptop has a small 28Gb disk, the MFT is 125Mb (110000 files), or 0.44% of the disk. That's 12% of the disk wasted!

JkDefrag v4 will have the capability to defrag the MFT (on Vista). So there is no need anymore for a move-to-end-of-disk trick. Also, even the current version of JkDefrag will move files out of the reserved zone (reclaiming the zone for the MFT). If you run JkDefrag every day they the MFT should never get fragmented.

[jeroen]

I have found it very useful at times!

Can you give an example? I have designed the scripting language to be very flexible, and I think it will be able to handle all needs without first having to move something to the end of the disk.

[Myriades]

But the MFT reserved area is way to big. Default is 12.5% of the disk.

self-evident truth

That's 12% of the disk wasted!

right

kDefrag will move files out of the reserved zone (reclaiming the zone for the MFT)

Oups, i missed it

If you run JkDefrag every day they the MFT should never get fragmented.

Right again
then, ok, leave it out. you convinced me

other subject, is there a new version of the grammar? I can't find the SetBeginOfZone option

CU, Laurent

[jeroen]

is there a new version of the grammar? I can't find the SetBeginOfZone option

Ah! Yes, there is, and I will upload it right now. I don't know exactly when I added SetBeginOfZone, but it was not in the grammar on the website.