Version 4 scripting language

[jeroen]

I am thinking about implementing a scripting language in version 4 of JkDefrag, because I am running into difficulties in implementing new features with the current commandline structure. I need something more flexible. As far as I know there are no other defraggers that offer a scripting language, so it would make JkDefrag totally unique.

I have prepared a first draft of the grammar, available for download at grammar.grm. It's a textfile, you can view it with any text editor. The grammar is intended for the Gold parser. I was wondering if anyone would like to think with me about this grammar?

p.s. This thread is for the version 4 scripting language only. It is NOT for questions if feature XYZ will be available in version 4.

[Lexar]

No defrag software author other than you seem to have hit upon the idea of inplementing a scripting system into a defrag program! (And that, despite the fact that defraggin is a very, ah, complicated, almost philosophical subject, resulting in a wide variety of ideas on how to go about defragging drives.)

Now, how about having the <MaskTypes>,

Code:

<MaskTypes>
::= <MaskType> <MaskTypes>

<MaskType>
::= 'WildCards' | 'RegularExpressions'

[jeroen]

Here is an example of how a script would look like. This example will do exactly the same as the current JkDefrag:

Code:

DebugLevel(1)
ForDisk(Attribute(Mounted) Attribute(Fixed) Attribute(Writable))
  begin
  OptimizeMFTzone
  Place(Attribute(Directory),Fast)
  PlaceFreeSpace(1,1)
  Defragment
  Place(Name("*"),Fast)
  ShowInfo(Summary)
  ShowInfo(UnmovableFiles)
  ShowInfo(FragmentedFiles)
  end

The script begins by setting options. There is only one setting in this example, but several others are available.

The ForDisk function will execute all the commands between Begin and End for the specified disk(s), in this case all the mounted, fixed, writable disks. One of the advantages of using a scripting language is that it becomes possible to make very complex selections, for example all disks that are mounted, but not fixed, smaller than 100 gigabytes, with a name between 'g' and 'r'. Or whatever.

The commands between Begin and End perform defragging and optimization actions. There are some actions that work on all the files on the disk, such as CheckDisk, OptimizeMFTzone, and Defragment, and other commands that will place blocks of files on the disk. Every block can have it's own sorting and optimization options. This makes it possible to begin the disk with a block of directories, then a block of files sorted by boot sequence, then a block of files sorted by last access, then a block at the end of the disk with huge files. Or whatever the user wants.

[jeroen]

Yes, the grammar could be extended with regular expressions to select filenames and disknames. The "'Name' '(' String ')'" line in the grammar is meant for wildcard strings. Adding regular expressions can be done by adding another line, something like "'NameRegExp' '(' String ')'". Other lines can also be added later to the same <selectors> section to quickly select a list of files. For example a statement to fetch filenames from a MySQL database, or whatever.

[Lexar]

Tried writing the script

Code:

DebugLevel(1)
ForDisk(Name("C:") Name("D:"))
  begin
    OptimizeMFTzone
    Place(Attribute(Directory),Fast)
    Place(NameRegExp(".exe$"),SortByName)
    Place(NameRegExp(".dll$"),SortByName)
    Defragment
    ShowInfo(Summary)
    ShowInfo(UnmovableFiles)
    ShowInfo(FragmentedFiles)
  end
ForDisk(Attribute(Mounted) Attribute(Fixed) Attribute(Writable))
  begin
    OptimizeMFTzone
    Place(Attribute(Directory), Fast)
    Place(FragmentSize(1048576), Fast)
    ShowInfo(Summary)
    ShowInfo(UnmovableFiles)
    ShowInfo(FragmentedFiles)
  end  

One thing I am curious about.

If the above is written as one script, will "C:" and "D:" be excluded from or included in ForDisk(Attribute(Mounted) Attribute(Fixed) Attribute(Writable))? Personally, I would like the scripting system to exclude a more specific statement from a less specific, more general statement if the less specific is written below the more specific.

[rdsu]

Could be a great feature!

[jeroen]

If the above is written as one script, will "C:" and "D:" be excluded from or included in ForDisk(Attribute(Mounted) Attribute(Fixed) Attribute(Writable))? Personally, I would like the scripting system to exclude a more specific statement from a less specific, more general statement if the less specific is written below the more specific.

Well, there are two choices: either the engine can be "smart" and make a list of things to do (disks, files) and cross them off when it has processed an item, so a disk or file will only be processed once, or the engine can be "stupid" and simply do whatever matches the script, so the "C:" and "D:" disk will be processed twice in your example. The script programmer would have to add Exclude statements to prevent that, like this:

Code:

ForDisk(
    Attribute(Mounted)
    Attribute(Fixed)
    Attribute(Writable)
    Exclude(Name("C:"))
    Exclude(Name(D:))
    )
  begin
  ...

I think I favor the "smart" method. It will result in smaller scripts, and will make sure things are only processed once. The disadvantage is less freedom for the script programmer, and some head scratching if he doesn't realise that a disk or a file is matched by an earlier statement. Perhaps I should provide both methods through a setting?

[Lexar]

I, too, am in favor of the "smart" as the default method, because with the "stupid", in a series of statements, each one would have to exclude everything that had come before itself, and as a whole, the script would be quite lengthy and difficult to manage.

[NickyZ]

The problem with 'smart' systems is as a user you never know what they are going to do, and over time the rules tend to change which makes things worse.

I would much rather see the script language designed such that it is unambiguous and relatively straightforward to design a script that does what the user wants with no 'smarts' from the software.

Targeting novice users would not be my priority since the current modes work just fine. If someone has the knowledge and wants to write a script then let them learn it and take responsibility for the results.

[Lexar]

Making it smart doesn't mean making it ambiguous. What is important is that if it is going to be made "smart" or "stupid", either way, it has to be made so all the way.

Code:

DebugLevel(1)
ForDisk(Attribute(Mounted) Attribute(Fixed) Attribute(Writable))
  begin
    OptimizeMFTzone
    Place(Attribute(Directory),Fast)
    PlaceFreeSpace(1,1)
    Defragment
    Place(Name("*"),Fast)
    ShowInfo(Summary)
    ShowInfo(UnmovableFiles)
    ShowInfo(FragmentedFiles)
  end

The above script code, originally an example by Jeroen,  is reasonably assumed to be "smart" from "begin" to "end". Directories get treated at "Place(Attribute(Directory),Fast)" and they will not be treated again at "Place(Name("*"),Fast)".

If <commands> are treated the "smart" way, so should the <statements> and all the others. Making it "smart" in one part and "stupid" in another will inevitably make it ambiguous as a whole.

[waika]

Wowow! This is exciting! With script like this one could configure JK Defrag behave like literally any commercial defrag application (or better).

I like what Lexar implies by suggesting that it be kept as unambiguous as possible regardless of approach...

 

[jeroen]

Well, my primary objective is a scripting language for programmers, so the grammar must be somewhat easy to read. XML is a format for transporting data between computers and programs, not very well suited for a scripting language.

GUI wrappers and other interfaces can easily generate defrag scripts by taking a pre-fabricated template and filling in values. Reading back and modifying a script might be a bit difficult, because parsing the script requires intricate knowledge of the grammar. That's where XML might come in handy, to store a script as individual tokens. Perhaps I can make the interpreter accept both flat text and XML, but I'm not sure about that yet.

[waika]

Ahh, you saw my post while I was editing it, where it was going, and took it to all it's logical conculusions... After rereading your topic post I wanted to spare the thread the XML discussion.

I imagine more then just Programmers will want to use the scripting language; a so called 'Power User', or somone that just wants to tweak his file layout using JKdefrag 4.x will obviously be confronted with an enormous number of options and ways of getting it done via script -- even if just editing predefined scripts.

Easy to read grammar will be very valuable to those of us that are terrible hacks when it comes to programming...

 :shock:

[alys]

hi!
I have few remarks for your scripting language.
1. pls put some comments to articles in your grammar file,  because a lot of times it is not clear - what a statement or predefined word means.
2. Are there incompatible statements, sentenses or blocks, is there a possibility to create incorrect script file, and how you want to manage incorrect scripts?... it is clear that you should pass all script at a start of programm to check its correctness, to avoid a crash after one hour of disk operstions. so you should have some programm option - test the script(looks like "compile the script" and report about errors).

3.in your grammar the order of "statements" and "commands" is not fixed. so questions are.

is there some nonsense if "logfile" is placed after "forDisk"?

or we have some intersected statements

LogFile ("A.log")
bla..bla...
LogFile ("B.log")
what log is actual?

or may be you want to redirect a logoutput to another file, so some job was done with one log and the rest of job - another log?

or intersected commands
PlaceFreeSpace( Number1,Number2)
PlaceFreeSpace( Number3,Number4)

in your grammar you have undefined number of commands and statements which actually should be only single in current block or script.

[jeroen]

1. pls put some comments to articles in your grammar file

Thanks for your excellent remarks, I appreciate it!

Yes, I will add some comments when I get back from holiday.

2. Are there incompatible statements, sentenses or blocks, is there a possibility to create incorrect script file, and how you want to manage incorrect scripts?

I have tried to make the grammar as unambiguous as possible... The interpreter engine will check the full script for syntax errors before execution. I actually already have the engine for that, see http://www.kessels.com/Gold/index.html. Syntax errors will generate an error message with line and column position.

is there some nonsense if "logfile" is placed after "forDisk"?

The "logfile" command is a setting. After the command all messages are placed in the new logfile. If the command occurs again then from that point on the messages are placed in the new logfile.

in your grammar you have undefined number of commands and statements which actually should be only single in current block or script.

I'm not sure what you mean. The script will be executed from top to bottom in sequence. The PlaceFreeSpace command will fully execute, and only then the next command will be executed.