Version 4 scripting language

[maurizio]

I think you should really have a look to Lua as a scripting language.
you can embed the entire compiler/interpreter in your application
in only about 50 kb.
you don't need to write an interpreter or scanner, it's just included.
You don't need to handle houskeeping work, like
if, loop handling, case, for, function definitions and calls, etc..
it's all just included.

You need only add your language statements to the language:
 OptimizeMFTzone
  Place(Attribute(Directory),Fast)
  PlaceFreeSpace(1,1)
  Defragment
  Place(Name("*"),Fast)
  ShowInfo(Summary)
  ShowInfo(UnmovableFiles)
  ShowInfo(FragmentedFiles)

you can even write only the low level routines in C, and write all the logic
of the work in Lua, or leavig such decision to the user.
You (or the user) get unlimited and automatic array management,
advanced language structure,
variable definition, routines and function definitions,
local and global variables, co-routines,
tables (dictionaries) handling,
automatically garbage collection, and much more...
you (or the user) don't need to write any statement to get this...
a lot of work saved...


What you have to do to get this is to create your basic routines in C
and, during initialization, letting Lua knows of their existance.

Your functions can be called directly by an embedded or user script.
Please have a look at it before committing yoursef to the hard work.

Thanks for the wonderful program.

Regards.

Maurizio.

[blackt1ger]

Well, I know that you are two months into this thread....  But you should consider using Lua51.  I'm currently working on a Lua version of stored procedures to a custom database system that I'm designing (ie you would write stored procedures in Lua).

Lua is:
a. Open Source
b. Has a C interface to host the language
c. compiles cleanly in win32/x64
d. has Visual Studio projects project/solution files
e. dll, lib, or embedded - extreme flexibility, just like JkDefrag.

[jeroen]

I think you should really have a look to Lua as a scripting language.

Thank you very much for your suggestion, I appreciate it! I haven't started work yet and I'm not exactly sure where I will end up. What you are asking for is basically a library with lot's of function calls, something like the GD library. Such a library could be used from within many programming languages, not just Lua.

I want to replace the JkDefrag commandline with something more flexible, a script that is executed by the stand-alone JkDefrag program. But when I have something that interpretes and executes such a script, then why offer a big library with lot's of function calls? A single function call will be a lot easier to add to any language, far less porting problems. And I will be able to give support, even if I do not know that particular programming language.

So I am back at the question at which language I should use for the JkDefrag script. I suppose I could use Lua. But then every programmer would have to learn Lua (including myself), which is a far steeper learning curve than a simple, small, dedicated language with a small set of commands. And it seems a bit silly to me to force programmers to learn Lua just to make a defrag script. Inserting Lua into other programming languages might even be dangerous, it might open up cross-scripting vulnerabilities.

That's why I am thinking of a small and dedicated scripting language. A model would be the SQL language, designed for a specific task, relatively small and easy to learn, and can be used from within virtually all programming languages.

[Lexar]

Lua is:
a. Open Source
b. Has a C interface to host the language
c. compiles cleanly in win32/x64
d. has Visual Studio projects project/solution files
e. dll, lib, or embedded - extreme flexibility, just like JkDefrag.

Looks like one would be able to write his own defragger in Lua. I am afraid it would be too big a thing for me to grasp.

Well, Luaphiles can write a script to generate scripts for JkDefrag, no matter what scripting system Jeroen Kessels may employ.

[celeri]

Wow that looks very cool!

The thing that really worries me however is a complete newbie messing up the scripts and then blaming the program/programmer for it not working properly.

So I forsee some concurrent avenues;
1. Make the script hard to reach for the regular user (no GUI button to edit)
2. Make the interpreter VERY FUSSY; if it doesn't look right than JKDefrag will not take a chance and run it.
3. Have a bunch a pre-made scripts that will answer most of the user's requests.
4. Have a pop-up window warning users about using custom scripts the first time they do it.
5. Lace the documentation with practical examples!

[Fox!MURDER]

Well, I found out about JKDefrag just about yesterday, but like it very much.  IMO it's the same to learn Lua as it is to learn any other language, even easy and specific. You don't need to learn every detail of the language to write a simple script (and I won't learn every detail). So the learning curve is not that steep. On the other hand, if you use some other tools (like elinks browser, or the game of World of Warcraft (yes famous game by blizzard uses lua as it's scripting language)), that uses lua, you don't have to learn new language. You just learn the exported functions. So it seems to make more sense to to me not to reinvent the wheel. Anyway just my $.02.

And to the subject of your grammar ... would it be possible to get two more placement options? Something like 'Middle' and 'LeaveRoom' ...
'Middle' placing stuff in the middle of the *physical* drive, where the mean seek time is shortest.
'LeaveRoom' leaving some space (i guess some % of the file size) just after the file so as to make it not fragment at the time of first append.

[jeroen]

'Middle' placing stuff in the middle of the *physical* drive, where the mean seek time is shortest.

Thanks for your sharing your ideas, I appreciate it! I'm not sure about seek times being the shortest in the middle of the drive, I think it's only true if a disk is 100% full and if all tracks are of equal size. If a disk is only 10% full then I'm sure seek times will be shortest if all data is placed at the beginning of the drive.

'LeaveRoom' leaving some space (i guess some % of the file size) just after the file so as to make it not fragment at the time of first append.

A fine idea, but the vast majority of files never change, so it would be a big waste of space. Also, Windows already leaves a bit of room after every file, it rounds filesizes up to "clusters" of 4096 bytes (can be set when the drive is formatted). I am thinking about inserting some room between directories, though.

[Fox!MURDER]

Thank you for your quick answer. I understand your points, but i guess my points deserve a little bit of clarification.

'Middle' placing stuff in the middle of the *physical* drive, where the mean seek time is shortest.

Thanks for your sharing your ideas, I appreciate it! I'm not sure about seek times being the shortest in the middle of the drive, I think it's only true if a disk is 100% full and if all tracks are of equal size. If a disk is only 10% full then I'm sure seek times will be shortest if all data is placed at the beginning of the drive.

well then its going to be in the middle of used blocks ... so lets take the distribution of data on the drive and put it in the middle ... I'm especially thinking about directory data and stuff like swap space. Moreover even 10% full disk doesen't have all blocks in the begining (with the exception with fresh new filesystem) as long as you don't intentionaly 'optimize' them that way. But if you do that, you probably should use the 'LeaveRoom', so growing any file doesen't fragment your FS right away.

'LeaveRoom' leaving some space (i guess some % of the file size) just after the file so as to make it not fragment at the time of first append.

A fine idea, but the vast majority of files never change, so it would be a big waste of space. Also, Windows already leaves a bit of room after every file, it rounds filesizes up to "clusters" of 4096 bytes (can be set when the drive is formatted). I am thinking about inserting some room between directories, though.

Well i got this idea from the article about unix filesystems and fragmentation. I'm not intending to use it on all files but rather on files, that tend to grow regularly (like my inbox, logfiles, registry, etc.) as there isn't too many of them. So it shouldn't be that much of a waste of space.

If I were good enough coder, I'd offer my help with these features. But as long, as i'm just a newbie in C, i have to stick with supplying ideas.

[jeroen]

well then its going to be in the middle of used blocks ... so lets take the distribution of data on the drive and put it in the middle ...

If the disk is 10% full, and all the data is optimized together in a single big block, then seek times will be considerably faster if that block is placed at the beginning of the disk, not in the middle.

I'm not intending to use it on all files but rather on files, that tend to grow regularly (like my inbox, logfiles, registry, etc.) as there isn't too many of them.

Yes, the JkDefrag script will support that.

[Fox!MURDER]

well then its going to be in the middle of used blocks ... so lets take the distribution of data on the drive and put it in the middle ...

If the disk is 10% full, and all the data is optimized together in a single big block, then seek times will be considerably faster if that block is placed at the beginning of the disk, not in the middle.

I'm talking about middle of the datablock.
I mean something like ... you have partition with blocks 1-1000. blocks 1-100 are used 101-1000 are free ... and you have lets say 10 blocks of data, you want with fast seek ... then i suggest putting them in blocks 45-54 instead of 1-10 ... and still there are some things you access independently from everything else on the disk (like swap) that in some situations better lie in the middle of the physical disk (or as close as possible to it).

btw. that should be the power of scripting. Let user choose what's best for him. ie. if he wants data in the middle, let it be in the middle.  (i can even imagine script which will decide about placement of the blocks based on the usage of the partition)

[celeri]

Well I might be wrong but to my knowledge tracks are faster at the beginning of a hard drive, hence the idea is to put rarely read/modifiend files at the end of the drive and leave the often read/modifiend files at the beginning.

Check out this program to prove my point: HD Speed (free, of course!)
http://www.steelbytes.com/?mid=20

So I'm not sure about this center thing. If it was me, I'd put the recycling bin and temporary files there; they change a lot but do not require that much speed (them not being fast does not affect the user experience). The swap file, frequently used OS files, directories and the important stuff would be at the beginning.

Installation files like I386 & Cabs, the DLLcache and backup registry would go at the end, where speed is least important.

All the personnal files would, ideally, be on another drive or at least another partition.

But then, as Jeroen says, that's why scripting is in the air; to each is own eheheh. Now if only we could script religion or politics what a better planet this would be  :lol:

[jeroen]

I'm talking about middle of the datablock.
I mean something like ... you have partition with blocks 1-1000. blocks 1-100 are used 101-1000 are free ... and you have lets say 10 blocks of data, you want with fast seek ... then i suggest putting them in blocks 45-54 instead of 1-10

Yes, in that case the 10 blocks are best placed in the middle if the 100 blocks. I don't have to add anything special to the scripting language I am proposing, because it is based on placing blocks of files one after another on disk. For each block the user specifies exactly which files it contains, and how they must be sorted.

[jeroen]

Check out this program to prove my point: HD Speed (free, of course!)
http://www.steelbytes.com/?mid=20

Interesting link, thanks! It would be more useful for my purposes if it plotted a graph showing the speed of the disk from begin to end, though.

[jeroen]

Just found another disk speed measurement. It measures the disk transfer rate by cylinder, showing that the begin of the disk is faster than the end. It's a little off-topic, but what the hell:
http://www.geocities.com/vgrinenko/DiskSpeed32/

[jeroen]

I have just uploaded the latest version of the grammar, same url at grammar.grm. The grammar is totally different from the old version. Here is an example of how a JkDefrag script would look like using this grammar:

Code:

# JkDefrag example script v4.0

Slowdown(90.0)
DebugLevel(6)
LogFile("xx.log")
AllowMultipleInstances(no)

VolumeSelect
  Mounted(yes) and
  Removable(no) and
  Writable(yes) and
  Size(2,60000000000)
VolumeActions
  ShowTitle('Checking the volume')
  CheckVolume

  ShowTitle('Reclaiming MFT zones')
  ReclaimMFTzone

  ShowTitle('Defragmenting the MFT')
  FileSelect
    Name("?:\$mft")
  FileActions
    FastFill
  FileEnd

  ShowTitle('Defragmenting and optimizing directories')
  FileSelect
    Directory(yes)
  FileActions
    SortByName(ascending)
  FileEnd

  PlaceFreeSpace(1.0,1.0)

  ShowTitle('Defragmenting and optimizing regular files')
  FileSelect
    Size(0,50000000) and
    not(Name("*.zip")) and
    not(Name("*.arj")) and
    not(Name("?:\pagefile"))
  FileActions
    Defragment
    FastFill
  FileEnd

  PlaceFreeSpace(1.0,1.0)

  ShowTitle('Defragmenting the pagefile')
  FileSelect
    Name("?:\pagefile")
  FileActions
    FastFill
  FileEnd

  ShowTitle('Defragmenting and optimizing the SpaceHogs')
  FileSelect
    All
  FileActions
    Defragment
    FastFill
  FileEnd

  ShowInfo(Summary)

  FileSelect
    Unmovable(yes) and Largest(25)
  FileActions
    ShowInfo(Summary)
  FileEnd

  FileSelect
    Fragments(1,0) and Largest(25)
  FileActions
    ShowInfo(Summary)
  FileEnd

  FileSelect
    Largest(25)
  FileActions
    ShowInfo(Summary)
  FileEnd

  ShowTitle('Finished')

VolumeEnd