SetBeginOfZone Absolute PERCENTAGE

[jimbo]

As I've mentioned at
http://www.kessels.com/forum/index.php?topic=1688.0
I am working on this.
I am advancing, slowly to reconstruct the API from assembly. Trust me, it is hard to dissect and trace the kernel32 and fat drivers. A lot of dependency's must be solved. All this in assembler.

When you first mentioned your plan to make FAT folders moveable, you may remember that I pointed out to you that it would be a breach of the MicroSoft EULA to reverse-engineer or decompile their code, and therefore would be illegal for you to do, and, further, illegal for Jeroen to use the results of your disassembly in any way whatsoever.

You stated quite clearly that you had no intention of doing such, but would instead be working forwards to try to find a way to "unlock the folders".

You are now stating again that you are tracing the MS code, and reconstructing the api from the assembly. This sounds very much to me like you are reverse-engineering the code, and I must again warn you that, if that is the case, then you are breaking the law, and Jeroen would risk being sued by MicroSoft if he used any code whatsoever that you should contribute.

[TeDDy]

 I am not intending to publish reversed api. But I must look to it to avoid same mistakes, still moving safely files and folders.
 I trace it low level because I can't figure out what line of code refuses the movement in other approved debuggers.
 So I reconstruct temporary the used functions to trace them closely.
 Can you reccomend me another approach to debug into active processes and modify result of variables ?
 I think I can look into something I've buyed. The punishment is if I use it.

 Do you think I must close my eyes whem I debug my own programs in FreePascal because the assembly code on the screen is in the middle of OpenFile crashed function, property of MS ? Try a search for OpenFile+assembly on google to find free assembly code then look in what reversed you want. If you see the differences in the first 5 minutes, bravo to you. Still not the same code or behaviour or results, but same purpose.

 If you will learn assembly and wish to write a function to open a file, better&faster, please pay to MS because you have used for sure somewhere same instruction, not because you wish to steal but because the CPU's have a finite set of instructions available.

 Depending on the result, we can complain to M.S. to repair it, or I can figure how to make, safely, my own.
 Why do you judge me before see the result ?
 Do you think I want SECRETLY compromise MyDefrag or Jeroen ? Well, NO WAY !!!
 When I finish, if I finish, I will ask Jeroen first and after that I let him see what I've found, to avoid discussions.

 Before judging me or reclaim me to MS, please, wait the results.
 I will explain in detail the used steps if this will be needed.
 Punish the result if appliable, NOT THE INTENTION.

[jimbo]

I am not intending to publish reversed api. But I must look to it to avoid same mistakes, still moving safely files and folders.
 I trace it low level because I can't figure out what line of code refuses the movement in other approved debuggers.
 So I reconstruct temporary the used functions to trace them closely.
 Can you reccomend me another approach to debug into active processes and modify result of variables ?
 I think I can look into something I've buyed. The punishment is if I use it.

You may think what you like. All I can tell you is what MicroSoft's lawyers will think, and that is that you will be violating their licence.

Do you think I must close my eyes whem I debug my own programs in FreePascal because the assembly code on the screen is in the middle of OpenFile crashed function, property of MS ? Try a search for OpenFile+assembly on google to find free assembly code then look in what reversed you want. If you see the differences in the first 5 minutes, bravo to you. Still not the same code or behaviour or results, but same purpose.

Try googling "clean room reverse engineering" - it may help you to understand the point that I am trying to get across. It is not about whether your code looks similar, or performs the same function, but about how you learned to do what you are doing, and how you developed your understanding of the tasks involved in (for example) opening a file.

If you will learn assembly and wish to write a function to open a file, better&faster, please pay to MS because you have used for sure somewhere same instruction, not because you wish to steal but because the CPU's have a finite set of instructions available.

I first coded in assembly language in 1981 (on a Z80a CPU if you are interested).
I first decompiled someone else's code to see how they had done something in 1982 (that was for 6502 though). If I had commercially produced an application that did the same based on their techniques, I could have been sued by them.

Depending on the result, we can complain to M.S. to repair it, or I can figure how to make, safely, my own.
 Why do you judge me before see the result ?

I am not judging you at all. I am trying to ensure that you are aware of the implications and ramifications of the methods you are using, so that you can ensure that you do this in the best possible way that maximizes the chance of a useful and productive outcome.

Do you think I want SECRETLY compromise MyDefrag or Jeroen ? Well, NO WAY !!!

Not at all! I fully believe that you have only the best interests of Jeroen and MyDefrag in your mind. It is because I believe this that I am trying to ensure that you do not waste effort on producing code that is tainted and unusable, where you could have possibly acted differently and reached the same place by legally acceptable means that meant that your code and suggestions could have been used.

When I finish, if I finish, I will ask Jeroen first and after that I let him see what I've found, to avoid discussions.

 Before judging me or reclaim me to MS, please, wait the results.
 I will explain in detail the used steps if this will be needed.
 Punish the result if appliable, NOT THE INTENTION.

When you have reverse engineered the product with access to and reference to the code, then the results are tainted by the methods used. You seem to be very dedicated to this task, and are coming up with many potentially hopeful avenues to explore. It would be a terrible shame if you were to find a solution to this, only for it to have to be dismissed due to legal issues.

[TeDDy]

 Ok, try to forget what I was write until now.
 Please tell me the right way people use to discover the malicious/hidden code in the applications they buyed legally, seem to have strange behaviour and the producer
- is too lazy to see it before you or
- don't answer to you, or
- don't repair it, knowing about his presence
- forcing you to upgrade (and pay this) to the buggy program.

I am not joking, really don't know, please tell me !

 Also, I am sure, as a programmer you have made programs with licenced/payed compiler and with a free/gpl one. A simple function to copy a file from hdd look differently written in delphi/Studio/FreePascal. After compilation, at runtime, in the memory, the routine to copy the file is the same between them and for sure with the MS copy routine. Except some different names/variables, the extracted routine from the decompilation(reverse eng.) can be suspected to be illegal ?
 If I am learning from this disassembled, personal, free compiled copy program to write right in assembly copy routines more advanced am I suspect of fraude ?
 Is there a law who forbids looking at runtime in my RAM to see what's running and how ?
 If I find this way a bug present(accidentally or not) and let MS know I found it, and suggest it how to fix it I can go to jail ?
 Perhaps I am wrong thinking at the first defrag for dos, by Symantec (much before MS did one from my knowledge) or any other defrag made since then. The low level move of data must be done by the same assembly routines. It's no other way for that.
I am not joking, really don't know, please tell me !

Well I don't like to buy a car and to be forced not to open its cover if I it smokes or makes noise in the warranty period or after to not be accused to steal his secrets of fabrication. If I can avoid flames, accidents or degradations, I will open it.
 If the producer tell me after buyng it I can't listen radio in 5'th speed because of some limitation, and can't return my money, don't want to solve it, offering me an upgrade, I search a guy/service to solve the bug or unscrew the car myself, searching the bug.

I am not joking, really don't know , please tell me !

 I looked at
http://ethics.csc.ncsu.edu/intellectual/reverse/study.php
http://en.wikipedia.org/wiki/Clean_room_design

 It's about product duplicating for comercial purposes and competition.
 I am not looking in kernel to duplicate it.
 Just to see if a sand particle was left in the engine by mistake or MS glue it there.

 Obviously the function able to move folders like are moved files is not present, not working properly or bypassed.
 If I look and inside and:
- no function or empty: making new one is legal.
- not working: unlocking it should be legal, can be done by programmer externally or MS by bugfixes or can be removed by MS, going to previous point.
- bypassed: deliberately or not. After see why, ask MS, analize the safety to not bypass it and if safe ose it like a pen found on the empty street, at your own risk to corrupt data.

 Please correct me and give me advices, because I can't see the fraud here, now you forced me to verbose it,  revealing what I intend to do, maybe prematurely and without results(yet).

[TeDDy]

As you can see, followed by the $MFT at LCN=2 doesn't happened on 2 test drives + a real one.

I guess I got confused by your earlier postings, I do not always understand what you write. Could you please run the script with the "-d 5" commandline parameter and post the debug logfile here on the forum? Only up until the program starts placing the directories.

Sorry causing confusions. It's not my intention !
 I admit my english is not as it should be.
 I have also the "engineer disease":
- the similarity between the engineer(me) and a dog (intelligent eyes but has trouble to express myself).

 I will run 'MyDefrag.exe -v f: -d 5 FastOptimize.MyD' on the same drives, when I go home. The used version wah 4.0b4(I see today the b6). Before posting, I will try this one too.
 Before each run, usually I delete the dat, log and debug files to avoid increasing, slow run and....
 I am getting almost same results after succesive runs.
 Is this OK to you ?

Only up until the program starts placing the directories.

You mean I must stop the program when it starts move folders and post this log
or
cut/truncate the attached debug logfile to make it smaller ?

Don't know if I have the time to complete it today. I had a hard day, it's late, I wait your answer and must run it on the same drive from wich I post the LCN's(home drive).

[jeroen]

Before each run, usually I delete the dat, log and debug files to avoid increasing

No need for that, none of these files increase.

You mean I must stop the program when it starts move folders and post this log

Yes, that will be fine.

[jimbo]

Ok, try to forget what I was write until now.
 Please tell me the right way people use to discover the malicious/hidden code in the applications they buyed legally, seem to have strange behaviour and the producer
- is too lazy to see it before you or
- don't answer to you, or
- don't repair it, knowing about his presence
- forcing you to upgrade (and pay this) to the buggy program.

Legally, you cannot. It is as simple as that. The term is caveat emptor,  buyer beware. Your only course of action is either to not buy the product in the first place, or, in the case of a truly severe flaw, you could attempt to prove in a court of law that the product was "not fit for purpose", meaning that it was actually impossible to perform the function for which the product was primarily designed using it.

Other than that, you're stuck with it.

Also, I am sure, as a programmer you have made programs with licenced/payed compiler and with a free/gpl one. A simple function to copy a file from hdd look differently written in delphi/Studio/FreePascal. After compilation, at runtime, in the memory, the routine to copy the file is the same between them and for sure with the MS copy routine. Except some different names/variables, the extracted routine from the decompilation(reverse eng.) can be suspected to be illegal ?
 If I am learning from this disassembled, personal, free compiled copy program to write right in assembly copy routines more advanced am I suspect of fraude ?

Every single individual case of this has to be judged separately. If the owner of the code chooses to sue you, it is up to them to prove in court that significant portions of your product were copied from theirs. Then it is down to the judge / jury to decide whether or not you copied what is, in their opinion, too much.

Is there a law who forbids looking at runtime in my RAM to see what's running and how ?

If the licence agreement for a running piece of sorfware forbids you from doing so, and you have agreed to that licence and are running the software, then yes, there is. You have entered into a contract with the licensor and you are in breach of that contract.

If I find this way a bug present(accidentally or not) and let MS know I found it, and suggest it how to fix it I can go to jail ?

In an extreme case, you could be sued by them for using their product in an unlicenced manner, and charged with copyright theft. It is unlikely that it would happen, though.

Perhaps I am wrong thinking at the first defrag for dos, by Symantec (much before MS did one from my knowledge) or any other defrag made since then. The low level move of data must be done by the same assembly routines. It's no other way for that.
I am not joking, really don't know, please tell me !

The very first defrag programs for dos did indeed look at the disk, look at the fat, decide what to do, copy data from one place to another, and manually update the FAT entries to point at the new locations. However, none of the system calls that were used to do this in DOS exist / work any more in modern windows operating systems. You could learn how to write a totally off-line defragmenter program, perhaps, but not one that would work under a windows NT-class system.

Well I don't like to buy a car and to be forced not to open its cover if I it smokes or makes noise in the warranty period or after to not be accused to steal his secrets of fabrication. If I can avoid flames, accidents or degradations, I will open it.
 If the producer tell me after buyng it I can't listen radio in 5'th speed because of some limitation, and can't return my money, don't want to solve it, offering me an upgrade, I search a guy/service to solve the bug or unscrew the car myself, searching the bug.

I am not joking, really don't know , please tell me !

If you buy a car, you are buying the title to a tangible, physical object. It becomes your actual posession, and, as such, you have certain legal rights over that posession.

If you "buy" software, you are not actually buying the title to anything. You never "own" Microsoft Windows, all that you get when you pay is a licence. This means that the vendor sells you the right to use the software under certain terms and conditions, and explicitly not to use it outside of those terms and conditions.

When you agree to the licence, you are agreeing to the contract terms of it, which means that you are agreeing not to act outside of its terms. Should you do so, for example by reverse engineering it, then the licence generally becomes voided, and you are then guildity of breaches of copyright and contract law.

I looked at
http://ethics.csc.ncsu.edu/intellectual/reverse/study.php
http://en.wikipedia.org/wiki/Clean_room_design

 It's about product duplicating for comercial purposes and competition.
 I am not looking in kernel to duplicate it.
 Just to see if a sand particle was left in the engine by mistake or MS glue it there.

The law generally doesn't care about why you are doing something, just what you are doing. Your intent may be taken into account to determine the severity of punishment, or, in certain cases, to determine exactly which law you have broken, eg was an unlawful killing premeditated or not, but breaking laws for altruistic reasons is not a defence in court. (at least not in most countries)

Obviously the function able to move folders like are moved files is not present, not working properly or bypassed.
 If I look and inside and:
- no function or empty: making new one is legal.
- not working: unlocking it should be legal, can be done by programmer externally or MS by bugfixes or can be removed by MS, going to previous point.
- bypassed: deliberately or not. After see why, ask MS, analize the safety to not bypass it and if safe ose it like a pen found on the empty street, at your own risk to corrupt data.

Whatever should or should not be legal, and no matter how much I may agree with you that an end user should have reasonable rights when they have paid out good money for a product, it doesn't matter. You have agreed to the exact terms of the Microsoft EULA, and are, therefore bound by it.

[TeDDy]

 I am afraid I/we have totally blowed the post originated by ToolmakerSteve.
 I think Jeroen should move the deviated posts to the topic about move folders for fat32.

I understand your point of view. You reveal me a dead end.
 I ask what to do to avoid troubles.
 It must be a way.
 Did somebody ask directly MS to do something with the limitation?

I am open to ANY suggestion wich permit me to continue.
 Please give me one. Anyone.
 No more "DON'T"

 As an alternative, the folders can be moved by other "conventional" functions like makedir, copyfile, delete, rename.....? As cmd do?

[TeDDy]

Before each run, usually I delete the dat, log and debug files to avoid increasing

No need for that, none of these files increase.

You mean I must stop the program when it starts move folders and post this log

Yes, that will be fine.

I will try to stop the run. I let him finish, the MyDefrag.debuglog has about 233MiB and I don't now where to cut it. Lot of files was vacated before the folders was moved.
 I will post ASAP, because the gaps are present after many consecutive runs with same parameters.
 
BTW, is it normal to always defrag the files on drive ? I haven't touch the files in any way and after each pass, visually they seem optimised.

[jeroen]

I am open to ANY suggestion wich permit me to continue. Please give me one.

Developers can buy the full Microsoft sources for Windows and all drivers. Microsoft gives away a debugger that will disassemble and show all the machine code. Legally speaking it is not allowed to trace a Windows code, but I am sure Microsoft will not sue anyone for using a debugger to find out why directories are not moved on FAT disks. When we know what the problem is we can think of ways to work around it. Perhaps different parameters to the FSCTL_MOVE_FILE driver command will do the trick, or perhaps some kind of preparation such as stopping a service.

As an alternative, the folders can be moved by other "conventional" functions like makedir, copyfile, delete, rename.....? As cmd do?

Yes, but that is no use because it is not possible to control or influence where the new directories are created physically on disk.

[TeDDy]

I have run: MyDefrag.exe -v f: -d 5 FastOptimize.MyD with 4.0b6.
The conclusion I take is not all ntfs drives have the mentioned behaviour. I attach 2 archives.
 Smallest has 2 logs, truncated at zone7/9, first defrag and second defrag. The drive(for storage) was formatted a year ago LOGICAL TYPE on extended partition.

 Second archive has the full logs with screenshots for a small drive, specially formatted TODAY, firstly fat32 with some data copied on it, 2 succesive runs on it(p1, p2) and after that formatted ntfs, copied SAME DATA on it and runned batch 3 times. This drive is PRIMARY TYPE at the end of the same hdd.

Primary partitions seem to be optimized different than logical on extended ?
 I always have windows on the active partition fat32, others are ntfs, logical on extended.
 After I see the same "strange" behaviour on all my tested computers I made the change to the small partition wich previously was logical too.
 Not inspired to keep the logs before change it, but I can undo to logical and post the logs if necessary.

 I think the different type of partition changes the final result of optimisation but should be tested deeper.

[TeDDy]

 The API code to do precise moves is licensed by MS.
 Assembly code to format drives(fat/ntfs), create files/folders, read a sector COULD NOT BE THE PROPERTY of MS.
 They haven't licenced those actions as my knowledge(maybe I am wrong again).

Basically, if the file/folder "x" can be located to be in cluster "y" and I get low level code(assembler) to copy all necesary data to another cluster, verify if the copy was OK and delete contents of cluster "y", code obtained without looking to kernel or other licensed stuff should be legal. Right ?

 I don't think someone can forbid the cluster duplication code used in the low level of copy routines.

 The verifications to be done before you can safely erase the contents of the source cluster can be patented by MS but they must be somewhere documented.
 I think the verifications can be done safely by the API (the result is folder "unmovable" for fat32), MyDefrag can check if any process has open handles to that folder and if none, use the new builded routine to move the folder.
 I've seen kernel routines documented to chech open handles and to close them (because some "light" handles can be closed safely, no need to close the process).

 Basically the verifications to be done to see if a file can be opened for read/write must be documented and legal.
 The low level copy/check/discard data in clusters is used from first day of fat32.

 In the document with fat specifications from MS, the cluster content reading is documented. From page 17 we have samples for cluster content read and folder stuff.
 Almost low level documentation.
 It must be legal to use it.

 The whole community of defrag lovers can contribute to the search of legal documentation.
 Old programmers, familiar with assembler can help with Unique new functions builded from ground, based on this documents.
 The result will be this way legal, free to use in MyDefrag.

 I think this is a better approach to solve the problem.
 I start in this direction.
 Maybe someone will follow.

 Any help, advice, friend with low level knowledge will be appreciated.

[TeDDy]

Developers can buy the full Microsoft sources for Windows and all drivers. Microsoft gives away a debugger that will disassemble and show all the machine code. Legally speaking it is not allowed to trace a Windows code, but I am sure Microsoft will not sue anyone for using a debugger to find out why directories are not moved on FAT disks. When we know what the problem is we can think of ways to work around it. Perhaps different parameters to the FSCTL_MOVE_FILE driver command will do the trick, or perhaps some kind of preparation such as stopping a service

Not the full pack but portions of the sources to be buyed ?
 Haha, buy the mountain to fick a flower !

 Well, the third direction is shown to my eyes. Seem to be the best one. Not the cheapest in fact.
 In the past one of my colworker was in contact with a box of CD's with sources/documentations his ex institution was buyed from MS for Visual Basic or C++.
 I will try to find him to see what and how I can solve our problems.

 The path is open and usable to other enthousiasts with relations/knowledge on how to get legally help and documentation or sources and demo code.

[jeroen]

Thanks for the logfiles, but I do not have time to look at 7 logfiles. Please upload only 1 logfile that shows me exactly the problem that you are having. I am not interested in any other logfile, such as for FAT disks (because there is no $MFT on a FAT disk).

[TeDDy]

 Sorry, I belived the differences between succesive runs on 2 ntfs types(primary & logical) was originated the behaviour.
 The fat32 logs was archived by mistake. Sorry again.
I attach the first log from first archive.