Regarding Multithreading

[SebastianJu]

I noticed that MyDefrag isnt Multithreaded and have read some threads here with answers from the developer.

So now i know that its the windows-api that isnt multithreaded and this api is doing the filemoving. And thats the problem. But when i read this i think why should this be the problem when the core is fully working? The API only gets the move-orders which means it handles the filesystem. And the cpu is only calculating where to move the files isnt it?

It would be a good thing to have myDefrag multithreaded because i always have the full used power of one core when defragmenting my system drive. So i guess it would speed up the process because when one core is fully working thats the bottleneck.

So now i tested to use 2 instances of mydefrag. Each defragmenting another disc. And it looks like now 2 cores are used. So it would work in principal or am i wrong? With that solution its possible for me to defragment 2 discs at the same time with less time than doing it one after another. I can be wrong here of course.

So if choosing to defragment 2 discs why is mydefrag only defragmenting one after another? It could open another instance or the main program always is creating a instance of a defrag.exe or so. Than at least multithreading can be used for more than one disk.

And regarding multithread for one disc or for 2 partitions on the same disc theres the problem that more than one file would be copied at the same time i have read. But why is that? Couldnt there be a queue so that it would start ony copying after another?

I only think that the cpu is the bottleneck for me so maybe there can be done something.

Thanks!
Sebastian

[Darlis]

Why JK/MyDefrag processes one volume/disk after another:

Harddisks use a lot of power, and most computers have minimal power supplies. There is a very real danger that you will overheat and burn out your power supply if you run JkDefrag on multiple disks at the same time. The same applies to the harddisk controller, most computers are simply not designed for such a workload.  This is the reason why JkDefrag has a build-in protection against multiple running.

[jeroen]

Running multiple instances of MyDefrag is only useful if the instances work on volumes that are on physically separate disks. If you start multiple instances of MyDefrag for volumes that are all on the same physical disk then everything will become extremely slow because the harddisk heads have to move huge distances between the various volumes. This is why MyDefrag by default will popup a window to ask if you are sure. See the OtherInstances setting to avoid the popup.

Multithreading on a single volume is theoretically possible, but there are some severe problems. The most important problem is that files may frequently not be moved at all by the Microsoft API (unmovable files), or are moved only partly (don't know why the Microsoft API does that sometimes, but it does). Moving a file therefore depends on how previous moves have finished, moves have to be performed one by one.

Using a queue is not possible, again because you cannot assume that a particular move is executed as desired, but also because on a multitasking system things happen all the time on the disk. Making a queue with commands for let's say the next 30 seconds is therefore impossible, the disk will have changed too much. MyDefrag therefore does not use a queue, and performs various checks before each and every move.

[SebastianJu]

@Darlis

*lol* Thats a creative excuse... i wouldnt think of getting such an answer. The little power that a harddisc is eating is nothing for the power supplies used nowadays. And regarding harddisc controller... when the harddisc-controller would be the problem the core wouldnt be maxed out in usage i think. Not to mention that there are esata and whatever else where it would be still a good thing even when the normal controller would be used to its max.

@jeroen

> Running multiple instances of MyDefrag is only useful if the instances work on volumes that are on physically separate disks. If you start multiple instances of MyDefrag for volumes that are all on the same physical disk then everything will become extremely slow because the harddisk heads have to move huge distances between the various volumes. This is why MyDefrag by default will popup a window to ask if you are sure. See the OtherInstances setting to avoid the popup.

I see what you mean but again in this case why not simply create another instance maybe called queue.exe that gets all commands to move the files and then decides what partitions are on separate discs. then it gives out multiple commands if there are more than one disc or it gives out one command after another. This way there wouldnt be a problem anymore while maxing out the power of cpu-cores.

> Multithreading on a single volume is theoretically possible, but there are some severe problems. The most important problem is that files may frequently not be moved at all by the Microsoft API (unmovable files), or are moved only partly (don't know why the Microsoft API does that sometimes, but it does). Moving a file therefore depends on how previous moves have finished, moves have to be performed one by one.

This would be solveable with a queue.exe i think.

> Using a queue is not possible, again because you cannot assume that a particular move is executed as desired, but also because on a multitasking system things happen all the time on the disk. Making a queue with commands for let's say the next 30 seconds is therefore impossible, the disk will have changed too much. MyDefrag therefore does not use a queue, and performs various checks before each and every move.

I see... but files are moved anyway. So what is done in such case? Cancelling the copy-process? If so the same could be done with the queue. By dropping the rest of the queue or only that commands that interfere with the failed file-move. Plus giving the info back that the file isnt moved there and the defrag.exe-Thread could correct its cache.

Am i wrong at some point?

[jeroen]

The little power that a harddisc is eating is nothing for the power supplies used nowadays.

Harddisks use a lot of power, just feel how hot they can get. Cheap and minimal power supplies are not efficient and generate a lot of heat, and there is a very real risk that they overheat. Same for the harddisk controller, minimal heatsinks, operation temperature is close to the maximum temperature. Many users have reported that their computer can get very hot when running MyDefrag.

I see what you mean but again in this case why not simply create another instance...

Yes, that would be possible. The keyword here is "simply" 

By dropping the rest of the queue or only that commands that interfere with the failed file-move.

Ok, I guess something like that is possible. Perhaps I should not have said "not possible", but "complicated".

[Kasuha]

I only think that the cpu is the bottleneck for me so maybe there can be done something.

CPU is not the bottleneck here, the most time spent on optimization is spent waiting for a disk operation to finish.

MyDefrag may not support optimizing multiple disks at once within one process but does not prevent you starting it multiple times and run such optimization manually or even starting such parallel optimization from a batch file.

Notice there is one more hurdle in trying to optimize multiple disks within single process - many Windows versions place a limit on how much RAM a single process can use. Certain users have disks large enough to not be able to optimize even that single disk - trying to optimize multiple disks can reach that state even on significantly smaller disks, creating negative user experience.

[SebastianJu]

@jeroen

> Harddisks use a lot of power, just feel how hot they can get. Cheap and minimal power supplies are not efficient and generate a lot of heat, and there is a very real risk that they overheat. Same for the harddisk controller, minimal heatsinks, operation temperature is close to the maximum temperature. Many users have reported that their computer can get very hot when running MyDefrag.

I really think the pc is getting hot because the cpu is working hard. When the heat is a problem that would it be a general thing. Otherwise you probably couldnt play newer games or do something that maxes out the cpu-usage. If the heat from harddiscs would be such a problem then it would need a better venting. Thats nothing were i would say that "we cant use multithread because of the harddisc-heat". And even when what about the users that dont have a badly vented pc? Only because there are cheap and rubbish pcs all cant use multithread? It would be better than to give the cheap-pc-users to disable multithreading instead the other way.

And regarding the heat at the power supply:
http://www.meisterkuehler.de/content/energierechner-fuer-computer-79.html

Sata 9W
IDE 10W
SCSII 16W (Whoever will use that)

While a system thats maybe a year old using a Intel Core2 Duo E8200 and a Geforce 7800GTX are together eating 150W. So if the power supply would get problems because of some harddiscs it would be a pc useable for nothing i think.

> Yes, that would be possible. The keyword here is "simply"  Smiley
> Ok, I guess something like that is possible. Perhaps I should not have said "not possible", but "complicated".

Yes probably. It would be another concept of course and would need a good chunk of code-changings... but maybe taking that in account when the next big change in sourcecode has to be done?

@Kasuha

> CPU is not the bottleneck here, the most time spent on optimization is spent waiting for a disk operation to finish.

Not at my system partition. There are many small files and much to calculate. At other harddiscs with movies and big files in general thats not the case but with many small files... and i doubt that it would be so different with other peoples system partition. DMA is activated by the way. So its not the case that the cpu needs to take part in the copying. With enabled DMA the CPU should only need to check out where the files can be moved i think. Or does the cpu take part in another way?

> MyDefrag may not support optimizing multiple disks at once within one process but does not prevent you starting it multiple times and run such optimization manually or even starting such parallel optimization from a batch file.

Yes... thats my workaround for the moment... but i think multithreading could speed up discs with many files too.

> Notice there is one more hurdle in trying to optimize multiple disks within single process - many Windows versions place a limit on how much RAM a single process can use. Certain users have disks large enough to not be able to optimize even that single disk - trying to optimize multiple disks can reach that state even on significantly smaller disks, creating negative user experience.

I once had this problem with defraggler. I had a 1,5TB-Disc to defrag. Emule-Files created as sparse. You can imagine how many fragmented files there were. And defraggler checked and loaded so much ram that it was at 2GB at some point. So the pc couldnt run forward anymore. i then used Auslogics disc defrag and it worked with way less power consumption... And MyDefrag is using maximum 150MB at my pc per disc. So i think in most of the cases there wouldnt be a problem. I checked a full 1,5TB-Dsic too.

[Darlis]

While a system thats maybe a year old using a Intel Core2 Duo E8200 and a Geforce 7800GTX are together eating 150W. So if the power supply would get problems because of some harddiscs it would be a pc useable for nothing i think.

You can calculate the power consumption of the components this way, but you can not use it to determine how powerful the power supply has to be, that's not how they work.
A power supply is divided into several lanes (3.3V, 5V, 12V (and optional 12V2, 12V3, 12V4)), each providing only so much ampere to supply the different components. The output printed on the power supply is basically the sum of all these lanes. So you could still be able to supply a powerful graphics card with (one or more of) the 12V lanes, but the 5V lane for the HDDs could have already reached it's maximum.

Also, 200W is not the same as 200W. Just take a look at the values of different power supplies with identical power output, you'll certainly notice some differences.

Edit: To illustrate: I have a FSPGroup 250W providing 25A on the +5V lane and an identical(?) LiteOn 250W power supply, providing only 16A on the +5V lane.

[SebastianJu]

> You can calculate the power consumption of the components this way, but you can not use it to determine how powerful the power supply has to be, that's not how they work.
> A power supply is divided into several lanes (3.3V, 5V, 12V (and optional 12V2, 12V3, 12V4)), each providing only so much ampere to supply the different components. The output printed on the power supply is basically the sum of all these lanes. So you could still be able to supply a powerful graphics card with (one or more of) the 12V lanes, but the 5V lane for the HDDs could have already reached it's maximum.
> Also, 200W is not the same as 200W. Wink Just take a look at the values of different power supplies with identical power output, you'll certainly notice some differences.
> Edit: To illustrate: I have a FSPGroup 250W providing 25A on the +5V lane and an identical(?) LiteOn 250W power supply, providing only 16A on the +5V lane.

Youre right on all points i think. The Ampere are important. Nearly no one knows this. So it happened once that i bought a "broken" pc for cheap at ebay after i have read the problems. I checked everything and it was like i thought. The power supply had a high Watt-Value but the Ampere was low at 12V. And the built in GPU needed 20A or so. Otherwise its not working. I sold it in pieces and made a chunk of money from it.

But if this means there is a problem for harddiscs i dont know. I never heard that a power supply was too weak for harddiscs. I guess even copying from one disc to another could be a problem then. And regarding heat of the power supply i guess the gpu will create more heat there because of the more Watt that is worked up. And when it comes to the heat of the harddiscs it would be needed a good venting only. But i said this before...

When you say 16A at 5V am i wrong or would this mean a power supply of 80Watt for this 5V? So I think even there could be so many harddiscs switched on that practically no one would have that many harddiscs working on one power supply.

[Darlis]

When you say 16A at 5V am i wrong or would this mean a power supply of 80Watt for this 5V? So I think even there could be so many harddiscs switched on that practically no one would have that many harddiscs working on one power supply.

The 5V lane is not only for the HDDs, is supplies also the optical drives and some components on the motherboard. And these components are actually powered by 5V and 12V (my mistake). Most power supplies should not have problems with several disks but considering the differences I posted above and that the power supply has to be as cheap as possible (it doesn't make you PC faster doesn't it?), especially in budget PCs, there could be problems.

And the venting in these budget PCs is really minimal. Stress a HDD on a hot day and you might have to buy a new one...

[Kasuha]

And the venting in these budget PCs is really minimal. Stress a HDD on a hot day and you might have to buy a new one...

Budget PCs usually have single disk so I believe this kind of argument against processing multiple disks at once is invalid. Yes, you can overheat your HDD in a budget PC using MyDefrag as well as using any disk heavy application, including standard defragmenter, virus scan, disk search program or a database server. But that's not relevant to the initial question.

[doug65536]

I have a 1200W power supply. You're restricting the software because SOME users have bad power supplies?

My Core I7 Extreme 990x 6-core @ 3.45GHz (not overclocked) with SLI (dual) GTX 580 24GB RAM SSD + 10K RPM HD running a massive stress test (prime 95 torture test + furmark GPU stress pegging all cores and GPU's to 100% continuous) uses a grand total of about 650W (according to my UPS) (and by the way, ANOTHER Core I7 system running folding @ home with the GPU pegged there is also running off the same UPS, and my entire system including routers, dual 27" 1920x1080 monitors, speakers, etc. are all running off it too).

I'm sorry but your claim that power supplies can't run two active hard drives is a load of bull.

[Darlis]

Nobody said that all (and especially your) power supplies would have problems with it.

[peptobismal]

Gosh, don't mean to bump this, but I just read it and found it very entertaining... that people cannot see the reason why a "Utility Program" would be geared towards "everyone" in a general sense? And were these people not able to Ignore other instances and run multiple at their own will? How many people really want to defrag multiple drives at once? Who says they think the CPU is the reason the computer is hot and nothing else? All, so very silly.

[tOM Trottier]

fwiw, google has found that HDs fail more over 50C and under 35C degrees. You can use HDtune to constantly display the temp for at least 1 drive. This is especially VIP in laptops.