| Posted by pipcaster, 01-04-2011, 07:18 PM | |
I am looking for a solution to a project,
I have software (closed source), that is very heavy on processor usage. Even a Intel® Core i7-980X is too slow. Even a Intel Xeon X5660 @ 2.80GHz is too slow to get the job done fast enough.
Even Amazon EC2 is too slow as well.
Even on these systems , running 24/7 at 100% cpu utilization it takes days to get it all done.
So I thought...hey what if i build a grid or a distributed server setup and then do it that way.
So...I was wondering if there was some software where i can install to multiple servers, and then channel all the CPU towards one server that would be running my software. Or even some software i can install so that my one server can draw upon the cpu resources of all servers?
I want all this on windows server.
So has anyone got ideas how to achieve this?
|
| Posted by arisythila, 01-05-2011, 12:36 AM | |
You can do that. Thats were Applogic came into play when it first came out was a GRID Computing system. I can basically have multiple applications running on multiple physical boxes, crunching your stuff.
Can you tell us what your doing exactly so we can get a feel? Your probably going to have to look at some pretty burly machines to crunch your data. I would look at a 2P or a 4P solution with the new Magny Cours. They crunch like no other.
Thanks,
|
| Posted by pipcaster, 01-05-2011, 02:53 AM | |
Ok Mike,
You seem to be the only person who has a good grasp of what i am trying to do.
Are you saying you can provide me with a solution?
If yes can we talk on skype?
Paul
|
| Posted by dazmanultra, 01-05-2011, 05:16 AM | |
How effectively can the problem you're solving be parallelised?
If only a small proportion of the problem can be parallelised, or, the overhead of parallelisation is too great (i.e. splitting the problem up is computationally expensive) then you may find that you need to re-evaluate the problem.
|
| Posted by pipcaster, 01-05-2011, 03:51 PM | |
Hi,
My tech said that it can be "embarrassingly parallel" if needed.
Does that make any sense to you ?
QUOTE=dazmanultra;7205213]How effectively can the problem you're solving be parallelised?
If only a small proportion of the problem can be parallelised, or, the overhead of parallelisation is too great (i.e. splitting the problem up is computationally expensive) then you may find that you need to re-evaluate the problem.[/QUOTE]
|
| Posted by dazmanultra, 01-06-2011, 05:43 AM | |
Embarrassingly parallel means there is virtually no overhead in splitting the task up into parallel 'jobs' - as such something like EC2 or any other cloud computing platform could be utilised very efficiently to do whatever it is you're doing.
This is one of the best things about cloud computing - you can now solve your problem by leveraging perhaps a few hundred EC2 instances for 24 hours or less (total cost a few hundred dollars) and then once done, switch your instances off. In the past you'd have needed to invest in actual hardware, spending more money and taking longer to complete.
I think there is a comparison somewhere of CPU power per $ on the cloud computing providers which you'd be wise to look at.
|
|
 Add to Favourites
 Print this Article
|
