Friday, July 24, 2009

Sun Grid Engine and the Soul of Cloud Computing

Cloud Computing! Ah.. the word is everywhere. Cloud is this, cloud is that... so much that many times you wonder, hope its not just a vapor hanging in middle of no where just waiting for right temperatures to set in to pour down the valley. There is a saying in Urdu (my neighboring country's national language, currently remains in news for all the wrong reasons). The saying goes like this,

Jo baraste hai
Woh garajte nahi
Jo garajte hai
Woh baraste nahi

Those clouds who thunder need not shower
and those who shower need not thunder

We hear a lot of noise created by early cloud resources providers about how scalable they are. How they can simply go till infinity and come back if demand for resources also fluctuates in the same range. But tomorrow what if, cloud computing is adapted by masses? will they really be able to scale up? Do they have a right kind of software to handle such loads? They will need to effectively schedule their individual virtual machines to run on servers most optimally, this is a must for the cloud to be a cost effective solution both for the provider and subscriber of cloud resources.

The thought of scheduling jobs (think individual virtual machine as a job) effectively over the given hardware reminds me of something which was considered as the baap (father in Hindi) of all sorts of distributed computing... Grid Computing. There we go... What if someone uses Sun Grid Engine (SGE) baap-of-all-cluster-schedulers in cloud environment?

Think of something like on these lines...

  • Imagine you are a guy who is going to provide the cloud resources to basically everyone over the internet. So you need to be truly a web scale provider.
  • You will have certain number of systems on which most probably you will be running Xen Servers.
  • These Xen Servers will run Xen virtual machines which you will lend out as a cloud service by your Web Service. ( I am just consider SAAS for this example).
  • Now you want to make sure those virtual machines should run on your physical systems in such a way that you get to push your input costs down much as possible.
  • Then why not think of SGE as a way to handle the start/stop/migration of these virtual machines? After all SGE is well known for handling and supporting large loads very effectively.
  • So that brings us to point that, like we start/stop a job using a job script in SGE, why not start/stop/restart/deploy/destroy a Virtual Machines.
  • You don't have to invest in writing your own scheduler either!

No comments: