Web Services + Grid = Crazy Delicious
The other day I happened to glance at the March 2005 issue of Cluster World magazine that was sitting around in the office. One article in particular caught my attention (sorry I can't link to it; ClusterWorld.com's "Old Stuff" link is broken, but it's also available here in PDF format). Lee Liming of Argonne National Lab wrote it; it's called Web Services and the Grid.
Lee's premise (and it's a good one) is that clusters and grids are a natural tool for developers of Web services.
As clusters become increasingly essential elements of the Grid's physical fabric, Web services are becoming essential elements of the Grid's application development toolset. Given the importance of clusters and Web services to Grids, cluster owners and operators need to understand the implications of Web services on the applications that run on their clusters.He's right. Web services require scalability. One of the inherent truths of creating good web service applications is that you must plan for success: if you've written your software well, people will use it. You have to be prepared for that success and therefore prepared to scale your application. Scaling to a cluster is a perfectly good way to do that. Scaling to a grid is even better. Why? Because it's more cost effective. As you need to grow, adding processing power to your application is as simple as adding more nodes to your grid. Even if the grid is supplementing a dedicated cluster, it gives you the processing power you need at peak times without the expense of dedicated hardware that sits idle 95% of the time.
Note that unless the Web service has been designed to use the back end nodes on the cluster, it will only use the cluster's head node. Web services that require significant processing power should be written to submit tasks to back end nodes via the cluster's scheduler or other tools. The application developer must explicitly implement this capability.This is an important point that many people miss or think that Network Load Balancing will handle automatically. In fact, NLB is great for serving web pages from multiple servers, but is not a good tool for handling compute-intensive process (for a list of disadvantages of NLB, see Brian Madden's article How to Configure Network Load Balancing).
Back to Lee's point: if you want to take advantage of the power of a cluster or a grid, you need to do that yourself. Of course, I happen think that you should simply get yourself a great toolset that will handle the job submission, guarantee execution, monitor the processes, handle node failure, etc. That frees you up to concentrate on the functionality of your Web service, rather than the technical details of scalability.
Lee sums it up like this:
...it seems likely that service-oriented applications—such as those based on Web services—will lead to significantly greater use of clusters (i.e., more business) than traditional, manually launched applications. Early efforts to gear clusters to become high-power hosting environments for these types of applications will position administrators well in a service-oriented era.
Lee is obviously talking to vendors who offer hosting solutions. But his points are broader and apply to anyone who is designing and building Web services or SOA. If you're writing software that will be available as a service (Web service, SOA, SaaS--you choose your favorite), grid it up and let it fly.
(And if you don't get the reference of the title of this post, you must not have seen the Chronic of Narnia yet, or don't understand the allure of Mr. Pibb and Red Vines).
|