Blurring the lines of HPC

O

  ver at scalability.org --one of my favorite reads, despite (or is it because of?) my occasional disagreements with Joe--you'll find an interesting note about using FPGAs for non-scientific applications: specifically, business applications.

However, his post isn't really so much about FPGAs as it is about the blurred lines of HPC these days. He talks about a couple of pieces of hardware out there and wonders how they may be applied to domains outside of HPC. Joe sums it up thusly: "HPC is not the only thing that needs acceleration."

This is such an important concept, and it's one that many people need to realize.

The computing universe is changing. It was once separated into many fiefdoms, with nearly no overlap: Business/back office Computing. Technical/HPC computing. Supercomputing. Desktop computing. The hardware, the OSs, and the software for each were unique to that fiefdom.

But ever-increasing commodity hardware speeds, ever-improving operating systems, and ever-more-connected systems are blurring the lines. Many of the Top 500 are no longer super computers--they're Linux clusters. One is even a Windows cluster.

Large businesses are routinely doing analysis that would make an HPC cluster wipe its brow with sweat. Some of the calculations that companies are doing as part of routine business intelligence wouldn't have even been possible a few years ago.

Even some technical computing is now being done on commodity Windows systems: the Army Corps of Engineers is doing some of their weather simulation on Windows PCs.

All of this line-blurring is why there is such a growing market for distributed computing. The hardware and OS have the ability. The software is coming along, too, to enable 'regular companies' to do the kind of processing that, until very recently, would have required 'HPC.' Joe says, in reference to FPGAs being used as Accelerator Processor Units in business computing, "I think the market for these might just be a whole lot larger than the “small” $9 billion that is HPC these days."

I think he's right. And I think his statement can be expanded: the market for general distributed computing is much, much larger than the HPC market is today.

This is exactly why I think Microsoft will succeed with their cluster operating system (and why my company is succeeding with our distributed computing product). We are finding customers who never dreamed about putting in a traditional, HPC cluster. Some are small customers who can't afford dedicated hardware or training on a new OS. Some are large customers who are simply scaling their SOA (and want to continue using Microsoft's development tools). They're not HPC users. This isn't HPC. This is, as Joe puts it, "Accelerated Computing."

Technorati tags: distributed computing, hpc