Ultra Low Latency Grid Computing
High Performance on Wall Street was the hands down the best HPC conference I've ever attended. The quality of the sessions was high. The quality of the attendees was high. Even the quality of the vendors was high (full disclosure: we were one of those high quality vendors). There was nary a "booth babe" nor "tchotchke hound" in sight--everyone was there to talk HPC.
The theme of this year's conference was "Low Latency," and my favorite panel was specifically about low latency and multicore processors. There was great talk about shaving microseconds off of compute times for complex event processing, but one of the customers (I can't remember if it was someone from Citigroup or from Deutsche Bank) made an interesting point: ultra low latency is not the biggest problem he faces, because it's really only involved in a small number of his applications. He said that latency is an issue for them, but brought out the old "when all you have is a hammer, the whole world looks like a nail" cliché: there's no need to apply ultra-low latency tactics to all of the computing they do.
Most applications they are running do not need to run in microsecond or milliseconds because of an external event--most applications are running either at user request or in batch processes. And he brought up what is one of his biggest problems: the time it takes them to develop an application to run on their grid.
In other words, he's not worried about the milliseconds it takes to start processing: he's worried about the days, weeks, and months it takes to develop applications and adapt them for distribution on the grid.
It struck home with me, of course, because reducing developer time is what we've been preaching since we first released our SDK. It's also the thing that our customers have liked the most about our products: developers can adapt their applications to run on a grid in as few as 20 lines of code.
I'm not the only one noticing the importance--Nikita Ivanov of GridGain (they have an open source grid product for Java) just posted about the same topic.
So is a grid ideal for computations that need to start in the next couple of microseconds? Nope.
But if you need your app running on the grid tomorrow instead of next week some time, we can help you shave billions of microseconds off of your development time.
Now that is what I call ultra low latency grid computing!
Photo credit: Darren Hester
