Thanks, Profes...I mean, John!

I had a great meeting yesterday with John Ousterhout of Electric Cloud.

John used to be a professor of Computer Science at Berkeley, and he's one of the reasons I majored in CS. He was really the ideal professor--the kind of guy who knows every single student's name (even in a class with close to 200 people in it), who blends education with humor (I remember the first day of class, he told us that his name is pronounced "OH-stir-howt," but he responds to anything from "Oosterhoot" to "Easterhat"), and is active in cutting edge technology.

In the early 90s when I worked at Quantum Consulting under Margo Seltzer, I even used his Tcl/Tk toolkit when we ported one of our Windows products to run under Solaris.

Professor Ousterhout moved on from Berkeley to Sun as a Distinguished Engineer (and, I guess, stopped being called "Professor"). After Sun, he started Scriptics, which was later purchased by Interwoven. In short, he's had a stellar career thus far. His latest venture is Electric Cloud.

Electric Cloud is a pioneer in distributed builds--they have great technology and an impressive client list.

I really enjoyed catching up with John, and I very much appreciated the hour we spent together. He is quite an incisive guy: within minutes of sitting down, he was offering fantastic advice about Digipede. We didn't talk so much about the technology (although, as someone who has been working with distributed computing for decades, I'm sure he'd be a great technical resource), but about the business.

Clearly, he has a head for business as much as he as a head for software. I valued his opinion because he's done something extremely difficult to do: taken a development tool (a decidedly unglamorous product when compared with the AJAXy/SocialNetworky/Web2.0ish companies getting funded these days), gotten it funded, and brought it to market.

We discussed pricing strategies a fair bit. None of his ideas were brand new, but it is always great to get an outsider's perspective.

One thing they concentrate on at Electric Cloud is the ability to determine who their customers are in three questions: What platform do you develop on? How many developers do you have? How long do your builds take? The answers to those three questions determine whether a company is a prospect.

I can't boil my customers down to three questions--but I'd really like to be able to. This is a great exercise, and it's one we need to undertake at Digipede. It's not just a useful tool for a junior salesperson. It's also great for us, as a company, to be able to define our customers that way. Investors need to know who the customers will be. We need to be able to refine our marketing to hit the right people. We are sure we've got a huge potential audience for our products--unfortunately, it's amorphous and hard to define. My hour with John made it clear that we need to define it clearly, even if that means shrinking it a bit.

Of course, when our customer base is doing things as different as risk management, Monte Carlo simulation, hurricane simulation, Bayesian analysis of text messages, map generation, gathering web content, PDF generation and genetic algorithms, it's a tough nut to crack.

Technorati tags: Electric Cloud, digipede, John Ousterhout

Strange floating point behavior on Xeon

One of the first applications I wrote using the Digipede Framework API was a distributed Mandelbrot app. It's almost a cliche application for a grid computing system--still, it looks pretty and calculates a heck of a lot faster on many machines than on one.

Recently, I was tweaking our calculation code to get better colors out of it. After tweaking it, I noticed something very strange happening: different machines were calculating different colors for the same pixels! Here's a screencap that shows how it manifested itself:


Because different machines calculate different portions of the picture, and those portions may end up right next to each other, it's pretty obvious. The rough edges give you an idea that things just weren't right--I've circled some of them.

At some point, I realized that there was one machine on my testbed in particular that was returning different values than the others. I also noticed that the gradients on the bitmaps from that machine lined up perfectly with the gradients on the bitmaps from the other machines--the colors were just different. Furthermore, the colors were always 1 color off (the colors in my Mandelbrot are stored in an array).



As it turns out, that machine has dual Xeon processors in it, and it's the only one on our network that does. It was pretty clear that the Xeons were causing a problem--and today I decided to pin it town.

To calculate a Mandelbrot set you do a repetitive calculation using complex math. Basically, you start with a complex number z, and you iteratively perform the calculation z = (z * z) + c. You count the number of times you can do that before z is too large to calculate. You then choose your color based on the number of iterations.

I added some code to my calculation object to spit out the results from the calculations. On most machines, it looked like this:

after 0: z = -1.78571428571429 + -1.19047619047619i
after 1: z = -0.0141723356009069 + 3.06122448979592i
after 2: z = -11.1566088075442 + -1.2772455921144i
after 3: z = 121.052849496283 + 27.3089826542847i
after 4: z = 13906.2261232719 + 6610.46985810096i
after 5: z = 149684811.460994 + 183853376.065174i
after 6: z = -1.1396521108449E+16 + 5.50401158655656E+16i
after 7: z = -2.89953366111956E+33 + -1.25453168454679E+33i
after 8: z = 6.83344570443359E+66 + 7.27511369656889E+66i
after 9: z = -6.23129910256229E+132 + 9.94281888781693E+133i
after 10: z = -9.84713565508731E+267 + -1.23913356825186E+267i
after 11: z = Infinity + Infinityi

But on the Xeon machine, it looked like this:

after 0: z = -1.78571428571429 + -1.19047619047619i
after 1: z = -0.0141723356009069 + 3.06122448979592i
after 2: z = -11.1566088075442 + -1.2772455921144i
after 3: z = 121.052849496283 + 27.3089826542847i
after 4: z = 13906.2261232719 + 6610.46985810096i
after 5: z = 149684811.460994 + 183853376.065174i
after 6: z = -1.1396521108449E+16 + 5.50401158655656E+16i
after 7: z = -2.89953366111956E+33 + -1.25453168454679E+33i
after 8: z = 6.83344570443359E+66 + 7.27511369656889E+66i
after 9: z = -6.23129910256229E+132 + 9.94281888781693E+133i
after 10: z = -9.84713565508731E+267 + -1.23913356825186E+267i
after 11: z = NaN + Infinityi
after 12: z = NaN + NaNi

See the difference? After 10 iterations, the calculations were identical. But on the 11th, the first goes to Infinity. The second, though, goes to NaN!

According to the .NET 2.0 documentation on NaN, "This constant is returned when the result of an operation is undefined." And the documentation for Double.PositiveInfinity says "This constant is returned when the result of an operation is greater than MaxValue." Double.NegativeInfinity is, of course, "This constant is returned when the result of an operation is less than MinValue."

MaxValue, by the way, is 1.79769313486232e308. So there's no doubt that the result of squaring z was going to overflow. But I don't see anything to indicate that the result should be NaN--and all of my other machines agree: it should be Infinity.

So is this a problem with the Xeon? The FPU? The .NET 2.0 libraries on my x64 machine? It's a 32-bit app, so I don't see how that would be.

I was able to work around this in code, but it's not the kind of thing developers should have to worry about (see Kim's post on how developers rely on the systems beneath their code).

Still - I'm glad this was only an application drawing pretty pictures. I'd hate to see the calculations on the trajectory of our next Mars probe to go awry based on something like this.

Has anyone else had any discrepancies using .NET 2.0 on the Xeon?

Technorati tags: xeon, mandelbrot, .net20, digipede

MSDN Webcast Resources

Welcome MSDN Webcast attendees!

I said during my webcast today that I'd post a bunch of links and stuff--here they are. Some of these are repeats from my last webcast, but some are brand new.

If you want to download a fully-functional copy of the Digipede Network Developer Edition, just visit www.digipede.net/faster and use the promotion code MSDN.

My first MSDN webcast was called "Object-Oriented Programming for Grid," and is available on demand here.

The paper I mentioned was Web Services and the Grid by Lee Liming of Argonne National Labs. I've referenced this here before. It's a great piece on why web services and grid computing go hand in hand.

I didn't mention this in my webcast, but you also may be interested in reading this article from Ian Foster, one of the fathers of grid computing: Grid's place in the service-oriented architecture.

The blogs I listed were:

Expert Texture: Robert Anderson (CTO of Digipede) on .NET development and grid

A Day in the Life: Kim Greenlee, Digipede Evangelist, on grid computing (including lots of Excel/grid interop)

But if you're interested in distributed computing, you should also check out:

All Things Distributed: Werner Vogel (CTO of Amazon) on, well, all things distributed

Grid Meter: Greg Nawrocki (President of the Globus Consortium) on grid computing

For those who asked about source code: e-mail me! (See link at right)

Update 4/25/06 12:16: Added new links.

Technorati tags: grid computing, digipede, OOP-G

Monday: New York. Tuesday: Everywhere Else!

It's been a busy few weeks at Digipede (which you may have noticed by the small number of posts here).

In addition to planning our next release and some meetings with potential partners, we've been preparing for a couple of events next week.

On Sunday, John and I are going to New York. Microsoft is hosting its fourth annual Financial Services Developer Conference at the Millenium Broadway Hotel. I'm really excited about this--Stevan Vidich of Microsoft has asked us to give a demo during the Capital Markets keynote address Monday morning. It'll be a great opportunity to show off our stuff in front of Manhattan's finest developers--assuming that we can get network connectivity! As usual, it's a pain in the neck to actually get network connectivity at the hotel, which means I may have to give a distributed computing demo without being connected to any computers.

Unfortunately, I'm going to have to miss Tuesday's sessions (I'd really like to see the one by Naresh Bhatia and Stephen Maltzman from Sapient). I'm giving another MSDN webcast Tuesday morning, and I wanted to do that from my office. That all adds up to a hellish itinerary--I won't get to bed Monday night until about 4:30AM Eastern time.

But this should be a great webcast. After giving a good, basic "100" level description of Object Oriented Programming for Grid in my last webcast (you can view that here, by the way), I'm really going to dive in deep in this one. It'll be the most extensive code demo I've ever done. I'll show some practical applications of the Digipede Network, including using it with Excel (which I've shown before) and using it behind a web service and behind a forms app (both new demos).

You haven't registered yet? No fear: register here. Date and time: Tuesday April 25, 11:00 AM PDT.

Technorati tags: excel, digipede

Photo credit: danzo08

Grid Computing and Excel

I get lots of hits on this blog from people searching on "grid computing and Excel" or "cluster computing and Excel," so I thought I'd point people to the Digipede whitepaper page. My colleague Kim recently finished a great whitepaper on using Excel with the Digipede Network; check it out. She describes a bunch of different architectures (depending on your needs and level of expertise), all of which we've implemented in house.

And one of which I'll be demonstrating next Monday at Microsoft's 4th Annual Financial Services Developer Conference in Manhattan.

We've got a couple of other whitepapers up on that whitepaper page as well; if you've been following my blog peripherally but would like to dive in, take a look!

Technorati tags: grid computing, digipede, excel

Resources from MSDN Webcast

After my MSDN webcast, one attendee (ok, it was my wife!) suggested that I put a reading list up on my blog. Thanks, Cindy--good idea (and definitely the first time she suggested that I blog something!). I plan on creating a reading list at some point and putting that in the sidebar. In the meantime, here are the resources I mentioned in my talk today.

The paper I mentioned was Web Services and the Grid by Lee Liming of Argonne National Labs. I've referenced this here before. It's a great piece on why web services and grid computing go hand in hand.

The blogs I listed were:

Expert Texture: Robert Anderson (CTO of Digipede) on .NET development and grid

A Day in the Life: Kim Greenlee, Digipede Evangelist, on grid computing (including lots of Excel/grid interop)

But if you're interested in distributed computing, you should also check out:

All Things Distributed: Werner Vogel (CTO of Amazon) on, well, all things distributed

Grid Meter: Greg Nawrocki (President of the Globus Consortium) on grid computing

[Update 4/11/2006 4:54pm] By the way, the link to the webcast will be up tomorrow. Of course, I'll post it as soon as I get it. The next MSDN Webcast is Tuesday, April 25th at 11:00AM. I'll post a link to that as soon as its listing is correct! Oh, one more thing: there's a Digipede webinar tomorrow morning at 10:00 am PDT; if you're interested, go here for an invitation.

Technorati tags: grid computing, web services

My MSDN Webcast on Tuesday!

My first MSDN webcast is Tuesday April 11th at 11:00AM Pacific time. It's the deepest Object Oriented Programming for Grid talk that I've given outside of Code Camp.

For most of you who are reading this Tuesday morning...what are you waiting for?

Register here!

Technorati tags: OOP-G

Digipede News

I like writing about distributed computing more than I like writing company stuff (I feel so cheap when I do this!), but I also really like to spread good news!

My Gada.be feeds turned up two interesting bits of press about Digipede in the last 24 hours. Just in case you haven't set yours up yet, I thought I'd share them with you:

First, Mario Morejon of CRN magazine wrote a positively glowing piece about the Digipede Network. The headline says it all: Grid Computing Turns .Net Into Enterprise Powerhouse. I won't quote much here--just read the review; it's awesome. He gave us 5 stars!

Second, we were pleased to accept an invitation to Microsoft's BioIT Alliance. The BioIT Alliance is a group of companies "working to further integrate science and technology as a first step toward making personalized medicine a reality." Don Rule put it together--he's a very smart guy, and we're excited to be playing on his team.

Technorati tags: BioIT Alliance, digipede

Vogels on Scalability

Last December I participated in a bit of a debate about scalability. My main point (I was agreeing with Jeremy Wright at the time) was that good software has scalability designed in from the beginning. To quote myself:

...you may actually succeed and build something that people eventually use. Many people. If you are going to make it available on the web, and you're not designing for scalability, then you just aren't planning for success: you're planning for failure.

Well, on Thursday my sentiment was echoed by a pretty powerful voice: Werner Vogels, CTO of Amazon.com.

In his post A Word on Scalability Vogels sums it perfectly:

Why is scalability so hard? Because scalability cannot be an after-thought. It requires applications and platforms to be designed with scaling in mind, such that adding resources actually results in improving the performance or that if redundancy is introduced the system performance is not adversely affected.

Right on! I couldn't have said it better myself (even though I tried).

Vogels goes on to talk about the difficulties inherent in scaling: one he comes up with is the heterogeneity that creeps in when systems are expanded by adding hardware.

Heterogeneity means that some nodes will be able to process faster or store more data than other nodes in a system and algorithms that rely on uniformity either break down under these conditions or underutilize the newer resources.

It's a good point, and it's another compelling reason to use grid behind your SOA or SaaS. Why?

Because when you use a grid infrastructure, you don't have to write the code to handle the heterogeneity. Rather than have to write the algorithms to distribute your code, use a grid system that was designed for that task. It will ensure that your newer, high powered servers are utilized to their fullest potential and will also use your older servers as effectively as they can be used.

Vogels is right. Scalability is difficult. Scalability is important. My suggestion? Throw a grid at it!

Technorati tags: scalability, grid computing

MSDN Webcast: Object Oriented Programming for Grid

If you've seen one of my (or Kim's) presentations or attended a Digipede webinar, you've had a brief introduction to Object Oriented Programming for Grid.

I've been asked by Anand Iyer, a Microsoft Developer Community Champion, to give a couple of webcasts this month to go in depth on this technology. I'm thrilled at the opportunity.

We've outlined two courses; the first will be an introduction to Object Oriented Programming for Grid, along with general information about grid computing. In the second, we'll dive in deeper and show practical applications of grid: using a grid behind a forms app, behind an Excel spreadsheet, and behind a web service.

Sound interesting? The first is a week from tomorrow. Tuesday, April 11th, 11:00 PDT (that's 1900 GMT for you internationalites).

Register here.

The second one is April 25th at 11:00 PDT, and the registration is here.

Technorati tags: grid_computing, digipede, msdn, OOP-G

Cool Financial Applications on Windows

Here are some of the cool Microsoft Partners I met at the Financial Services Partners Summit. As usual, the best part of the summit was meeting other ISVs and find out what amazing things they're doing on the Microsoft Platform.

TAPSolutions: Their TAPMaster product acquires, stores, and manages market feed data.

Xenomorph: High performance database product, along with analytics and pricing tools to go along with it.

Eze Castle Software: Named for a real castle in France, Eze Castle has created a whole fleet of software for investment management firms.

ClusterSeven: They have tools and technology for managing, analysing and auditing Excel spreadsheets--a critical tool for compliance, among other uses.

Unrelated note: this is my 100th post at West Coast Grid. I've certainly enjoyed writing this blog so far, and I'm glad to get comments, questions and notes. I look forward to continuing this conversation!

Technorati tags: Microsoft partner, TAPSolutions, Eze Castle, Xenomorph, ClusterSeven

Microsoft Capital Markets Partner Summit

John and I spent Wednesday in Redmond at the Microsoft Capital Markets Partner Summit: a group of ISVs who provide tools into the Capital Markets space, and the Microsoft team that supports them.

There were some interesting companies there--I'll have another post on that later.

It was very interesting to see how Microsoft is changing their approaches to serving their customers.

On one hand, Microsoft doesn't sell very much direct to customers--indeed, something north of 95% of their revenue comes through partners.

On the other hand, Microsoft has a vested interest in talking with customers, ensuring that the end customers understand both the value of the OS and the value of the tools available on that OS.

That means that Microsoft has to work closely with their partners when talking to customers. The partners--ISVs--need to sell their products; but it's Microsoft's job to sell the platform.

For years, when a large financial company wanted to buy a large system, they went to a consulting company--say, IBM. IBM would put together a team of companies: hardware, OS, middleware, software vendors, and consulting services. It was an easy way for the customer to buy: one company was championing the whole thing (IBM in this example, but there are others). Moreover, from the customer's perspective, there was one "throat to grab" if things went wrong.

Microsoft wasn't selling that way at all. Microsoft got into financial companies the way they got into every other company on earth: selling operating systems for desktops and office applications. Then, in the late 90s and into the 21st century, a funny thing happened: Microsoft wrote a server operating system and a powerful database. Software vendors started writing high end packages on the platform.

But there's a rub: how do you sell this? Microsoft didn't write applications for finance, nor did they employ experts in the field. So they didn't have an effective way to tell potential customers about the virtues of their platform. And the software vendors knew a lot about their area of expertise, but they weren't heavyweight enough to push platform decisions on their customers. (I ought to know--I was writing enterprise software built on NT, SQL Server and IIS back in 1997. The platform was a tough sell)

In the last couple of years (especially in the last year), Microsoft has learned many lessons about selling into these types of enterprises. They have hired aggressively, bringing experts on staff who can speak the language of finance while preaching the benefits of the platform. They've also pursued the integrators: with Accenture, they created Avanade. They also work with many of the other big integrators.

They're holding events like yesterday's Capital Markets Partner Summit to help create a sense of community among partners--partners who may bid against each other on some deals, but are even more likely to have complementary products in many instances.

Finally, they're providing coordination as these efforts go forward. They can approach customers not piecemeal, but together: the platform, the software, and the integrators to make it work. And, by taking the lead, Microsoft is essentially offering up theirs as the "throat to grab."

Microsoft is doing the exact same thing in other industries, too (banking, manufacturing, health care, etc). They are helping us (the ISVs) bring our products to the market.

Most importantly, they are giving their customers a better, more complete picture of what they can have when they choose this platform: OS, database, software vendors, and integrators, all one one page.

It was a great couple of days. Thanks to Ed Muth, Kenny McBride, Rich Feldmann, Stevan Vidich, Christina Fritsch, and everyone who gave the great sessions.

Technorati tags: microsoft partner, microsoft

In Redmond tonight...

I'm in Redmond tonight and tomorrow for the Microsoft Capital Market Partner Summit--a gathering for partners with products and services in the financial realm.

John and I are going to a reception tonight, then we'll be painting Redmond, um, red. Or going back to the hotel and working. We'll see what happens.

We had a productive meeting with one finance ISV already this week--I hope this summit leads to more!

Anyone recommend a dinner place in Redmond? Anyone in Redmond want to talk distributed computing? 510-816-7551.

Technorati tags: microsoft, microsoft partner

What a week!

It was a busy week in Digipede-land, and I haven't found any blogging time at all.

We've been in final QA on our new release, Digipede Network 1.2. Most of the last few weeks has been install and upgrade QA. As it turns out, that's a bunch of work! The new version is fully compatible with both .NET 1.1 and .NET 2.0, and supports mixed-mode operation (some agents running .NET 1.1, some running .NET 2.0, and some running both, and all combinations thereof).

The week was made all the more interesting because my co-worker Nathan's wife had a baby! Exciting for them, and Nathan has been a trooper (literally dashing out of the hospital to hand software off to me on a USB key at one point). But he hasn't been in the office, which means I've been directly supporting customers and potential customers. That's always fun (as a product manager, talking to customers is probably the most rewarding part of my job)--but it's hard to be direct support when trying to get a release out the door!

But all's well that ends well. The release is done (!), and we've got at least one new customer this week (I hope we can release some details about it soon). We've also had some good talks with potential partners.

Technorati tags: digipede

Happy Pi Day

Happy Pi Day, everyone!

Technorati tags: pi day

How will they host it: Writely or wrongly?

Everybody has heard that Google bought Writely today. Congratulations to everyone involved.

Not everyone knows that Writely is built on .NET.

My question: who's going to host it?

Is Google going to port Writely to run on the Googleplex? Will they host it on .NET? Does Google already have .NET hosting ability? Will they use Mono?

To me, it's one of the most interesting facets of the acquisition.

Technorati tags: google, .net, writely

It's the Infrastructure

Update 2006-03-14 11:31 Changed a nonsensical word.

I spend a lot of time explaining the value of what Digipede does to people: whether it's potential customers, potential investors, my mom, or people riding next to me on BART--it's something I like talking about, and it's something I believe very strongly in. Add that together with my love for mellifluous voice, and it's a recipe for me talking about Digipede a lot!

One thing that's come up frequently lately is the idea of grid infrastructure.

Before I go too deeply into that, let me explain something about our customers. I can divide our customers neatly into two camps: customers who bought the Digipede Network to accelerate/scale/distribute a particular application, and customers who are using the Digipede Network for a much broader suite of applications.

To the former group of customers, it's a tool. They had a particular need for a particular piece of software, and they found that the Digipede Network filled that need.

To the second group of customers, though, it's much more than just a tool. It's now part of their development and IT infrastructures. We have one customer who is running at least three different applications on their grid now, and they will undoubtedly do more: they are porting current applications that need increased speed or scalability, but they are also looking at developing new applications. Their developers don't just have a new tool they can use: they also have an infrastructure that allows them to do something they've never been able to do before.

Having a grid infrastructure in place means that developers can begin to take on assignments that seemed impossible before--analyzing more data, or deeper analysis of a particular trade. Having much, much more powerful software means that the users' lives change. Rather than analyze hundreds of trades, they can analyze thousands. Rather than run a trade run once a day (at night, when everyone has gone home), they can run it frequently throughout the day.

A good grid infrastructure does more than just speed up an application. It changes the way the developers work. It changes the way the users work.

Technorati tags: grid_computing, infrastructure, digipede

Snark it up

Scoble, Carr, Winer and Searls have been snarking it up lately.

Robert has written a hilarious XML extension that will finally allow everyone to have full control over their snark...

Snark it up:

To this end, I want to introduce the HyperText Snarkup Language (HTSL) which will initially be described as simply an extension of XHTML with a namespace. This will allow publishers to have full control over their snark.

Technorati tags: snarkiness htsl

How's the generator business?

Nicholas Carr is vigorously defending his stance (Is the server market doomed?) in his latest post (More thoughts on servers).

I think he's still missing the boat.

In response to some counterarguments from Charles Zedlewski and John Clingan, Carr says:

Both argue that if servers become more efficient (through virtualization, for instance), then companies will tend to buy more of them, not fewer. If a product becomes more valuable, after all, you'll want more of it. That's a great point (for unit sales, if not for revenues), though I'm not sure it applies in this case. It's important to remember that what's really being consumed is computing cycles, not servers; through consolidation and virtualization companies may both consume a lot more cycles and buy a lot fewer boxes.

I don't think Carr gets the point at all. He makes a great realization ("what is being consumed is compute cycles"), but he doesn't really follow through with the thought.

The history of computing has shown very, very consistently: the consumption of compute cycles is on an ever-increasing path. Why? Because the faster computers get, the more uses people find for them. Carr seems to be implying that we have finally reached a point that our servers are doing everything they could possibly do: from here on out, making them faster will just diminish the number of servers we need.

He writes "companies may both consume a lot more cycles and buy a lot fewer boxes," but his argument sounds more like "if the number of cycles they need doesn't increase too much, they'll be able to buy fewer boxes." That would doom the server market indeed.

But that's not what happens with computers. Computers get faster. With each increase in speed, incredible new uses are found. They tax the machines. They need faster ones. Repeat.

Virtualization is another great use for machines--but it won't keep software developers from innovating, and it won't keep companies from inventing new, faster servers. Those things will continue to happen.

Moreover, Carr keeps blurring utility computing into his argument. Quoting Frank Sommers, he says:

And with standard application interfaces, such as J2EE, shouldn't a company's IT department be able to deploy an enterprise app into a remote data center's hosting environment?

Ah, there's the rub. There is no one standard. As I pointed out earlier, computes are not all alike.

While I believe strongly that there is a market for selling computes and for data centers, there is no way that will doom the server market.

And one last point to show that, while electricity is not computes, even the electricity analogy doesn't spell doom for the server companies. There was tremendous consolidation in the electric power industry when the idea of a "power plant" came about. But did that kill the industry that manufactures generators? No--there are still companies making billions of dollars manufacturing power generation equipment (I used to work for one of them). There is still tons of research going into ways to make power better.

As I said before: the server industry isn't doomed. It's evolving.

Technorati tags:grid computing, distributed computing, utility computing

Because 1.21 gigaflops just aren't 1.21 gigawatts

In his normally insightful blog this week, Nicholas Carr made a rather off-the-wall suggestion. He posits that the server industry is doomed, and as proof he writes about trends he sees happening: shifts away from high-end servers toward either blades or grids of commodity, off-the-shelf hardware (COTS).

He cites two examples: Sumitomo Mitsui Bank reduced 149 traditional servers with 14 blade servers, and Google runs all of its software on machines it assembles itself.

Of course, I don't think these two examples prove much at all. After all, the blade systems that he is referring to need to be supplied by somebody--and I think we will continue to see the traditional server manufacturers continuing to move in that direction. Blades won't kill the server market; they'll be part of it.

As for Google ("It buys cheap, commodity components and assembles them itself into vast clusters of computers"). Not every company is Google--we can't all buy so many machines that no one even notices when one dies. We can't all have people on staff to build our own machines, then spend their days roaming our aisles of racks pulling out the dead ones. Many of us depend on the quality that the server manufacturers deliver. Google isn't a typical company, or even a typical web company; one might say they're unique. So I don't think their lack of name-brand servers is a harbinger of doom.

But Carr gets way off-track when he then suggests that utility computing will kill the server industry:

If large, expert-run utility grids supplant subscale corporate data centers as the engines of computing, the need to buy branded servers would evaporate. The highly sophisticated engineers who build and operate the grids would, like Google's engineers, simply buy cheap subcomponents and use sophisticated software to tie them all together into large-scale computing powerplants.

I've seen many references to utility computing before, and I just don't buy it.

Partly, it's just physics. All electrons look alike (let's not get into electron spin here: as far as my appliances are concerned, every electron looks the same). It doesn't matter to me if the power that's lighting up my life, running my refrigerator, and powering my PC came from a wind farm, a hydroelectric plant, or a diesel turbine. Well, for environmental reasons, I might prefer the former two, but the point is that when an electron gets to me, I can't tell where it came from.

Computes just aren't the same. Computes look different on different operating systems. Not all software runs on all operating systems. Different people prefer different toolsets, and they always will. Some OSs are better for some things than others, and people choose the appropriate OSs for them. Yes, we've all read about "write once, run everywhere" software--but a small minority of software actually runs that way. OSs are different, and they will continue to be different. People will continue to write software that takes advantage of particular OSs.

Not all compute problems can be "shipped out" easily. There are huge data concerns. First of all, there are the ubiquitous privacy and security issues: some data people just don't want leaving their building.

Beyond that, though, there's the issue of data size and compute-to-byte ratio. If I need to do a quick search of a huge dataset I just collected from my genome lab or my jet design windtunnel, it may not make sense to move that to a "computing powerplant." Heck, I may be collecting tons of data in real time, and I need to analyze it in real time. I need my computes where my data is. As Jim Gray says, "Put the computation near the data." If your data are with you, that means your computes are with you as well.

Don't get me wrong: I'm a big believer in distributed computing, and I'm a big believer in grid computing. But I don't think that, in the future, I'm going to flip on the "compute switch" the way I flip on a light switch today.

Is the server market changing? Of course it is. Blades, virtualization, distributed computing: these are all changing the needs of the market. There will continue to be a high end market. There will continue to be a low end market. But utility computing will not kill servers.

1.21 gigawatts? What was I thinking?

Technorati tags: grid computing, distributed computing, utility computing, google

New SaaS book coming

Scanning Robert's Expert Texture link blog, I see a link to Fred Chong over at MSDN. Fred is a solutions architect at Microsoft, and he's been thinking a lot about SaaS.

In his latest post (SaaS is a journey, walk with us), he gives the table of contents of a book about that he and Gianpaolo are writing about SaaS.

The outline looks great; his summary of key points for architects is perfect:

Scale the application

Enable multi-tenant data

Facilitate customization

At my previous startup, Energy Interactive, we offered a product in the electric industry that we offered as an ASP (that's what we called SaaS back in the twentieth century). We went through everything Fred describes--only we weren't lucky enough to have a book to help us along!

The issues we had with scaling are part of what convinced us that we needed to start Digipede--it just wasn't easy to scale an application. When we looked around our datacenter and saw so many servers that could have been used (and even more when we looked around the rest our enterprise), we wanted an easy way to take an application and scale it out. The tools just weren't out there.

One nit with the table of contents: Fred's chapter on scaling doesn't seem to address distributed or grid computing at all. I'm a little surprised by that. Given the adaptability of grid to service oriented architecture (and I certainly view SaaS as a flavor of SOA) that has been noted lately by experts like Lee Liming and Greg Nawrocki, it seems that Fred and Gianpaolo would mention it. Fred lists the following issues:

Pools: thread, connections etc.

Async

Locks

States

UI/Presentation

How can you scale that application to a cluster? To a data center? How do you plan your hardware so you can handle peak demand without overspending? Whether grid is the answer or not, I'm certain that much SaaS will involve some flavor of distributed computing, and I hope these two go into some detail there.

By the way, it looks like Fred and Gianpaolo are going to cover a lot more than just technical issues--they've got chapters planned on everything from Business Model to Security to Instrumentation and Monitoring. That's fantastic. SaaS is so different in so many ways from traditional enterprise software, and I'm glad to see Microsoft folks lending their considerable wisdom and experience to help people along.

Technorati tags: digipede, grid computing, distributed computing, SaaS, SOA

Case Study: Distributed Computing in Financial Services

We've got a new case study up over at the Digipede site, and it's one of the most exciting customer stories we have.

The customer is a large financial services company (who, being very secretive about the methods of their success, prefers not to be named). Nameless, but big: they manage scores of billions of dollars.

They do a lot of analysis, naturally--minimizing value at risk, satisfying portfolio constraints, and managing transaction costs.

They have a great .NET development team, and they had been developing their own distribution system to farm all of this work out across 20 servers. They made what is a common discovery: they were spending more time writing the code to do the distribution than they were working on their own algorithms. They looked around, and they found the Digipede Network.

Within a week they had a proof-of-concept implementation, and within two weeks of that they were ready for production. That was my favorite part--their lawyers took longer to approve the sale than their programmers did to port their software!

I'm not bringing this up just to brag about a great product. It really speaks to the core of how important a good development framework is. By allowing their .NET and COM developers to take advantage of many machines without rearchitecting their solution, the Digipede Framework simply removed distribution as an obstacle. They are now using the Digipede Network for a variety of applications--everything from C++ to VB to C#, using both COM and .NET.

By bringing in a third party tool, they now have a distribution system that's more scalable, more powerful, and, most importantly, lets them concentrate on their core competency.

This is a scenario I hear about all the time. Developers need to scale an application (maybe it's a web application, maybe a 3-tier application) so they start down a path of multi-threading or .NET remoting (or DCOM or COM+). But soon they realize that the distribution part is hard (especially if they want robustness, good monitoring and reporting, guaranteed quality-of-service, etc.).

Technorati tags: digipede, grid computing, distributed computing,.NET, scalability

Windows Grid: SDForum Talk Wednesday Night

For those of you in the Bay Area: I'm giving a talk at the SDForum Windows SIG in Palo Alto on Wednesday night.

The content will be similar to the talk I gave at Code Camp Seattle, but with a longer slot, I'm going to be able to go into more detail.

I'll give the basics of Object Oriented Programming for Grid, and I'll build some applications from scratch. I'll then grid-enable an existing application or two, showing how you can take an app and "retrofit" it to run on the grid.

The last thing I'll do is a brand new demo (and one that I'm really excited about): I'm going to talk about grid and SOA, and why grid can help scale an SOA well. I'll also demonstrate a Web service application that's been grid enabled, and show how that improves its scalability.

The whole shebang starts Wednesday at 7:00 in Palo Alto; details available here.

Koppel and Nawrocki on Grids and SOA

As I sat on the couch for the second consecutive day fighting a miserable cold, two of my favorite blog postings of the year popped into my inbox (thanks, Attensa).

First was Muli Koppel's blog. Are you reading this yet? He writes what is positively the most intelligent blog I've ever read. He doesn't get the attention that some other bloggers do (because he doesn't write about Web 2.0) but believe it or not: there are really smart people writing things that aren't Web 2.0. Muli is one of them. Every post is well thought out and well written, and I read each several times. He has more content per word than anyone blogging.

Anyway, Muli's post today was called Meant for Each Other: Enterprise Architecture and SOA. He begins with the most practical definition for Enterprise Architecture that I've ever read:

Enterprise Architecture is an infrastructure and a set of Machines constructed in order to manage a chaotic, dynamic, unpredictable, complex, organic, prone to error, frustrating, Enterprise IT, which has to support an ever increasing, dynamic portfolio of products and services, through constant "ASAP, Now, Right-Away" modifications of business processes.

He goes on to talk about the discussions that are happening in the industry about EA and SOA, and he sums it up thusly:

EA will shake off its image as a "documentation, procedures and guidelines" body, repositioning itself as a practical, implementation-oriented discipline aimed at the creation of an Enterprise Management Infrastructure, while SOA will be repositioned, no longer as an Integration/Interoperability architecture, but rather as an Enterprise Management architecture.

One of Muli's gifts (both as he blogs and, I surmise, in his career) is to make sense of the buzzwords of the day in practical terms. Seriously, stop reading here, and go read his post. Then come back.

Of course, now that you've read it, you know my favorite quote from the post. I'll quote it here:

There are, clearly, other infrastructures and machines to add to this picture. For instance, grid/utility computing is an essential part in the real-time Enterprise...

Ok, enough stealing Muli's material. The other post that fell into my inbox today was from Greg Nawrocki's Grid Meter: today's SOA developers .... tomorrow's Grid developers.

Greg wants to know why, while SOA and grid have been hyped for a few years now, SOA is gaining much more traction.

As reasons for SOA's success, he notes that there are a wide range of languages available (I was glad to see him include the .NET languages along with Java), but notes that the real reason is "that you can actually take relatively mainstream enterprise applications and write to a service with a simple API." He's right of course. While as many people (including Muli have pointed out), SOA ≠ Web services, SOA is certainly enabled by web services.

Greg notes that the Globus Tookit 4.0 embraces WSRF and he points to an IBM paper that details building a grid and implementing Globus.

Greg is right on in noting that there is a real convergence between grid and SOA. SOA cries out for a scalable architecture, and a Web service enabled grid is perfect for building that architecture. But as long as an implementation requires a lengthy consulting contract (which, after all, is why IBM is writing papers about it), grid can't really take off.

People need tools that are easier to use and grid systems that are easier to implement. Globus is the pre-eminent standard toolset for building a multi-organizational grid, but its implementation is too difficult for a small to medium sized venture to undertake.

In the meantime, of course, there are vendors who are making the tools to make these types of architectures available today. (We're not the only ones, but we're the only ones doing it on .NET). Anyone who wants a grid right now (not in the future) should be looking to these toolsets to help them build that grid today.

Technorati tags: digipede, grid computing, SOA, globus, wsrf

Tame Your Technology

I was interviewed this week by Neal Miller, host of Taming Technology. Taming Technology is a radio show that tells its listeners "how to survive and succeed in the high tech jungle."

Neal is quite an affable guy, and he and I spent about 45 minutes discussing grid computing (and, of course, I got in a plug or two for the Digipede Network). The topic for the episode was "Grid computing--what is it and how is it being used?"

If you'd like to listen, watch the Taming Technology website--it'll come up soon. Or you can listen to it when it first streams on the VoiceAmerica Business channel--tomorrow at noon, Pacific. (VoiceAmerica is "the industry leader in Internet talk radio").

It was my first long format interview; I think I was wordier than I should have been. I hope to do these more in the future, and I hope to get more concise when I do it. If you listen this weekend, let me know what you think!

Technorati tags: digipede, grid computing

4th Story Gets Faster

If you read Bobsguide (the guide to software and technology in the finance industry), then you've already seen this: Digipede and 4th Story Join Forces to Grid-Enable Financial Software. But if you don't, I'll tell you a bit about it.

4th Story is a firm with deep knowledge in two areas: finance, and software development. Their products (all named after National Parks and Monuments) do incredible things for both buy-side and sell-side firms. They've got some huge clients (Barclays, for one), and they do amazing things with analytics.

They've developed their whole suite in .NET. As they've gotten into very complex algorithms (think genetic algorithms) and large amounts of historical data, they've found the need to take advantage of multiple machines in order to scale properly. Rather than write that portion themselves, they decided to plug into the Digipede Network. Without rearchitecting their solution, and by taking advantage of our OOP-G methodology (Object Oriented Programming over Grid), they were able to quickly modify their software to automatically take advantage of the Digipede Network. Their objects are now being distributed across the Digipede Network and executed in parallel.

The result? They've now got a very scalable solution. Their customers will be able to scale the 4th Story suite as much as they want--just by adding more machines to the grid.

4th Story will be the first ISV logo on the Digipede partner page--but we've got a few more deals in the works, so look for more soon!

Technorati tags: digipede, 4TH Story, OOP-G, grid computing

Catching Up Is Hard to Do

You can tell I'm busy when the only post I can come up with in 3 days is a "check this out" post. Still, this one has some good reads. Check this out.

I just discovered Jeff Schneider's blog Service Oriented Enterprise. This guy knows his stuff. The first post I found (The Sheep that Shit Coleslaw is a great representation of system architectures using Legos as examples. Did I say "Legos?" Yes, I did. If you're a fan of SOA or toys with interlocking pieces, check it out. His points are not dissimilar from what Kim said the other day. But when you say it with Legos it's so much funnier! Jeff, consider me subscribed. And I've got a lot of catching up to do. Jeff's been blogging since September 23, 2001!

The other interesting find I had this week was Nicholas Carr's presentation on utility computing and open source. I hope I can find some time to write a bit more about it. In the meantime, check out his pdf here.

Digipede Webinar Tomorrow

If you're interested in seeing a webinar about the Digipede Network, I'm giving one tomorrow at 10:00am PST.

The presentation isn't given in "marketingese." I'm going to give an overview of how our distributed computing solution works, but then I'm going to dive into real code. I'll use Visual Studio 2005 and VSTO to take an Excel spreadsheet with .NET code running behind it and grid-enable the .NET code to run on the Digipede Network.

If you've been wondering what this Digipede thing is all about, this is your chance. It runs about 30 minutes long. Audio will be available via a call-in phone number.

Send me an e-mail (click the link at right) or fill out the form here.

Technorati tags: digipede, visual studio, vsto

GMail: It's a SaaS World After All

[Update: 2/13/2006 12:02 PST]: I just realized that Google's first implementation is at San José City College, not San Jose State. Updated accordingly.

Over at Expert Texture, Robert had a great post yesterday raising concerns about Google branding GMail (Corporate GMail). According to their post here, Google is going to start offering GMail to corporate customers--with their domain names. They're starting by taking over San José City College's e-mail.

Robert raises a couple of salient issues:

How will GMail isolate company data and ensure that it never gets shared with other customers?

How secure is the GMail database? Leaving aside the issues raised by the EFF yesterday, what are the security procedures in place at Google to ensure that customer data are secure?

Those are great questions. Few pieces of software contain as many corporate secrets as e-mail servers. In this day and age, virtually all communication goes through e-mail servers (much, much more gets done this way than through phones or IM)--and that means that corporate secrets are in that system.

Certainly many small-to-medium companies don't worry so much about intellectual property and therefore won't worry so much about this. After all, if I were operating Dan's Frisbee and Waffle Shop, and corporate GMail would give me the ability to have e-mail addresses like dan@dansfrisbeeandwaffleshop.com without my investing any time, staff, or money in hardware or software, I may think that is terrific. Who cares if someone finds out that I'm introducing a new flavor of syrup next week?

But if I'm a large corporation (and let's face it: by announcing the service with a 10,000+ account implementation, Google is is making it clear that they aren't aiming at the frisbee and waffle shops of the world), I have grave concerns about putting my most secret intellectual property in the hands of any other corporation; even if their motto is "Don't be evil."

But in a larger sense, this same question is being faced all over the software industry as people create and offer Software as a Service. And in many cases, companies are finding ways to assure their clients that their secrets are safe. Look what Salesforce.com has been able to convince companies to do: hand over all of their customer and sales data. For many companies, those secrets are as valuable as the ingredients to their special sauce.

SaaS companies are finding ways to convince customers that their data are safe (from both outside and inside vulnerabilities). But it will continue to be a major issue as SaaS becomes a more frequent business model. And, undoubtedly, just as we hear today about credit card companies accidentally exposing credit card numbers, in the not-to-distant future we'll here of some SaaS provider who inadvertently revealed the ingredients of the special sauce.

So, SaaS developers, I give you this. I've already told you that you have to design your system to be scalable from the outset. Now: make sure you're thinking about security from the ground up, too.

Technorati tags: Google, GMail, SaaS

Web Services + Grid = Crazy Delicious

The other day I happened to glance at the March 2005 issue of Cluster World magazine that was sitting around in the office. One article in particular caught my attention (sorry I can't link to it; ClusterWorld.com's "Old Stuff" link is broken, but it's also available here in PDF format). Lee Liming of Argonne National Lab wrote it; it's called Web Services and the Grid.

Lee's premise (and it's a good one) is that clusters and grids are a natural tool for developers of Web services.

As clusters become increasingly essential elements of the Grid's physical fabric, Web services are becoming essential elements of the Grid's application development toolset. Given the importance of clusters and Web services to Grids, cluster owners and operators need to understand the implications of Web services on the applications that run on their clusters.

He's right. Web services require scalability. One of the inherent truths of creating good web service applications is that you must plan for success: if you've written your software well, people will use it. You have to be prepared for that success and therefore prepared to scale your application. Scaling to a cluster is a perfectly good way to do that. Scaling to a grid is even better. Why? Because it's more cost effective. As you need to grow, adding processing power to your application is as simple as adding more nodes to your grid. Even if the grid is supplementing a dedicated cluster, it gives you the processing power you need at peak times without the expense of dedicated hardware that sits idle 95% of the time.

Note that unless the Web service has been designed to use the back end nodes on the cluster, it will only use the cluster's head node. Web services that require significant processing power should be written to submit tasks to back end nodes via the cluster's scheduler or other tools. The application developer must explicitly implement this capability.

This is an important point that many people miss or think that Network Load Balancing will handle automatically. In fact, NLB is great for serving web pages from multiple servers, but is not a good tool for handling compute-intensive process (for a list of disadvantages of NLB, see Brian Madden's article How to Configure Network Load Balancing).

Back to Lee's point: if you want to take advantage of the power of a cluster or a grid, you need to do that yourself. Of course, I happen think that you should simply get yourself a great toolset that will handle the job submission, guarantee execution, monitor the processes, handle node failure, etc. That frees you up to concentrate on the functionality of your Web service, rather than the technical details of scalability.

Lee sums it up like this:

...it seems likely that service-oriented applications—such as those based on Web services—will lead to significantly greater use of clusters (i.e., more business) than traditional, manually launched applications. Early efforts to gear clusters to become high-power hosting environments for these types of applications will position administrators well in a service-oriented era.

Lee is obviously talking to vendors who offer hosting solutions. But his points are broader and apply to anyone who is designing and building Web services or SOA. If you're writing software that will be available as a service (Web service, SOA, SaaS--you choose your favorite), grid it up and let it fly.

(And if you don't get the reference of the title of this post, you must not have seen the Chronic of Narnia yet, or don't understand the allure of Mr. Pibb and Red Vines).

Technorati tags: grid_computing, distributed_computing, cluster, webservices

New Case Study!

Over on the Digipede site there's a new case study about a Digipede customer: Trekk Cross-Media.

Trekk is a forward-thinking marketing company with high-tech savvy. They came up with an interesting product--for their clients (large retail chains), they will create direct-mail packages that include maps from homeowners' houses directly to the nearest chain store. They wrote the application themselves, and their customers liked it a lot. In fact, too much. The product was a hit in test markets, and the customers wanted to take it nationwide.

But Trekk's application could only produce 750 maps per hour. To handle large jobs would take days. They looked into a difficult and expensive option: converting their application to a multi-threaded architecture, then buying expensive new hardware.

Luckily, before they got too far along that road, they found Digipede. Using a trial copy of the Digipede Network, the Digipede Workbench, and a command-line version of their product, they quickly determined that they could get the scalability they needed. They bought it. Using the Digipede Framework SDK, they modified their application to integrate directly with the Digipede Network; within a few days, they were in production.

The case study has some nifty pictures and some quotes from Jeff Stewart of Trekk. You should check it out.

What I love about this project was how quickly it all happened. Jeff attended a webinar and decided to try it out. Workbench let him distribute his command-line tool immediately. And after deciding to go with the Digipede Network, they were up and running within days--all without a single visit from Digipede. They did everything themselves. I like to think it's testament to the ease of use we bring to Windows distributed computing, but I'll have to give them credit for being savvy software developers as well.

Technorati tags: grid_computing, distributed_computing, .net, digipede, trekk

Slow and steady wins the race

I don't know if I've ever made a New Year's resolution in my life. They've always rung hollow for me--if I need to improve myself, I don't need to wait for the change of year to start doing it. In that sense, they seem more like a way to postpone self-improvement more than initiate self-improvement (i.e., the fellow who says to himself in mid-November "I really need to take off a few pounds; that'll be my New Year's resolution," then proceeds to gorge himself through two months of holidays before dieting for the first few weeks of the New Year).

Well, this year that changed. I finally found a resolution worth doing. Not for the first time in my life, I'm going to copy my older brother.

Last year, my brother Dave had a great resolution: on January 1 he did a pushup and a situp. On the 2nd, he did two pushups and two situps. And so on. By April, he was doing 100 of each per day. By July, 200 per day. And, of course, on December 31st, he did 365 pushups and 365 situps. And during the course of the year, his physique changed drastically. In September, when he turned 40, his wife gave a toast at his birthday dinner and said that he looked better then than he had in their entire relationship.

So I decided to do it this year. It's a fun resolution because it starts so easy (it felt almost silly when I got down on the floor and did one pushup; but I did it, and I was on my way). Even now, 31 days in, it's a very brief "workout," but it's starting to feel like an honest set of pushups (and crunches).

The sheer numbers involved get staggering. Over the course of the year, I'll do 66,795 pushups (you can figure this out in Excel, or you can use the fun method: 1 + 2 + ... + n = (n + 1) * (n / 2)). I know I'll be breaking them into sets eventually, but for now it's fun to see how long I'll be able to do them in one set.

What's making this even more fun is that a bunch of my friends are joining me: John, Robert, and Nathan from Digipede are doing it, and my old friend Marc. It's great having other people doing it, because we can harangue each other to make sure no one is sliding.

I'll do my 500th pushup and situp of the year tomorrow morning; I'll hit 1,000 on Valentine's Day and 10,000 on May 21st. On my birthday in August I'll do my 26,000th situp. And in the month of December alone I'll do over 10,000. I hope I (and all of my friends) can stick with it. I think the inspiration for me is to see what I can accomplish by dedication, tenacity, and an incremental increase in effort. And the next trick will be to apply those principles to other parts of my life (certainly working at a startup for the last two years feels similar!).

Duke Listens! : Weblog

Over at Paul Lamere's blog last week, he had a post about The simplest possible grid computing platform:

In his Ongoing blog Tim Bray mentions that he'll be giving a talk at this year's JavaOne called: Sigrid: The simplest possible grid computing platform. If I make it to JavaOne this year, I'll put this talk at the top of my list. There's a big gap right now in Grid Computing. There are Grid Computing APIs, but these seem to be designed for the traditional big iron apps like those used by chemical companies and Wall Street. It is hard to write grid apps with these APIs. In order to get more programmers and companies to think about moving their apps to the grid, it has to be easier to write grid apps. In particular this means:

Use a friendly programming language like Java

Be able to develop and test on a desktop system

Have a minimal API

That's a great idea, Tim. Of course, not everyone programs in Java, so that solution isn't for everyone.

I love the qualifications, though, because they apply perfectly to the Digipede Network. Friendly language? See my post from yesterday to see a comparison of C# and Java (of course, you can use the Digipede Network in any language that has a .NET or COM interface). Test on a desktop system? I run the whole thing on my laptop routinely. Minimal API? If you haven't seen my webinar where I grid enable an existing app in 20 lines of code, drop me a line and I'll invite you to the next one and show you the coolest API known to grid computing!

As soon as we get permission, we'll publish case studies about companies that have ported their systems to our APIs in less than a day!

Technorati tags: grid_computing, java, .net, digipede

(Nearly) VSLiveBlogging

It looks like Robert is LiveBlogging from VSLive (well, almost live).

I don't know how good his connectivity will be, but if you're interested in what's happening at VS Live SF, follow him at Expert Texture.

Technorati Tags: VSLive

AgentMine.com on C# and Java

Over on AgentMine.com there is a great post comparing C# to Java Paul Graham’s Essay “Java’s Cover”:

In 2001, Paul Graham wrote an essay about Java in which he summed up his opinion of Java thusly:

Java seems like a stinker to me.

Graham then goes on to give a 12 point list of why he thinks Java stinks. I won't go into it point-by-point, because AgentMine does.

AgentMine then does something very interesting: he takes the same 12 points, and applies them to C#. He looks at them to decide if, well, C# seems like a stinker. As it turns out, C# has a much better score than Java.

His final analysis:

The results:
Microsoft’s C# language should only have 25% of the total level of suck that Java has.

It's tongue-in-cheek, but it's very informative. Take a peek.

Technorati tags: java, .net, c#

And the winner is...

I love awards ceremonies as much as the next guy.

Well, to tell you the truth, I don't love awards ceremonies. I habitually avoid the Oscars/Emmies/Grammies/Tonies/YouNameIties broadcasts unless someone really cool is throwing a party. In fact, I think I've been avoiding awards ceremonies ever since my band, Lawsuit, was nominated for a SAMMIE Award (Sacramento Area Music Award) and invited to play at the ceremony, then suffered the indignity of performing immediately before finding out that we didn't win. Apparently the booker liked us more than the voters.

Anyway, here's an award ceremony I am excited for:

The Digipede Network has been named a finalist for a 2006 CODiE Award in the Distributed Computing Solution category!

For those of you not in the know, the CODiEs are awards presented by the Software & Information Industry Association (SIIA). This is their 21st year; they are the longest running, most prestigious independent award in the industry. Getting recognition from their panel of experts is fantastic.

The competition is fierce and widely varied: Everdream, Gigaspaces, Novell, and Solace Systems are all finalists as well.

Every SIIA member company gets a vote; the CODiE Awards Gala will be on May 16th (my mom's birthday!) at the St. Francis Hotel in San Francisco.

I hope that the wonderful, intelligent, good-looking SIIA voters (hey, a little brown nosing never hurt!) like what we're doing!

More .NET SaaS

For a while now, I've been preaching the importance of both .NET and grid computing in the area of SaaS; they're both enabling technologies.

I'm always happy to find validation.

Over on the dotnetSaaS blog, Glen Cameron has a long quote from Romesh Wadhwani of Symphony Technology Group.

Romesh writes a lot about the software industry (some of which is right on, some of which I don't entirely agree with). One part I do like is this:

Enterprises want unified platforms and we're increasingly seeing a move towards the use of grid computing. But the grid is still nascent. There is a lot of infrastructure, underlying operating systems, middleware and visualization tools that are missing or not robust enough today.

There is a significant opportunity to develop the enabling tools and technology that accelerate the delivery of applications and solutions on grid computing platforms.

Absolutely correct. I'd argue that the essential middleware components are increasingly available. Adoption is starting to happen now.

And, on a side note: I hope that Sam Ramji knows that there's a blog out there called dotnetSaaS. Sam promotes SaaS for Microsoft's Emerging Business Team, and he'll be glad to see it being promoted!

Technorati tags: grid computing, saas, .net

Windows grid: cheaper than you think

As I mentioned in my post yesterday (Grids: All HPC? All *nix? Not anymore), I promised to follow up on one of the arguments used against using a Microsoft OS on a compute grid: cost.

In Kim's eBig talk, one of the audience members mentioned why he thought that Microsoft was behind in grid computing, and he gave a concrete example. He talked about the cost involved in setting up a 256 node compute cluster. If you're paying $400 retail for each copy of Windows Server 2003, that adds $100,000 onto the price of your cluster (even $300 copies of XP Pro would run about $75K). As the audience member observed, that's nothing to sneeze at.

When you compare it to $0 for 256 copies of your favorite Linux flavor, the Microsoft solution does seem expensive.

However, on further reflection, it becomes clear that there are costs beyond OS, and that OS cost alone is not a fair comparison.

First of all, the example assumes a purpose-built cluster. This goes against one of the primary reasons for grid computing: taking advantage of the computers that already exist. If you want 256 Linux boxes in a typical organization, you need to go buy them. But if you need the power of 256 Windows machines--you probably already own them! Because Windows is the dominant operating system, your organization probably has thousands of Windows machines that can contribute to your grid.

Even if you need to buy some new hardware, you can have an "extended" cluster featuring a combination of dedicated hardware (in your cluster) and shared hardware (underutilized servers and/or idle desktops). By using existing hardware, you save not only OS costs but hardware costs as well.

Furthermore, the example only counts the operating system cost--not the cost of any other aspects of the system. Some of the most popular *nix based distributed computing solutions cost one to two thousand dollars per node! Sure, you saved $300 by getting a free OS--then you spent more than triple that on your grid solution. That dwarfs the cost of the OS.

And, it doesn't count what is often the largest cost of all--the cost of setting up the grid. Many of the *nix solutions are what we like to call "thinly disguised consulting projects." They are so complicated that setting them up involves hundreds (or sometimes thousands) of hours of consulting time. Some of the big consulting companies have excellent grid computing practices--but I assure you, they're not cheap. Put a small team of consultants on your payroll at $200 an hour for a couple of months and watch how quickly they, too, outpace your OS costs.

So, are the Microsoft OSs too expensive for compute grids? Probably not. Most likely, you can get an extended cluster using some existing hardware (and OS), some new hardware, add the Digipede Network (at under $200 a node), and set it up without an expensive consulting project.

One more X-factor here: as far as I know, Microsoft has not yet announced pricing for its Compute Cluster Solution, which will be out later this year. Who knows? When that price is announced, it may make the OS decision even more economical...

Grids: All HPC? All *nix? Not anymore.

Kim's eBig talk on Thursday night went very well. It was her first public speaking engagement as a Digipede evangelist, and I thought she did great.

Her audience was diverse and obviously very well versed in the concepts (and practices) of distributed computing.

One of the great things that Kim was able to do was to let the audience understand that distributed computing does not apply only to HPC or technical computing anymore. Even the attendees who had experience using grid systems or HPC systems before understood that, while distributed computing was once the hallmark of HPC, its applications today go far beyond technical computing.

One question that came up was why so much distributed computing occurs in the *nix OSs, and not on the Windows platform. A little discussion ensued, and the ideas that came forth were pretty accurate.

First, it was pointed out that much of the research into distributed computing has come from academia. Academics, of course, have always preferred UNIX and the Linux alternatives to Windows (I know that when I graduated from Berkeley in Computer Sciece in 1991, I had never used Windows at school).

Second, the commercial push to grid (as always, I'll alternate between the terms "grid" and "distributed" without going into the distinctions between the two) has come from firms with strong *nix leanings: IBM and Sun. Both have UNIX histories; more recently, of course, IBM has been pushing Linux.

The third reason that the group came up with was the price of the OS. If you are buying 256 boxes for a dedicated cluster, the OS cost becomes a large part of the cost.

There's no doubting the first two reasons. They're indicative of the history of grid computing, but things are changing. The third, though, is a bit of a red herring. Tomorrow I'll get into the reasons why.

Something to do on a Thursday...

If you're in the Bay Area, you should mosey on over to Pleasanton to see Digipede's #1 Evangelista Kim Greenlee as she gives an eBig presentation entitled "Grid Computing for Windows."

Kim's going to give a introduction to and history of grid computing, but then she's going to drill down on Windows. How do the concepts of grid computing trnanslate to a Windows environment? How can you take advantage of Windows machines?

If you're interested, register here.

Four Stars for Digipede Network

As I mentioned previously, Software Development Magazine published a review of the Digipede Network in the January edition. The verdict? Four stars!

Now they've got the article available online. Read the full article here.

On Demand in 2006

eWeek has been doing a series called "Innovations 2006 Analysis," in which it takes in-depth looks at technologies that will have great impacts in 2006. One of the technologies they highlighted was On-Demand Software, or Software as a Service.

The attention paid to SaaS sometimes confuses me a bit--SaaS has certainly been possible for 10 years now (my previous company, Energy Interactive, had a successful product called Energy Profiler Online, which has been sold as a service for nearly a decade).

So why the attention now?

The underlying technologies are now available that make SaaS better able to compete with traditional software offerings. The biggest difference between 1996 and 2006? High bandwidth internet availability. It seems archaic to think about now, but when we began selling EPO to electric utilities, one of our big concerns was the time it would take their users (who were often connected to the net via modem) to download the graphics we used for buttons!

Another change: the quality and security in OSs/web servers/databases/etc. No matter what platform you're talking about, great strides have been made in the security and quality of service available from the underlying tools. Microsoft's initial forays into these areas (early versions of IIS, NT, and SQL Server) did not provide the quality necessary for vendors to guarantee quality of service; the open-source community had some decent tools but were nascent at the time. In the ensuing decade, both camps have made significant improvements--Microsoft's tools have improved dramatically, and the open-source community has continued to improve Apache, MySQL, Linux, etc.

But I'll add one more innovation that has made SaaS a viable alternative to traditional software delivery: distributed computing. Key technologies have emerged in the last couple of years that make it easier to scale software across many machines; as noted in my previous post here, scalability is one of the keys to providing software online.

Back in 1997 when we were writing EPO, scalability was one of our biggest concerns--there was no easy way to take the compute intensive operations inherent in the software and distribute them across many servers. We had to write that code ourselves. It was laborious work, and it wasn't really part of our core competency at the time.

Nowadays, products like the Digipede Network make distributed execution easy. Instead of spending months hardcoding solutions to allow their software to scale, SaaS developers can write a few lines of code that add inherent scalability to their product. That, in turn, enables people starting SaaS companies to spend their programmer dollars adding features to their software. They end up with a product that not only has a better feature set (because they spent more time and money on it), but is also more scalable and robust (because they aren't using a jury-rigged solution for distribution).

2006 may be the year of SaaS--but if that's true, than it's the year for distributed computing as well.