Because 1.21 gigaflops just aren't 1.21 gigawatts

In his normally insightful blog this week, Nicholas Carr made a rather off-the-wall suggestion. He posits that the server industry is doomed, and as proof he writes about trends he sees happening: shifts away from high-end servers toward either blades or grids of commodity, off-the-shelf hardware (COTS).

He cites two examples: Sumitomo Mitsui Bank reduced 149 traditional servers with 14 blade servers, and Google runs all of its software on machines it assembles itself.

Of course, I don't think these two examples prove much at all. After all, the blade systems that he is referring to need to be supplied by somebody--and I think we will continue to see the traditional server manufacturers continuing to move in that direction. Blades won't kill the server market; they'll be part of it.

As for Google ("It buys cheap, commodity components and assembles them itself into vast clusters of computers"). Not every company is Google--we can't all buy so many machines that no one even notices when one dies. We can't all have people on staff to build our own machines, then spend their days roaming our aisles of racks pulling out the dead ones. Many of us depend on the quality that the server manufacturers deliver. Google isn't a typical company, or even a typical web company; one might say they're unique. So I don't think their lack of name-brand servers is a harbinger of doom.

But Carr gets way off-track when he then suggests that utility computing will kill the server industry:

If large, expert-run utility grids supplant subscale corporate data centers as the engines of computing, the need to buy branded servers would evaporate. The highly sophisticated engineers who build and operate the grids would, like Google's engineers, simply buy cheap subcomponents and use sophisticated software to tie them all together into large-scale computing powerplants.

I've seen many references to utility computing before, and I just don't buy it.

Partly, it's just physics. All electrons look alike (let's not get into electron spin here: as far as my appliances are concerned, every electron looks the same). It doesn't matter to me if the power that's lighting up my life, running my refrigerator, and powering my PC came from a wind farm, a hydroelectric plant, or a diesel turbine. Well, for environmental reasons, I might prefer the former two, but the point is that when an electron gets to me, I can't tell where it came from.

Computes just aren't the same. Computes look different on different operating systems. Not all software runs on all operating systems. Different people prefer different toolsets, and they always will. Some OSs are better for some things than others, and people choose the appropriate OSs for them. Yes, we've all read about "write once, run everywhere" software--but a small minority of software actually runs that way. OSs are different, and they will continue to be different. People will continue to write software that takes advantage of particular OSs.

Not all compute problems can be "shipped out" easily. There are huge data concerns. First of all, there are the ubiquitous privacy and security issues: some data people just don't want leaving their building.

Beyond that, though, there's the issue of data size and compute-to-byte ratio. If I need to do a quick search of a huge dataset I just collected from my genome lab or my jet design windtunnel, it may not make sense to move that to a "computing powerplant." Heck, I may be collecting tons of data in real time, and I need to analyze it in real time. I need my computes where my data is. As Jim Gray says, "Put the computation near the data." If your data are with you, that means your computes are with you as well.

Don't get me wrong: I'm a big believer in distributed computing, and I'm a big believer in grid computing. But I don't think that, in the future, I'm going to flip on the "compute switch" the way I flip on a light switch today.

Is the server market changing? Of course it is. Blades, virtualization, distributed computing: these are all changing the needs of the market. There will continue to be a high end market. There will continue to be a low end market. But utility computing will not kill servers.

1.21 gigawatts? What was I thinking?

Technorati tags: grid computing, distributed computing, utility computing, google

New SaaS book coming

Scanning Robert's Expert Texture link blog, I see a link to Fred Chong over at MSDN. Fred is a solutions architect at Microsoft, and he's been thinking a lot about SaaS.

In his latest post (SaaS is a journey, walk with us), he gives the table of contents of a book about that he and Gianpaolo are writing about SaaS.

The outline looks great; his summary of key points for architects is perfect:

Scale the application

Enable multi-tenant data

Facilitate customization

At my previous startup, Energy Interactive, we offered a product in the electric industry that we offered as an ASP (that's what we called SaaS back in the twentieth century). We went through everything Fred describes--only we weren't lucky enough to have a book to help us along!

The issues we had with scaling are part of what convinced us that we needed to start Digipede--it just wasn't easy to scale an application. When we looked around our datacenter and saw so many servers that could have been used (and even more when we looked around the rest our enterprise), we wanted an easy way to take an application and scale it out. The tools just weren't out there.

One nit with the table of contents: Fred's chapter on scaling doesn't seem to address distributed or grid computing at all. I'm a little surprised by that. Given the adaptability of grid to service oriented architecture (and I certainly view SaaS as a flavor of SOA) that has been noted lately by experts like Lee Liming and Greg Nawrocki, it seems that Fred and Gianpaolo would mention it. Fred lists the following issues:

Pools: thread, connections etc.

Async

Locks

States

UI/Presentation

How can you scale that application to a cluster? To a data center? How do you plan your hardware so you can handle peak demand without overspending? Whether grid is the answer or not, I'm certain that much SaaS will involve some flavor of distributed computing, and I hope these two go into some detail there.

By the way, it looks like Fred and Gianpaolo are going to cover a lot more than just technical issues--they've got chapters planned on everything from Business Model to Security to Instrumentation and Monitoring. That's fantastic. SaaS is so different in so many ways from traditional enterprise software, and I'm glad to see Microsoft folks lending their considerable wisdom and experience to help people along.

Technorati tags: digipede, grid computing, distributed computing, SaaS, SOA

Case Study: Distributed Computing in Financial Services

We've got a new case study up over at the Digipede site, and it's one of the most exciting customer stories we have.

The customer is a large financial services company (who, being very secretive about the methods of their success, prefers not to be named). Nameless, but big: they manage scores of billions of dollars.

They do a lot of analysis, naturally--minimizing value at risk, satisfying portfolio constraints, and managing transaction costs.

They have a great .NET development team, and they had been developing their own distribution system to farm all of this work out across 20 servers. They made what is a common discovery: they were spending more time writing the code to do the distribution than they were working on their own algorithms. They looked around, and they found the Digipede Network.

Within a week they had a proof-of-concept implementation, and within two weeks of that they were ready for production. That was my favorite part--their lawyers took longer to approve the sale than their programmers did to port their software!

I'm not bringing this up just to brag about a great product. It really speaks to the core of how important a good development framework is. By allowing their .NET and COM developers to take advantage of many machines without rearchitecting their solution, the Digipede Framework simply removed distribution as an obstacle. They are now using the Digipede Network for a variety of applications--everything from C++ to VB to C#, using both COM and .NET.

By bringing in a third party tool, they now have a distribution system that's more scalable, more powerful, and, most importantly, lets them concentrate on their core competency.

This is a scenario I hear about all the time. Developers need to scale an application (maybe it's a web application, maybe a 3-tier application) so they start down a path of multi-threading or .NET remoting (or DCOM or COM+). But soon they realize that the distribution part is hard (especially if they want robustness, good monitoring and reporting, guaranteed quality-of-service, etc.).

Technorati tags: digipede, grid computing, distributed computing,.NET, scalability

Windows Grid: SDForum Talk Wednesday Night

For those of you in the Bay Area: I'm giving a talk at the SDForum Windows SIG in Palo Alto on Wednesday night.

The content will be similar to the talk I gave at Code Camp Seattle, but with a longer slot, I'm going to be able to go into more detail.

I'll give the basics of Object Oriented Programming for Grid, and I'll build some applications from scratch. I'll then grid-enable an existing application or two, showing how you can take an app and "retrofit" it to run on the grid.

The last thing I'll do is a brand new demo (and one that I'm really excited about): I'm going to talk about grid and SOA, and why grid can help scale an SOA well. I'll also demonstrate a Web service application that's been grid enabled, and show how that improves its scalability.

The whole shebang starts Wednesday at 7:00 in Palo Alto; details available here.

Koppel and Nawrocki on Grids and SOA

As I sat on the couch for the second consecutive day fighting a miserable cold, two of my favorite blog postings of the year popped into my inbox (thanks, Attensa).

First was Muli Koppel's blog. Are you reading this yet? He writes what is positively the most intelligent blog I've ever read. He doesn't get the attention that some other bloggers do (because he doesn't write about Web 2.0) but believe it or not: there are really smart people writing things that aren't Web 2.0. Muli is one of them. Every post is well thought out and well written, and I read each several times. He has more content per word than anyone blogging.

Anyway, Muli's post today was called Meant for Each Other: Enterprise Architecture and SOA. He begins with the most practical definition for Enterprise Architecture that I've ever read:

Enterprise Architecture is an infrastructure and a set of Machines constructed in order to manage a chaotic, dynamic, unpredictable, complex, organic, prone to error, frustrating, Enterprise IT, which has to support an ever increasing, dynamic portfolio of products and services, through constant "ASAP, Now, Right-Away" modifications of business processes.

He goes on to talk about the discussions that are happening in the industry about EA and SOA, and he sums it up thusly:

EA will shake off its image as a "documentation, procedures and guidelines" body, repositioning itself as a practical, implementation-oriented discipline aimed at the creation of an Enterprise Management Infrastructure, while SOA will be repositioned, no longer as an Integration/Interoperability architecture, but rather as an Enterprise Management architecture.

One of Muli's gifts (both as he blogs and, I surmise, in his career) is to make sense of the buzzwords of the day in practical terms. Seriously, stop reading here, and go read his post. Then come back.

Of course, now that you've read it, you know my favorite quote from the post. I'll quote it here:

There are, clearly, other infrastructures and machines to add to this picture. For instance, grid/utility computing is an essential part in the real-time Enterprise...

Ok, enough stealing Muli's material. The other post that fell into my inbox today was from Greg Nawrocki's Grid Meter: today's SOA developers .... tomorrow's Grid developers.

Greg wants to know why, while SOA and grid have been hyped for a few years now, SOA is gaining much more traction.

As reasons for SOA's success, he notes that there are a wide range of languages available (I was glad to see him include the .NET languages along with Java), but notes that the real reason is "that you can actually take relatively mainstream enterprise applications and write to a service with a simple API." He's right of course. While as many people (including Muli have pointed out), SOA ≠ Web services, SOA is certainly enabled by web services.

Greg notes that the Globus Tookit 4.0 embraces WSRF and he points to an IBM paper that details building a grid and implementing Globus.

Greg is right on in noting that there is a real convergence between grid and SOA. SOA cries out for a scalable architecture, and a Web service enabled grid is perfect for building that architecture. But as long as an implementation requires a lengthy consulting contract (which, after all, is why IBM is writing papers about it), grid can't really take off.

People need tools that are easier to use and grid systems that are easier to implement. Globus is the pre-eminent standard toolset for building a multi-organizational grid, but its implementation is too difficult for a small to medium sized venture to undertake.

In the meantime, of course, there are vendors who are making the tools to make these types of architectures available today. (We're not the only ones, but we're the only ones doing it on .NET). Anyone who wants a grid right now (not in the future) should be looking to these toolsets to help them build that grid today.

Technorati tags: digipede, grid computing, SOA, globus, wsrf

Tame Your Technology

I was interviewed this week by Neal Miller, host of Taming Technology. Taming Technology is a radio show that tells its listeners "how to survive and succeed in the high tech jungle."

Neal is quite an affable guy, and he and I spent about 45 minutes discussing grid computing (and, of course, I got in a plug or two for the Digipede Network). The topic for the episode was "Grid computing--what is it and how is it being used?"

If you'd like to listen, watch the Taming Technology website--it'll come up soon. Or you can listen to it when it first streams on the VoiceAmerica Business channel--tomorrow at noon, Pacific. (VoiceAmerica is "the industry leader in Internet talk radio").

It was my first long format interview; I think I was wordier than I should have been. I hope to do these more in the future, and I hope to get more concise when I do it. If you listen this weekend, let me know what you think!

Technorati tags: digipede, grid computing

4th Story Gets Faster

If you read Bobsguide (the guide to software and technology in the finance industry), then you've already seen this: Digipede and 4th Story Join Forces to Grid-Enable Financial Software. But if you don't, I'll tell you a bit about it.

4th Story is a firm with deep knowledge in two areas: finance, and software development. Their products (all named after National Parks and Monuments) do incredible things for both buy-side and sell-side firms. They've got some huge clients (Barclays, for one), and they do amazing things with analytics.

They've developed their whole suite in .NET. As they've gotten into very complex algorithms (think genetic algorithms) and large amounts of historical data, they've found the need to take advantage of multiple machines in order to scale properly. Rather than write that portion themselves, they decided to plug into the Digipede Network. Without rearchitecting their solution, and by taking advantage of our OOP-G methodology (Object Oriented Programming over Grid), they were able to quickly modify their software to automatically take advantage of the Digipede Network. Their objects are now being distributed across the Digipede Network and executed in parallel.

The result? They've now got a very scalable solution. Their customers will be able to scale the 4th Story suite as much as they want--just by adding more machines to the grid.

4th Story will be the first ISV logo on the Digipede partner page--but we've got a few more deals in the works, so look for more soon!

Technorati tags: digipede, 4TH Story, OOP-G, grid computing

Catching Up Is Hard to Do

You can tell I'm busy when the only post I can come up with in 3 days is a "check this out" post. Still, this one has some good reads. Check this out.

I just discovered Jeff Schneider's blog Service Oriented Enterprise. This guy knows his stuff. The first post I found (The Sheep that Shit Coleslaw is a great representation of system architectures using Legos as examples. Did I say "Legos?" Yes, I did. If you're a fan of SOA or toys with interlocking pieces, check it out. His points are not dissimilar from what Kim said the other day. But when you say it with Legos it's so much funnier! Jeff, consider me subscribed. And I've got a lot of catching up to do. Jeff's been blogging since September 23, 2001!

The other interesting find I had this week was Nicholas Carr's presentation on utility computing and open source. I hope I can find some time to write a bit more about it. In the meantime, check out his pdf here.

Digipede Webinar Tomorrow

If you're interested in seeing a webinar about the Digipede Network, I'm giving one tomorrow at 10:00am PST.

The presentation isn't given in "marketingese." I'm going to give an overview of how our distributed computing solution works, but then I'm going to dive into real code. I'll use Visual Studio 2005 and VSTO to take an Excel spreadsheet with .NET code running behind it and grid-enable the .NET code to run on the Digipede Network.

If you've been wondering what this Digipede thing is all about, this is your chance. It runs about 30 minutes long. Audio will be available via a call-in phone number.

Send me an e-mail (click the link at right) or fill out the form here.

Technorati tags: digipede, visual studio, vsto

GMail: It's a SaaS World After All

[Update: 2/13/2006 12:02 PST]: I just realized that Google's first implementation is at San José City College, not San Jose State. Updated accordingly.

Over at Expert Texture, Robert had a great post yesterday raising concerns about Google branding GMail (Corporate GMail). According to their post here, Google is going to start offering GMail to corporate customers--with their domain names. They're starting by taking over San José City College's e-mail.

Robert raises a couple of salient issues:

How will GMail isolate company data and ensure that it never gets shared with other customers?

How secure is the GMail database? Leaving aside the issues raised by the EFF yesterday, what are the security procedures in place at Google to ensure that customer data are secure?

Those are great questions. Few pieces of software contain as many corporate secrets as e-mail servers. In this day and age, virtually all communication goes through e-mail servers (much, much more gets done this way than through phones or IM)--and that means that corporate secrets are in that system.

Certainly many small-to-medium companies don't worry so much about intellectual property and therefore won't worry so much about this. After all, if I were operating Dan's Frisbee and Waffle Shop, and corporate GMail would give me the ability to have e-mail addresses like dan@dansfrisbeeandwaffleshop.com without my investing any time, staff, or money in hardware or software, I may think that is terrific. Who cares if someone finds out that I'm introducing a new flavor of syrup next week?

But if I'm a large corporation (and let's face it: by announcing the service with a 10,000+ account implementation, Google is is making it clear that they aren't aiming at the frisbee and waffle shops of the world), I have grave concerns about putting my most secret intellectual property in the hands of any other corporation; even if their motto is "Don't be evil."

But in a larger sense, this same question is being faced all over the software industry as people create and offer Software as a Service. And in many cases, companies are finding ways to assure their clients that their secrets are safe. Look what Salesforce.com has been able to convince companies to do: hand over all of their customer and sales data. For many companies, those secrets are as valuable as the ingredients to their special sauce.

SaaS companies are finding ways to convince customers that their data are safe (from both outside and inside vulnerabilities). But it will continue to be a major issue as SaaS becomes a more frequent business model. And, undoubtedly, just as we hear today about credit card companies accidentally exposing credit card numbers, in the not-to-distant future we'll here of some SaaS provider who inadvertently revealed the ingredients of the special sauce.

So, SaaS developers, I give you this. I've already told you that you have to design your system to be scalable from the outset. Now: make sure you're thinking about security from the ground up, too.

Technorati tags: Google, GMail, SaaS

Web Services + Grid = Crazy Delicious

The other day I happened to glance at the March 2005 issue of Cluster World magazine that was sitting around in the office. One article in particular caught my attention (sorry I can't link to it; ClusterWorld.com's "Old Stuff" link is broken, but it's also available here in PDF format). Lee Liming of Argonne National Lab wrote it; it's called Web Services and the Grid.

Lee's premise (and it's a good one) is that clusters and grids are a natural tool for developers of Web services.

As clusters become increasingly essential elements of the Grid's physical fabric, Web services are becoming essential elements of the Grid's application development toolset. Given the importance of clusters and Web services to Grids, cluster owners and operators need to understand the implications of Web services on the applications that run on their clusters.

He's right. Web services require scalability. One of the inherent truths of creating good web service applications is that you must plan for success: if you've written your software well, people will use it. You have to be prepared for that success and therefore prepared to scale your application. Scaling to a cluster is a perfectly good way to do that. Scaling to a grid is even better. Why? Because it's more cost effective. As you need to grow, adding processing power to your application is as simple as adding more nodes to your grid. Even if the grid is supplementing a dedicated cluster, it gives you the processing power you need at peak times without the expense of dedicated hardware that sits idle 95% of the time.

Note that unless the Web service has been designed to use the back end nodes on the cluster, it will only use the cluster's head node. Web services that require significant processing power should be written to submit tasks to back end nodes via the cluster's scheduler or other tools. The application developer must explicitly implement this capability.

This is an important point that many people miss or think that Network Load Balancing will handle automatically. In fact, NLB is great for serving web pages from multiple servers, but is not a good tool for handling compute-intensive process (for a list of disadvantages of NLB, see Brian Madden's article How to Configure Network Load Balancing).

Back to Lee's point: if you want to take advantage of the power of a cluster or a grid, you need to do that yourself. Of course, I happen think that you should simply get yourself a great toolset that will handle the job submission, guarantee execution, monitor the processes, handle node failure, etc. That frees you up to concentrate on the functionality of your Web service, rather than the technical details of scalability.

Lee sums it up like this:

...it seems likely that service-oriented applications—such as those based on Web services—will lead to significantly greater use of clusters (i.e., more business) than traditional, manually launched applications. Early efforts to gear clusters to become high-power hosting environments for these types of applications will position administrators well in a service-oriented era.

Lee is obviously talking to vendors who offer hosting solutions. But his points are broader and apply to anyone who is designing and building Web services or SOA. If you're writing software that will be available as a service (Web service, SOA, SaaS--you choose your favorite), grid it up and let it fly.

(And if you don't get the reference of the title of this post, you must not have seen the Chronic of Narnia yet, or don't understand the allure of Mr. Pibb and Red Vines).

Technorati tags: grid_computing, distributed_computing, cluster, webservices

New Case Study!

Over on the Digipede site there's a new case study about a Digipede customer: Trekk Cross-Media.

Trekk is a forward-thinking marketing company with high-tech savvy. They came up with an interesting product--for their clients (large retail chains), they will create direct-mail packages that include maps from homeowners' houses directly to the nearest chain store. They wrote the application themselves, and their customers liked it a lot. In fact, too much. The product was a hit in test markets, and the customers wanted to take it nationwide.

But Trekk's application could only produce 750 maps per hour. To handle large jobs would take days. They looked into a difficult and expensive option: converting their application to a multi-threaded architecture, then buying expensive new hardware.

Luckily, before they got too far along that road, they found Digipede. Using a trial copy of the Digipede Network, the Digipede Workbench, and a command-line version of their product, they quickly determined that they could get the scalability they needed. They bought it. Using the Digipede Framework SDK, they modified their application to integrate directly with the Digipede Network; within a few days, they were in production.

The case study has some nifty pictures and some quotes from Jeff Stewart of Trekk. You should check it out.

What I love about this project was how quickly it all happened. Jeff attended a webinar and decided to try it out. Workbench let him distribute his command-line tool immediately. And after deciding to go with the Digipede Network, they were up and running within days--all without a single visit from Digipede. They did everything themselves. I like to think it's testament to the ease of use we bring to Windows distributed computing, but I'll have to give them credit for being savvy software developers as well.

Technorati tags: grid_computing, distributed_computing, .net, digipede, trekk

Slow and steady wins the race

I don't know if I've ever made a New Year's resolution in my life. They've always rung hollow for me--if I need to improve myself, I don't need to wait for the change of year to start doing it. In that sense, they seem more like a way to postpone self-improvement more than initiate self-improvement (i.e., the fellow who says to himself in mid-November "I really need to take off a few pounds; that'll be my New Year's resolution," then proceeds to gorge himself through two months of holidays before dieting for the first few weeks of the New Year).

Well, this year that changed. I finally found a resolution worth doing. Not for the first time in my life, I'm going to copy my older brother.

Last year, my brother Dave had a great resolution: on January 1 he did a pushup and a situp. On the 2nd, he did two pushups and two situps. And so on. By April, he was doing 100 of each per day. By July, 200 per day. And, of course, on December 31st, he did 365 pushups and 365 situps. And during the course of the year, his physique changed drastically. In September, when he turned 40, his wife gave a toast at his birthday dinner and said that he looked better then than he had in their entire relationship.

So I decided to do it this year. It's a fun resolution because it starts so easy (it felt almost silly when I got down on the floor and did one pushup; but I did it, and I was on my way). Even now, 31 days in, it's a very brief "workout," but it's starting to feel like an honest set of pushups (and crunches).

The sheer numbers involved get staggering. Over the course of the year, I'll do 66,795 pushups (you can figure this out in Excel, or you can use the fun method: 1 + 2 + ... + n = (n + 1) * (n / 2)). I know I'll be breaking them into sets eventually, but for now it's fun to see how long I'll be able to do them in one set.

What's making this even more fun is that a bunch of my friends are joining me: John, Robert, and Nathan from Digipede are doing it, and my old friend Marc. It's great having other people doing it, because we can harangue each other to make sure no one is sliding.

I'll do my 500th pushup and situp of the year tomorrow morning; I'll hit 1,000 on Valentine's Day and 10,000 on May 21st. On my birthday in August I'll do my 26,000th situp. And in the month of December alone I'll do over 10,000. I hope I (and all of my friends) can stick with it. I think the inspiration for me is to see what I can accomplish by dedication, tenacity, and an incremental increase in effort. And the next trick will be to apply those principles to other parts of my life (certainly working at a startup for the last two years feels similar!).

Duke Listens! : Weblog

Over at Paul Lamere's blog last week, he had a post about The simplest possible grid computing platform:

In his Ongoing blog Tim Bray mentions that he'll be giving a talk at this year's JavaOne called: Sigrid: The simplest possible grid computing platform. If I make it to JavaOne this year, I'll put this talk at the top of my list. There's a big gap right now in Grid Computing. There are Grid Computing APIs, but these seem to be designed for the traditional big iron apps like those used by chemical companies and Wall Street. It is hard to write grid apps with these APIs. In order to get more programmers and companies to think about moving their apps to the grid, it has to be easier to write grid apps. In particular this means:

Use a friendly programming language like Java

Be able to develop and test on a desktop system

Have a minimal API

That's a great idea, Tim. Of course, not everyone programs in Java, so that solution isn't for everyone.

I love the qualifications, though, because they apply perfectly to the Digipede Network. Friendly language? See my post from yesterday to see a comparison of C# and Java (of course, you can use the Digipede Network in any language that has a .NET or COM interface). Test on a desktop system? I run the whole thing on my laptop routinely. Minimal API? If you haven't seen my webinar where I grid enable an existing app in 20 lines of code, drop me a line and I'll invite you to the next one and show you the coolest API known to grid computing!

As soon as we get permission, we'll publish case studies about companies that have ported their systems to our APIs in less than a day!

Technorati tags: grid_computing, java, .net, digipede

(Nearly) VSLiveBlogging

It looks like Robert is LiveBlogging from VSLive (well, almost live).

I don't know how good his connectivity will be, but if you're interested in what's happening at VS Live SF, follow him at Expert Texture.

Technorati Tags: VSLive

AgentMine.com on C# and Java

Over on AgentMine.com there is a great post comparing C# to Java Paul Graham’s Essay “Java’s Cover”:

In 2001, Paul Graham wrote an essay about Java in which he summed up his opinion of Java thusly:

Java seems like a stinker to me.

Graham then goes on to give a 12 point list of why he thinks Java stinks. I won't go into it point-by-point, because AgentMine does.

AgentMine then does something very interesting: he takes the same 12 points, and applies them to C#. He looks at them to decide if, well, C# seems like a stinker. As it turns out, C# has a much better score than Java.

His final analysis:

The results:
Microsoft’s C# language should only have 25% of the total level of suck that Java has.

It's tongue-in-cheek, but it's very informative. Take a peek.

Technorati tags: java, .net, c#

And the winner is...

I love awards ceremonies as much as the next guy.

Well, to tell you the truth, I don't love awards ceremonies. I habitually avoid the Oscars/Emmies/Grammies/Tonies/YouNameIties broadcasts unless someone really cool is throwing a party. In fact, I think I've been avoiding awards ceremonies ever since my band, Lawsuit, was nominated for a SAMMIE Award (Sacramento Area Music Award) and invited to play at the ceremony, then suffered the indignity of performing immediately before finding out that we didn't win. Apparently the booker liked us more than the voters.

Anyway, here's an award ceremony I am excited for:

The Digipede Network has been named a finalist for a 2006 CODiE Award in the Distributed Computing Solution category!

For those of you not in the know, the CODiEs are awards presented by the Software & Information Industry Association (SIIA). This is their 21st year; they are the longest running, most prestigious independent award in the industry. Getting recognition from their panel of experts is fantastic.

The competition is fierce and widely varied: Everdream, Gigaspaces, Novell, and Solace Systems are all finalists as well.

Every SIIA member company gets a vote; the CODiE Awards Gala will be on May 16th (my mom's birthday!) at the St. Francis Hotel in San Francisco.

I hope that the wonderful, intelligent, good-looking SIIA voters (hey, a little brown nosing never hurt!) like what we're doing!

More .NET SaaS

For a while now, I've been preaching the importance of both .NET and grid computing in the area of SaaS; they're both enabling technologies.

I'm always happy to find validation.

Over on the dotnetSaaS blog, Glen Cameron has a long quote from Romesh Wadhwani of Symphony Technology Group.

Romesh writes a lot about the software industry (some of which is right on, some of which I don't entirely agree with). One part I do like is this:

Enterprises want unified platforms and we're increasingly seeing a move towards the use of grid computing. But the grid is still nascent. There is a lot of infrastructure, underlying operating systems, middleware and visualization tools that are missing or not robust enough today.

There is a significant opportunity to develop the enabling tools and technology that accelerate the delivery of applications and solutions on grid computing platforms.

Absolutely correct. I'd argue that the essential middleware components are increasingly available. Adoption is starting to happen now.

And, on a side note: I hope that Sam Ramji knows that there's a blog out there called dotnetSaaS. Sam promotes SaaS for Microsoft's Emerging Business Team, and he'll be glad to see it being promoted!

Technorati tags: grid computing, saas, .net

Windows grid: cheaper than you think

As I mentioned in my post yesterday (Grids: All HPC? All *nix? Not anymore), I promised to follow up on one of the arguments used against using a Microsoft OS on a compute grid: cost.

In Kim's eBig talk, one of the audience members mentioned why he thought that Microsoft was behind in grid computing, and he gave a concrete example. He talked about the cost involved in setting up a 256 node compute cluster. If you're paying $400 retail for each copy of Windows Server 2003, that adds $100,000 onto the price of your cluster (even $300 copies of XP Pro would run about $75K). As the audience member observed, that's nothing to sneeze at.

When you compare it to $0 for 256 copies of your favorite Linux flavor, the Microsoft solution does seem expensive.

However, on further reflection, it becomes clear that there are costs beyond OS, and that OS cost alone is not a fair comparison.

First of all, the example assumes a purpose-built cluster. This goes against one of the primary reasons for grid computing: taking advantage of the computers that already exist. If you want 256 Linux boxes in a typical organization, you need to go buy them. But if you need the power of 256 Windows machines--you probably already own them! Because Windows is the dominant operating system, your organization probably has thousands of Windows machines that can contribute to your grid.

Even if you need to buy some new hardware, you can have an "extended" cluster featuring a combination of dedicated hardware (in your cluster) and shared hardware (underutilized servers and/or idle desktops). By using existing hardware, you save not only OS costs but hardware costs as well.

Furthermore, the example only counts the operating system cost--not the cost of any other aspects of the system. Some of the most popular *nix based distributed computing solutions cost one to two thousand dollars per node! Sure, you saved $300 by getting a free OS--then you spent more than triple that on your grid solution. That dwarfs the cost of the OS.

And, it doesn't count what is often the largest cost of all--the cost of setting up the grid. Many of the *nix solutions are what we like to call "thinly disguised consulting projects." They are so complicated that setting them up involves hundreds (or sometimes thousands) of hours of consulting time. Some of the big consulting companies have excellent grid computing practices--but I assure you, they're not cheap. Put a small team of consultants on your payroll at $200 an hour for a couple of months and watch how quickly they, too, outpace your OS costs.

So, are the Microsoft OSs too expensive for compute grids? Probably not. Most likely, you can get an extended cluster using some existing hardware (and OS), some new hardware, add the Digipede Network (at under $200 a node), and set it up without an expensive consulting project.

One more X-factor here: as far as I know, Microsoft has not yet announced pricing for its Compute Cluster Solution, which will be out later this year. Who knows? When that price is announced, it may make the OS decision even more economical...

Grids: All HPC? All *nix? Not anymore.

Kim's eBig talk on Thursday night went very well. It was her first public speaking engagement as a Digipede evangelist, and I thought she did great.

Her audience was diverse and obviously very well versed in the concepts (and practices) of distributed computing.

One of the great things that Kim was able to do was to let the audience understand that distributed computing does not apply only to HPC or technical computing anymore. Even the attendees who had experience using grid systems or HPC systems before understood that, while distributed computing was once the hallmark of HPC, its applications today go far beyond technical computing.

One question that came up was why so much distributed computing occurs in the *nix OSs, and not on the Windows platform. A little discussion ensued, and the ideas that came forth were pretty accurate.

First, it was pointed out that much of the research into distributed computing has come from academia. Academics, of course, have always preferred UNIX and the Linux alternatives to Windows (I know that when I graduated from Berkeley in Computer Sciece in 1991, I had never used Windows at school).

Second, the commercial push to grid (as always, I'll alternate between the terms "grid" and "distributed" without going into the distinctions between the two) has come from firms with strong *nix leanings: IBM and Sun. Both have UNIX histories; more recently, of course, IBM has been pushing Linux.

The third reason that the group came up with was the price of the OS. If you are buying 256 boxes for a dedicated cluster, the OS cost becomes a large part of the cost.

There's no doubting the first two reasons. They're indicative of the history of grid computing, but things are changing. The third, though, is a bit of a red herring. Tomorrow I'll get into the reasons why.

Something to do on a Thursday...

If you're in the Bay Area, you should mosey on over to Pleasanton to see Digipede's #1 Evangelista Kim Greenlee as she gives an eBig presentation entitled "Grid Computing for Windows."

Kim's going to give a introduction to and history of grid computing, but then she's going to drill down on Windows. How do the concepts of grid computing trnanslate to a Windows environment? How can you take advantage of Windows machines?

If you're interested, register here.

Four Stars for Digipede Network

As I mentioned previously, Software Development Magazine published a review of the Digipede Network in the January edition. The verdict? Four stars!

Now they've got the article available online. Read the full article here.

On Demand in 2006

eWeek has been doing a series called "Innovations 2006 Analysis," in which it takes in-depth looks at technologies that will have great impacts in 2006. One of the technologies they highlighted was On-Demand Software, or Software as a Service.

The attention paid to SaaS sometimes confuses me a bit--SaaS has certainly been possible for 10 years now (my previous company, Energy Interactive, had a successful product called Energy Profiler Online, which has been sold as a service for nearly a decade).

So why the attention now?

The underlying technologies are now available that make SaaS better able to compete with traditional software offerings. The biggest difference between 1996 and 2006? High bandwidth internet availability. It seems archaic to think about now, but when we began selling EPO to electric utilities, one of our big concerns was the time it would take their users (who were often connected to the net via modem) to download the graphics we used for buttons!

Another change: the quality and security in OSs/web servers/databases/etc. No matter what platform you're talking about, great strides have been made in the security and quality of service available from the underlying tools. Microsoft's initial forays into these areas (early versions of IIS, NT, and SQL Server) did not provide the quality necessary for vendors to guarantee quality of service; the open-source community had some decent tools but were nascent at the time. In the ensuing decade, both camps have made significant improvements--Microsoft's tools have improved dramatically, and the open-source community has continued to improve Apache, MySQL, Linux, etc.

But I'll add one more innovation that has made SaaS a viable alternative to traditional software delivery: distributed computing. Key technologies have emerged in the last couple of years that make it easier to scale software across many machines; as noted in my previous post here, scalability is one of the keys to providing software online.

Back in 1997 when we were writing EPO, scalability was one of our biggest concerns--there was no easy way to take the compute intensive operations inherent in the software and distribute them across many servers. We had to write that code ourselves. It was laborious work, and it wasn't really part of our core competency at the time.

Nowadays, products like the Digipede Network make distributed execution easy. Instead of spending months hardcoding solutions to allow their software to scale, SaaS developers can write a few lines of code that add inherent scalability to their product. That, in turn, enables people starting SaaS companies to spend their programmer dollars adding features to their software. They end up with a product that not only has a better feature set (because they spent more time and money on it), but is also more scalable and robust (because they aren't using a jury-rigged solution for distribution).

2006 may be the year of SaaS--but if that's true, than it's the year for distributed computing as well.

Maximizing computation, minimizing data

I had a Digipede customer ask me last week about how to optimize his processes for running on the Digipede Network. He is grid-enabling an existing application--he's got a class that does a lot of calculation, and he wants to have a bunch of those objects calculating in parallel. Each object is already independent.

Sounds perfect for distributed execution, right? "Twenty lines of code" away from being Digipede-enabled?

Well, not quite.

See, the objects that he does his calculation on are pretty darn big--on the order of 20-25 megs each. He actually had no idea they were so big; but the class has lots of members, and many of the members are large classes themselves. Now, the Digipede Network can certainly move objects of that size--but those objects have to be moved in order to calculate on them, and moving data is often more time consuming than the computation you need to do on it. (See Jim Gray's Distributed Computing Economics to get an idea of what that means, but bear in mind that we are only discussing LAN-wide computation here).

The answer can frequently be to create a class that contains only the data relevant to the distributed calculation. In the case with this customer, he was "retrofitting" the application to work on the Digipede Network. His class wasn't designed for distribution and, as a result, had a lot of data in it that wasn't necessary for the calculation that he wanted to happen remotely. In other words, his objects did a lot in their lifetime, only a portion of which was going to be distributed.

The customer needed to create a class that contained only the data that was relevant to his distributed process, and use that as a member in his huge class. Only this new, smaller class gets distributed across the network. Instead of moving 20MB per object, he was now moving only a few kilobytes. When the small class returns from its journey across the network, its result data is then copied back into the main object.

Our customer needed to do a bit more work than the fabled "twenty lines of code"--but he ended up with a more structured application and vastly improved performance.

Technorati tags: digipede, distributed computing

Good Reads

[Update: looking for my discussion of Microsoft's tools? It's down here]

Here are a few links for your perusal...

Don Dodge, always an interesting read, echoes Dan'l Lewin's list of hot startups using Microsoft tools here:

Digipede—Its "many legs make light work" and turn any combination of servers and desktops into a grid for .NET apps.

Over at the Science Library Pad, they point out that SearchCIO has published Gartner's 2006 Top 10 Strategic Technologies.

Grid computing. Who's doing grid computing? Charles Schwab, Royal Dutch Shell and Sony are among the companies tapping the technology. Definitions of grid computing vary, but its popularity continues to rise.

And, my favorite read of the day, Software Development Magazine has just published a review of our software, the Digipede Network. Paid registration required for the article, but suffice it to say: Four stars! Thanks guys!

It's a great review, and not just because he liked our product. As it turns out, the author (Rick Wayne) isn't just a tech writer--he also does soil analysis. He knows his way around compute intensive simulations. So he converted his office into a mini-grid, using nothing but the Digipede Network software (and included documentation, of course)--with not a drop of help from us. He got it working in a snap, and he was one happy camper.

Technorati tags: grid computing, digipede, .net

MS Tools Too Expensive? Think Again.

This started as a thread in some comments over on Scoble's blog, but it got to be too long for a comment so I moved it over here.

An anonymous commentor wrote:

There are startups using .NET, but they aren’t the majority, and those who chose to do so are buying themselves into a trap with expensive licenses and a locked-in platform.

Jeremy Wright from b5Media, who has a great blog called Ensight, contradicts him:

The startup can grab ISV packs which’ll cost about 2500$ to get the company up and running with all the dev tools and server bits they’ll need. Toss in another 2500$ and they’ll get all the MSDN stuff they need. 5000$ is not that much to get a 5-10 man shop up and running, even when bootstrapping.

Jeremy has a great point, but he's off by an order of magnitude!

I work at a startup. We joined Microsoft's Empower program, which exists to help startups with initial costs. It cost us $375. It included a universal MSDN subscription with 5 user licenses, as well as 5 licenses for Office, and "the full array of server products including Windows Server 2003, Exchange 2003 Server and SQL Server."

Um, does $375 seem like an exorbitant amount for that?

Also included: technical support and training. Individuals at Microsoft assigned to help us with technical details, our marketing, and even our sales.

As soon as we could, we became Partners, then Certified Partners, then Gold Certified Partners. The benefits are enormous: Microsoft helps with marketing, they help with architectural issues, we get early releases of software, we get direct access to the product teams and their roadmaps. Oh, and we get GREAT license benefits.

If you haven't worked with Microsoft, then you just don't understand this: they work very, very hard to create an ecosystem that actually fosters innovation. They want startups like mine to choose their tools, so they do a tremendous amount of work to make it a good choice. And with programs like Empower, the cost of all those tools, operating systems, and support, is virtually nothing.

Windows may not be your OS of choice, and if your users are all running Linux boxes, obviously these tools and programs aren't for you.

But if you think it's too expensive for a startup to use Microsoft products, you just haven't done the research.

From our perspective: it's a slam dunk.

Technorati tags: microsoft, devtools, msdn

Not done Christmas shopping yet?

Dan'l Lewin, who runs Microsoft's Emerging Business Team, has picked out his list of "coolest companies under the tree."

Guess who made the cut?

Thanks, Dan'l. I predict there will be no coal in your stocking this year!

Of Course Scaling Matters!

The controversy over at Jeremy Wright’s Blog, which started with this post, hasn't slowed down much. He's gotten a ton of comments, flames, and posts. He defends himself here:

Which is why designing for scale is so important. I don’t believe any startup “needs” to achieve anything more than around 2 9s of uptime, which is what a properly configured server should do for you. However, even at the beginning, you need to be coding and planning for growth.
Small things like managing how transactions occur, having separate database connections for reading and writing, making your app able to handle variable state sessions, etc are key.

(emphasis mine)

One of his posters, Ian Holsman, responds:

The reasoning behind not worrying about scaling is that in a lot of cases people worry about the wrong things. They will spend hours getting that code tuned just so, and have it running in 10ms less time, only not to realize that the code is only run once a day.

Scott Sanders from Feedlounge also responded:

The FeedLounge development process was more along the lines of:
1. Build a webapp, see if the features are compelling to a set of users, keeping a design in mind that is capable of scaling
2. Overrun the shared server that you are using, switch to dedicated server, so you can properly measure the effects of the application.
3. Add more users, adding requested features from the users, measuring the load in a fixed, known environment, and start work on “Distributed” part of ladder. The is where the build portion of the scalability starts.
4. Now that you believe you have something that has value, invest in the hardware and software development necessary to scale. Continue working on priority based tasks towards release of your product.

(again, emphasis mine)

Scott's point is valid, and the italicized portion makes it valid: you need to be designing your software so it's capable of scaling. No, Ian, this doesn't mean spending weeks optimizing the code to trim every last microsecond off of every transaction. It means designing your software well from the beginning.

Most importantly, it means acknowledging the possibility, however remote, that you may actually succeed and build something that people eventually use. Many people.

This point applies equally to those designing web sites and those planning on deploying SaaS. If you are going to make it available on the web, and you're not designing for scalability, then you just aren't planning for success: you're planning for failure.

Ian does make one point validly: in the beginning, you can't spend too much time on scalability. You need to make sure you get the <content, features, service, whatever> right. But you need to be prepared for scaling; that's why it's important to choose a toolset in the beginning that makes scaling later as easy as possible.

Plan on succeeding.

As a product manager, I'd be remiss if I didn't point out that that is exactly what we designed our SDK for--so developers can spend their time in their area of expertise, but have a framework underpinning their software that will scale when the time comes.

Technorati tags: scalability, web2, web2.0, web 2.0, SaaS

How to Buy a Grid

I was starting to prepare a post on how to buy a grid--what are the steps you can take, and what's involved.

Then I remembered this post by my colleague Kim. It's a great place to start: 10 easy steps, and guaranteed success (well, they may not all be easy, and no one can guarantee success). But it's a very good list of things to consider when you are educating yourself about a purchase.

I'd add one more thing to watch out for: don't be swayed by features that you don't need (and won't be able to take advantage of). If someone tells you that their software is the most powerful on the planet because it works on 12 different operating systems and can tie together PCs in Idaho with mainframes in Kirkutsk, but you have 200 PCs in one office building, how does that help you? It doesn't. So look for something that will help you the way you work.

So read Kim's post. It's a good one. Plus, she used it to invent the word gridified, which is my new favorite word.

Hey, Blogger! Spell check much?

I tend not to use spell-checkers; I try to read my writing carefully (although anyone who has been paying attention knows that I certainly don't always do that successfully).

Today, for the first time, I used Blogger's built-in spell checker.

It choked on a word, that, well, I thought it would know...



Technorati tags: blogger

Web 2.0 Companies NEED To Scale

Sometimes I read a blog post and it just makes me smile.

Today Jeremy Wright talks about the need for scaleability for Web 2.0 companies (he's specifically talking about an announcement from FeedLounge). He says:

Listen up. If your company relies on the web to stay alive, you’d damn well better be using at least some of the following “ladder to high availability”:

Backups, Redundant, Failover, Cluster, Distributed, Grid and finally Mesh

Each step up is a massive increase in cost, but it’s also a massive increase in uptime and such. I hate it when companies say they want 99.9% uptime (or even worse 5 9s of uptime) without thinking about what that’ll cost them.

Distributed/Grid computing should be the kind of thing that these companies are thinking about from the time they begin planning their architecture. They have to plan for success. They have to plan on hundreds of thousands or millions of users, right?

And I'll go a step further than Jeremy--the other thing Web2.0 companies shouldn't do is write that portion of their applications from scratch. I mean, no one in their right mind writes their own database to sit under web apps, right? You go get one--SQL Server, MySQL, PostgreSQL , whatever is right for you--but it would be a complete waste of your time to sit down and write one from scratch.

So why do people try to do that with a distributed/grid system? Most do. It's too bad; it's a waste of their valuable time and money. And in all likelihood, they'll end up with a solution that isn't nearly as scalable as they think is.

I'll let Jeremy finish this one up for me:

If your business depends on your website being up, look at your code, look at your infrastructure and for your users sake figure out what you actually need and build the damn thing properly!

Technorati tags: Web20, Web 2.0, Web2

VS 2005: VSTO Improvements

A couple of weeks ago, I wrote a post about my difficulties in upgrading a VSTO project from VSTO 2003 to VSTO 2005 (for those of you unfamiliar with VSTO, it stands for Visual Studio Tools for Office). It's replacing VBA as the method to put code behind spreadsheets, Word documents, etc.)

I had some difficulties upgrading a VSTO project from Visual Studio 2003 to Visual Studio 2005; they've changed some of the architecture, and it was a pain (read that post if you want the gory details).

However, my post was remiss--I neglected to mention any of the good changes in VSTO.

First and foremost: Integrated development environment. In the past, if I wanted to put buttons on my Excel spreadsheet, I had to put Excel into "Design mode", create buttons using the Control Toolbox, then switch over to Visual Studio to wire those to methods in my C# code. It was pretty cumbersome, and dealing with modality in Excel was a pain ("No, I meant to select the button, not click it!"). Now, all of the design work is done directly in Visual Studio 2005--I open my Excel spreadsheet there, and I work on the UI using the regular toolbox in Visual Studio. It's a much more cohesive experience.

Secondly: now the controls look better! The buttons that appear on the Excel spreadsheet just plain look better than they used to.

Third: this is the part I don't quite understand. For some reason, the .NET code in the spreadsheet seems to load much faster. I can't tell if this is my imagination or not--but I've done this a bunch, and I don't think it is. It seemed in the past that when I made my first call into .NET code behind my spreadsheet, I had a several-second delay (disclaimer: I have the slowest laptop on the planet). For whatever reason, that seems to have disappeared. Now, when I make my first call, it happens immediately. This must be a .NET 2.0 change (because I'm still running the same version of Excel 2003), but it's certainly welcome.

The change to the object model took some getting used to, and it was annoying to have to rewrite some of my code. But I'm coming around to it.

Last, the methodology of associating the binary with the spreadsheet has been improved. In the past, there were two custom file properties in the spreadsheet--one which gave the assembly location, and one which gave the assembly name. The location defaulted to NameOfSpreadsheet_bin, and the name of the DLL was NameOfSpreadsheet.dll. It always looked pretty unwieldy--you'd have your NameOfSpreadsheet.xls, and next to it a NameOfSpreadsheet_bin folder with a NameOfSpreadsheet.dll in it. And if you ever moved anything, it was a pain to tell .NET 1.1 how to give permission to the new DLL.

Now, the custom property of the spreadsheet has the GUID of the DLL in it, and .NET 2.0 gives permission to that GUID. This means that you no longer need to have a *_bin folder, and you have a much easier time if you need to move/deploy your spreadsheet.

So the upgrade process is a pain. But once you get in there, VSTO for Visual Studio 2005 is definitely better to work with that VSTO for Visual Studio 2003. I haven't built anything extravagant yet (unless you consider a grid-enabled, supercomputing spreadsheet extravagant--come to my webinar in a half an hour to hear more about that), but I think the product is much improved.

[Updated 13:17 adding Technorati tags] vsto visual studio excel

Want to learn what I'm talking about?

I'm giving a Developer webinar tomorrow at 10:00 AM, Pacific time.

I'll talk a bit about the Digipede Network, I'll use Visual Studio 2005 to grid-enable a .NET application, and show that application running faster by running on a cluster of Windows boxes.

If you haven't seen a demo of the Digipede Framework SDK yet, you'll be amazed at how little I have to modify an existing application in order to make it run on the Digipede Network. Click here to register and join in on the fun.

Windows Cluster/HPC links

One of the questions I hear most frequently about Windows clusters is "Windows clusters?"

The answer: "Yes!"

People do use Windows for clusters and HPC. Here are some links to valuable resources for anyone considering a Windows Cluster.

Microsoft's Windows Compute Cluster Server home page

Microsoft's HPC Discussion Group

Windows Cluster Resource Centre

Windows HPC site

And, of course, there is Microsoft's HPC Partners Page, featuring this little company!

Nothing to see here

No Need to Click Here - I'm just claiming my feed at Feedster

Outbound Compliance: Getting More Important (and Better!)

The InfoWorld TechWatch blog keeps track of, well, all things tech. It's a good all-around gatherer of information. Today there's a post on OCC: Outbound Content Compliance.

Corporations are increasingly needing to monitor their employees' communications to ensure that they are compliant with any regulatory issues as well as any internal guidelines.

One great product that handles this is OutBoxer by Audiotrieve. How do I know these guys? Well, I first met them at the Demo conference in Phoenix last January. They were demoing Outboxer, and we were demoing the Digipede Network.

As it turns out, they "train" their product by running millions of messages through it, using Bayesian analysis to make their algorithms more accurate. It's the kind of thing that scales linearly. We struck up a conversation with their CTO Sean True at the conference, and they were one of our early beta customers. They became a commercial customer as soon as we released. Their analysis runs went from overnight to under an hour. Faster analysis has two huge benefits for them: more accuracy and better use of their employees' time.

It was a great proof-of-concept sale for the Digipede Network. We never visited their offices; they were able to download and install the software by themselves. After seeing it work, they bought new, dedicated hardware to scale their solution even more. They were the first users of our COM API (which they were calling from Python!), and Sean got into .NET programming and even ported one of our samples to IronPython for the Digipede Community Boards.

If you're interested more in the implementation, we've got a case study here.

Saving the screen, wasting everything else...

If you're not reading Nicholas Carr's Rough Type blog, you're missing one of the best writers in the blogosphere.

His post today is called "Kill All Screensavers," and he tells an interesting story about screensavers in corporate America--he's talked to at least two CIOs who have been prevented from successfully implementing grids on their PCs because of the official corporate screensaver.

He said that while grids were theoretically attractive as a cheap means of harnessing lots of processing power, he faced a big roadblock: his company's official screensaver. It turns out that the corporate communications department created an elaborate screensaver, complete with video clips featuring the CEO, to promulgate a “corporate values” program. Installed on all the company’s PCs, the screensaver sucks up the processing cycles that might otherwise be put to a productive use – like finding a cure for cancer.

When we were first designing the Digipede Network, we looked at a lot at what had been done by other distributed computing networks. Many of them implemented a fancy screen saver (SETI@Home is a good example, but there are many others).

Without thinking twice, we decided not to implement one. Why? Because it's an enormous waste of resources! (See Nicholas's post for some staggering statistics on the power wasted alone). It's bad enough for a pharma company to willingly waste power and CPU hours; it's ludicrous for a grid company to do it. They're wasting the very commodity they're supposed to be saving!

Needless to say: don't look for a Digipede screensaver when you implement the Digipede Network.

Technorati Tags: .NET, green, grid, net, screensavers

(Social) and (Distributed) Networking

I've spent a lot of time over the last couple of months thinking about software as a service and service oriented architecture, especially how they relate to distributed computing.

Today my colleague Kim sent me a link to Threeminds blog on Digital Marketing. In this post, 3minds makes an interesting point that I hadn't considered very much: distributed collaborative development (SourceLabs, SWiK, Sourceforge, et. al.) enable "extreme acceleration" in collaborative development by distributing the development, accomplishing more work by developing in parallel. Distributed computing offers extreme acceleration in the work that can be done by that software, accomplishing more work by computing in parallel.

The networks (social and computing) are becoming more powerful. The tools available on both sides are becoming more sophisticated every day.

Interview with John Powers

While I work on a post about Supercomputing 2005, feel free to chew on this interview with Digipede CEO John Powers on WindowsHPC.org.
http://www.windowshpc.org/stories.php?story=05/11/16/1667730

(Not) Blogging from SC05

I had intended to blog directly from Supercomputing 2005. However, our days ended up being so full, I couldn't find a free 15 minutes to write. So I'm back in Oakland and ready to pour the thoughts out of my head. We had a pretty huge week here at Digipede, so it's been tough to find time to blog.

This was my first trip to Supercomputing. There were some things that struck me as odd, some things that wowed me, and other things that just struck me.

I was amazed at the amount of computing power there. John overheard someone say that they basically loaded as much computing power into the building as the power grid could handle. There were huge racks of servers. There were supercomputers. The vendors brought computers. The national labs brought computers. It was an amazing amount of compute (and networking) power. They put up some pretty good information about the network they built for the show here.

One thing that surprised me about the show was the incredibly large presence of the not-for-profit agencies that do supercomputing. The national laboratories (Argonne, Brookhaven, Idaho, Lawrence-Berkeley, Los Alamos et. al.) were there in force. They had large booths (20x20 or 30x30, with many displays, multi-story structures, etc.). I was surprised to see them because I think of conferences like this (and exhibits, in particular) as a way to attract customers. Clearly these institutions use this conference as a way to keep the industry abreast of what they've done in the last year. It was all very informative, and made the exhibit hall a little a little more tolerable because not everyone there was trying to sell their wares.

One thing that made this show very different than Supercomputings past was Microsoft's large presence. They probably had the most floor space of any exhibitor: one large booth for themselves, and another devoted to Microsoft partners. In addition, Bill Gates gave one of the keynote addresses.

From speaking with other attendees, I gathered that Bill's speech went better than expected. I think a lot of people were expecting him to give a "Rah! Rah! Microsoft!" speech; he didn't. Instead, he talked about the benefits of supercomputing, and the dawning of the "personal supercomputing" era (with many CPUs on each desktop). Kyril Faenov did a demo of Microsoft Compute Cluster Solution in action. I think the things that impressed people the most is that they showed CCS working on a real world problem and interoperating with a Linux cluster.

As with most conferences, the best part for us was meeting with the people who attended. Going to a conference like this saves you 15 business trips because so many people are in the same place at the same time. The place was just full of partners, potential partners, and potential customers. We had meetings nearly all day both days and got some terrific leads out of it.

See you at Supercomputing 2005?

I'm heading up to Supercomputing 2005 for Tuesday and Wednesday.

Anyone going to be up there? Shoot me an e-mail, or comment below.

The MS-Trifecta

One of the coolest things about being at the Global Launch of Visual Studio 2005 and SQL Server 2005 was being able to demo a product that utilizes both: our SDK plugs directly into VS 2005, and our server runs on top of SQL Server 2005.

Not only that, but we're also running on the "not really announced loudly but it's out there now, too" product: .NET 2.0. (As an aside--I wonder why they didn't make a bigger deal about this? I guess developers know about it, and non-developers don't understand or care)

So how many companies can say they have a single product that plugs in to VS 2005, runs on top of SQL Server 2005, and takes advantage of .NET 2.0?

I'm giving a webinar on Thursday next week to show it all off. Sign up here if you're interested.

VSTO: A Difficult upgrade to VS 2005

WARNING! This post will only be interesting to anyone upgrading Visual Studio Tools for Office projects from VSTO 2003 to VSTO 2005. All fans of distributed computing, you may want to skip this one!

Last week I wrote an entry about installing some minor difficulties installing VSTO 2005--it was not as painless and trivial as it should have been.

After getting things installed, I immediately tried to upgrade a VSTO project that I had written for Excel. I needed to get it done quickly, because we needed to demonstrate it at the VS 2005 Launch. The upgrade of the VSTO 2003 project did not go as easily as I had hoped.

It seems Microsoft has made some pretty significant changes to both the mechanics and the object model of the VSTO projects. Most importantly, they've changed things from a workbook-centric model to a worksheet-centric model. This means that, in practical terms, you may need to do some re-architecture of your model before it will work.

I opened up my old project in VS 2005, and it offered to upgrade (making a backup, of course). After a few changes I was able to get my project to compile; however, when I tried to run it I got the dreaded "Office document customization is not available" error. Of course, I had no idea why I was getting it; my first assumption was that my project is dependent on two different libraries, and that one of them didn't have permission to load (this was a difficulty in VSTO 2003).

So I went off to my Administrative Tools->Microsoft .NET 2.0 Configuration tool. When using VSTO 2003, I generally used the .NET 1.1 Wizard to give full trust to any libraries loaded by my assembly. However, .NET 2.0 doesn't have a wizard. Instead, I had to use the Configuration tool. The VSTO projects in the .NET 2.0 Configuration Tool are set up differently than they were in .NET 1.1. Formerly, I'd see either a Wizard entry under the Machine->Code Groups->All Code entries, or I'd see an entry under User->Code Groups->All_Code->Office_Projects that was based on the directory where I built my project. In the .NET 2.0 Configuration Tool, projects are listed under User->Code Groups->All_Code->VSTOProjects, and each is listed by its GUID rather than its folder.


Because I had had such difficulties with security when using VSTO 2003, I assumed that my problem here was the same. I was wrong. When I tried tweaking the settings in the .NET 2.0 Configuration Tool, I started getting a new error that made it clear that now I didn't have permission to load that assembly.

Eventually, I gave up and tried recreating the project. Once again, I upgraded my VSTO 2003 project. At Rob's suggestion, I followed the directions in the "How to Upgrade Solutions from Visual Studio Tools for Office" document (I was wondering why the upgrade hadn't told me to read that first, and also why that document is so hard to find when searching in MSDN). I copied and pasted code as directed; some of my code used to get called when the workbook opened, so I put it in the ThisWorkbook_Startup method. Again, it wouldn't open.

At this point I was pretty frustrated, having spent the better part of a day trying to get this working.

Finally, I "started from scratch." Rather than upgrading my old project, I created a new project. When the wizard asked if I wanted a new workbook or a copy of an existing workbook, I pointed to my old workbook. It created a brand new project, with files for each of my sheets (that's another change between 2003 and 2005--there are .cs files for each sheet of your workbook).

I then added references to the DLLs that I needed my project to load; I built and ran--it loaded! Now I just needed to add my code. I copied and pasted most of the code from my old project into Sheet1.cs (this time, I didn't put any code in ThisWorkbook.cs).

Almost everything was ready. The only problem I had was that I had no way to get values from multiple sheets--in the VSTO 2003 paradigm, your workbook had access to all of the sheets; the new object model seems to make that harder. I used to use code snippets like
mwksResultsWorksheet = (Excel.Worksheet)ThisWorkbook.Worksheets["Results"];, then I'd use that worksheet to get values. Now I didn't have access to the other sheets, because I was working within a sheet rather than within a workbook.

Eventually I came upon the Globals class. This class gave me access to all of the sheets, using construction like this: Excel.Range cellBetas = Globals.Sheet3.get_Range("cellBeta1", "cellBeta30");

The most inconvenient thing about it is that I have to refer to the sheets by sheet number rather than by their name.

Anyway, at long last, I had my VSTO sample working. It's awesome. Using distributed code behind my Excel spreadsheet, I can have 10 machines sharing in the 700,000,000 calculations necessary for my retirement simulator. All in all, I get 3 minutes of work done in about 20 seconds.

Other than the difficult upgrade process, I actually like some of the new aspects of VSTO 2005. But I'll have more on that later...

VS2005/SQL2005/BT2006 Launch: No Cheap Trick!

I may give away my age if I say that one of my favorite parts of the Microsoft Visual Studio/SQL Server/Biztalk Launch was that Cheap Trick played both the keynote session and the party. After 30 years, I have to say, they sound great. I've seen over-the-hill bands before, and this is not one of them: Cheap Trick still rocks. (Rob has some pictures of them here).

But there was so much else going on, it's hard to pick a favorite part. Steve Ballmer gave his customary high energy keynote (available here). There were tons of Microsoft partners there, and it was great to catch up with people from Softagon, Allin, and many others. All of the major hardware vendors were there.

The development communities were there in force, too. INETA, the International .NET Association, was there, as was its local affiliate Bay.NET. Oliver and Bennett from Bay.NET put together tons of great informational programs; I think that both Kim and Rob will be presenting for them soon.

But, as always, the best thing about a Microsoft event is the chance to meet with and talk to the Microsoftians themselves. No, Steve Ballmer didn't stop by the Digipede kiosk. But Kari Martin and the ISV Connect team was there (is it true, Kari, that you put the "kari" in "karaoke?"). Sam Ramji stopped by for a few minutes--we are very excited about the work he's doing; we think that Microsoft is building a great platform for SaaS, and we think we have an important piece of the puzzle for SaaS providers. Dan'l Lewin spent some time with our CEO, John Powers. And both Matt Pease and Diana Beckman from the partner program both came by at some point. Many Microsoftians made it a point to come by, say hi, and ask what they could do to help.

And at the party, I let Jason Mauer school me in air hockey ;-). I should have known better than to play against anyone who uses the word "pimpiest" in a blog entry.

So what did I take away from the show? Well, I already knew all about the great new technologies, and how fast SQL Server 2005 is, and how powerful Visual Studio 2005 is. But it's not what you know, it's who you know. Having spent a day where we got to talk to the folks in charge of Microsoft's partner programs, Emerging Business Team, and the ISV Connect team--oh, and got to show off our latest release to 3000 potential customers--was a darn good day.

MSDN Blogger Community

Just found out I've been accepted to the "MSDN Architects Bloggers" community--because I signed up to be part of the Visual Studio/SQL Server Launch blogging community.

I was part of the PDC Blogger community, and I enjoyed reading other people's posts. It was impossible to go to every session and meet every interesting person there, so having the ability to experience the PDC through other attendees' eyes was really useful. I hope this leads to the same thing.

For those of you who are reading that feed: welcome! And if you're interested in grid/distributed computing on the Microsoft platform, add this feed to your aggregator.

And if you're interested in hearing about my experience upgrading a VSTO project to VS2005, stick around: more on that adventure later today. The good news is that we will have our demo ready for the VS launch on Monday!

Programs are great; people are better

At Digipede, we embraced the Microsoft partner program from the day we decided on building a product on their platform. We immediately enrolled in the Empower Program, which was great. It saved us thousands of dollars in license fees.

As soon as we qualified, we became a Certified Partner and then a Gold Certified Partner. Really, the programs are invaluable. Access to lots of software. Access to great people within Microsoft. And, best of all, access to lots and lots of other ISVs and Microsoft partners. As I've blogged about before, they put lots and lots of effort into their partner programs.

But you know what the most useful thing we've found is? It's not the programs at Microsoft--it's the people at Microsoft. There are certain people there who absolutely bend over backwards to help partners. In particular, we've had amazing help from Suzanne Lavine here in the Bay Area, and Kate Bothell up in Redmond. They both do a wonderful job helping partners. They both take the time to understand what individual partner's needs are, and take time to try to see how they can help. They're willing to step outside of their defined roles and just lend a helping hand. I get lots and lots of e-mails from the different partner programs that I'm in, but 100 of those aren't worth as much as an e-mail from Kate telling me about a program we should be in or an introduction from Suzanne to a vendor she knows we could partner with.

Of course, not everyone at Microsoft is this outward oriented--I've met plenty of people who would barely give me the time of day (or maybe just terse e-mail now and again). Microsoft tries to emphasize the importance of its partners to all employees (walk the halls in Redmond and you'll see posters reminding employees that 96% of Microsoft's revenue comes through partners), but clearly some people grok that better than others.

For us, finding those people who truly embrace partners has been critical. So: thanks Suzanne and Kate!

VSTO Changes / VS Release

I'm preparing my machine to be our demo machine at the Visual Studio launch event on Monday.

One of the very effective demos that we've done in the past involves Digipede-enabling some .NET code behind a spreadsheet in Excel. It drastically reduces the time it takes to run, and only takes about 20 lines of code to do it. Really, really cool stuff.

So I installed VS 2005 on my machine, only to learn that VSTO isn't part of it. So I downloaded the 400MB VSTO install and installed it. It's a little strange, because it seems to be its own installation (not just an "add on" to Visual Studio).

However, now when I try to open an office project, I get a message telling me that I need to install Microsoft Office Professional Edition 2003 SP 1. This is a problem, because I have Office Professional Edition 2003 installed already! I have SP 2, though, not SP 1. In any case, I can't actually create or open an Office project.

Has anyone else seen this problem?

Just a quick one...

There are millions of people out there writing blogs; the variety in terms of both content and quality is staggering. Many people seem to be just providing links to other blogs in many (if not most) of their entries; now that great aggregation services exist, those blogs are mostly a waste of space.

There's one blog I read that stands out in its insight and incisiveness: Nicholas Carr's Rough Type. He writes effectively, and each and every post has good analysis and content. I bring him up today because he has an interesting take on Microsoft's announcements from yesterday about Live Software and live.com.

So add Nicholas to your subscription list.

Oh, look. I just became one of those people who writes a post that just links to someone else's post! More content later; I promise.