MSDN Webcast: Excel Services and Grid Computing

B

  ack in April I did two MSDN webcasts: Object-Oriented Programming for Grid: Grid Computing for .NET and Advanced .NET Programming for Grid Computing (both are available on-demand at those links).

I had a blast doing them and had more than 100 attendees at each. Recently, they asked me if I could do anything regarding the upcoming Office 2007 release. Given the very cool work we've been doing with Microsoft's Financial Services Group with Excel Services and Compute Cluster Server, I jumped at the chance!

I'll spend an hour giving a general introduction to Excel Services, talking a bit about grid computing, then bringing the two topics together and explaining why the two technologies complement each other so well.

Details:
Using Grid Computing to Scale Excel Services (Level 200)
Wednesday, October 4th, 10:00AM PDT

Register here.

Technorati tags: grid computing, digipede, msdn, excel services, excel2007

Scoble Is Dead Wrong about DEMO

Y

  esterday Scoble had a post about DEMO (in which he was responding to a post over on Webreakstuff).

The issue? The DEMO conference charges money to demonstrate. The question? Does that mean that DEMO isn't relevant, because only funded companies can go?

It started with Fred (of WeBreakStuff) saying:

I’d much rather go to an event where companies were invited for their merits instead of the money they can cash out in order to tell me how great they are.

Scoble disagreed with him: he thinks the fact that companies are charged to demonstrate acts as a filter (and he needs filters because he sees so many pitches).

They're both dead wrong.

First, Fred: Companies aren't invited because they have money. To participate in DEMO you undergo a rigorous application process and a very critical evaluation. When my company demonstrated in 2005, there were over 700 applications--and around 70 were accepted. That's close to an Ivy League acceptance rate! Chris Shipley and the team have a great eye for technology, and you have to pass their sniff test (and outshine the competition) to get in. Their track record is fantastic (products like Palm, Java, and TiVo were all featured at DEMO in the past--prior to their great success).

Second, Scoble (and this is really just a corollary to the previous point): money would be a crappy filter, anyway! There are tons of companies with bad ideas, bad products, and buckets full of cash. The fact that they could scrape together the $15K (or $25K or whatever it is) to go to the conference means very little.

The fact that a tech luminary like Chris thinks that these products may have the ability to fundamentally change their markets means a lot.

And that is why DEMO is a good filter.

Technorati tags: demo2006

Bay.NET Concurrency Talk on Wednesday Thursday

F

  or those of you Bay Area developers who keep hearing the terms "concurrency" or "concurrent development" and aren't sure exactly how they may apply to you, here's the talk for you.

This week's Bay.NET talk is "Concurrent Software Development - The Next Big Thing." My colleague Kim Greenlee will take a look at concurrency from the ground up: from multi-threaded applications running on a dual-core box to massively parallel grid applications.

For finance developers in the Bay Area, or anyone else who needs concurrency in their applications, tomorrow night's session is a must-see.

When: Thursday, 9/28/2006 at 6:30 PM
Where: San Francisco, 1 Market Street, Landmark Building, 2nd Floor

Click here to register

Update: Fixed day in the title. Really, Kim's talk is Thursday.

Technorati tags: grid computing, concurrency

Grid and SaaS: Callidus and Sun Say Yes

I

  came to SaaScon hoping to find an intersection between grid computing and software as a service. In this morning's session from Sun and Callidus, I found the nexus.

Callidus provides compensation tools for very large (> $1B annual sales) corporations; their tools use models to figure out compensation for individuals, to fine-tune incentive programs, to improve the channels, etc. For the last several years, they've offered both on-premise or on-demand versions of their toolset.

In the session this morning, Callidus VP Shanker Trivedi detailed how Callidus dealt with the widely-variable resource need that their on demand customer base creates. Everyone does payroll at the same time, and they have to handle millions of transactions to properly account for payroll. That means that twice a month they have immense computational needs.

Their solution, by the way, is Java based.

So what did they do? They turned to Sun Grid. As far as I could tell, Sun was already helping them with hosting their applications on its servers. For Callidus's huge computational needs, they simply ported their app to run on SunGrid, Sun's on-demand compute utility. Now, they have dynamic access to all of the computes they need--and they pay by the CPU hour.

This is the reason I came to SaaScon: to find out if there is an intersection between grid computing and SaaS. Just as I had imagined, the need is out there.

For Callidus, the choice to use SunGrid was an obvious one: they didn't want to host it themselves, and their app was written in Java. Sun had the perfect tool.

Of course, not everyone is in the same situation. Dan'l Lewin wrote a column the other day espousing the view that Microsoft has a great platform for SaaS; you should check it out.

Now, if any of those companies that have made the decision to use the Microsoft stack would like to investigate options for grid computing on that platform...well, they should send me an e-mail.

Technorati tags: grid computing, saas, saascon

Wait. I thought "S-a-a-S" spelled "panacea."

T

  hat splashing sound you heard at 9:30 this morning was a big, cold bucket of reality being poured on the heads of the SaaS vendors here at SaaScon.

The second morning keynote was a panel featuring not SaaS vendors and pundits, but rather CIOs of "real" companies--companies that buy both SaaS and on-premise software.

Tom Murphy of AmerisourceBergen, Jesus Arriaga of Keystone Automotive Industries, and James Woolwine of Majestic Insurance were the CIOs. Their 45-minute panel was interesting, informative, and grounded in reality.

Are these guys down on software as a service? Not at all. They think it holds great promise for certain applications. But they're CIOs: they're pragmatic, they're realists, and ultimately they're responsible for all of the software that gets implemented in their companies.

They've all had experience with SaaS, and certainly some of it was positive. But they seemed to find that SaaS vendors just aren't owning up to the reality of implementing software. They particularly took to task some of the mantras that SaaS companies repeat ad nauseum: "It's easier to implement," "It's easier to use," and "You can pay as you go and stop at any time."

These CIOs know that implementation is always difficult. Whether that software is running on-premise or not, you still have to integrate with the same systems. Ease-of-use isn't guaranteed just because the software is coming in a browser. And the idea that, when all of your critical data is stored in a SaaS vendor's database and datacenter, you could just change your payroll app one day is, well, ridiculous.

One interesting point: these CIOs wanted different things out of SaaS. Woolwine, the CIO of a $100M insurance company, would love to outsorce e-mail. Murphy, the CIO of a Fortune 100 pharmaceutical supply company, would never dream of outsourcing e-mail.

What do they like about SaaS? Well, naturally, it's ease-of-deployment. For anything that's delivered in a browser, there's no install, and that's nice.

But, for them, installation on desktops isn't the hard part about an installation. Integration with existing systems and the training of users on the system are two huge components of any roll-out, and those feel the same for them whether it's a SaaS system or not.

They raised two big concerns: first, data. Not so much security, but who owns it? What happens when the vendor goes out of business? What happens when the contract ends? It's a big concern.

Second, Murphy doesn't like the idea that software can be updated "underneath" him. He's got tens of thousands of employees, and the training and education necessary for rolling out new versions of software is a difficult task: they have strict procedures in place. His question for SaaS vendors: how do I control upgrades and updates? With all of the vendors here preaching "one code base" and "multi-tenancy," it's a great question.

Ultimately, as I said, these guys weren't negative about SaaS. But to them, it's just a delivery model. It's still software. It's got the same difficulties.

Photo credit: justcola

Technorati tags: saas, saascon

SaaScon: I Know You Are, But What Am I?

W

atching a panel discussion this morning break into friendly argument between an SAP representative and the CEO of RightNow Technologies, I realized that it typified my experience at SaaScon (the Software as a Service conference).

I've heard conflicting opinions about, well, just about everything. Some of things I've heard:

I've heard 3 different companies claim to be the "largest Software as a Service company in the world."

I've heard that software is dead. I've heard that enterprise software is dead. I've heard that, with its current 20% CAGR, SaaS will become a $25B market by the end of the decade...roughly 10% of the enterprise software market.

I've heard that the biggest hurdle for SaaS is security. I've heard that the biggest hurdle for SaaS is compliance. I've heard that the biggest reasons to use SaaS are security and compliance.

I've heard it's cheaper in the long run to buy software (on premise). I've heard it's cheaper in the long run to buy software as a service. RightNow Technologies charges the same, hosted or on premise.

I've heard that it's easier to get companies to buy SaaS faster. I've heard that the problem with SaaS as a revenue stream is that it takes a long time to get customers. I've heard that SaaS customers are "stickier:" more likely to stay with the vendor. I've heard that SaaS customers can leave at any moment. I've heard that customers with on premise software are more likely to stay with the vendor.

I've heard that SaaS is the on-ramp to on-premise software. I've heard that SaaS is the off-ramp from on-premise software.

What does all of this point to? Clearly, this is an immature and not-well-understood market. Many companies have many different ideas.

The biggest point that jumps out at me was the CAGR--with huge growth over the next 4 years, SaaS will become 10% of the enterprise software market (leaving 90% on-premise).

This hasn't soured me on SaaS at all. Companies are learning their way around the technologies as they figure out new business models. Disagreement is natural. A lot of what's happening here is so new that the standard answers haven't been agreed upon yet.

A sure sign that SaaS is still defining itself? I've heard several people refer to "SaaS 2.0"--a not-so-subtle attempt to grab on to the coattails of Web 2.0. When your marketeers are grabbing at meaningless labels that have been attached to other technologies, you know that you haven't found your own identity yet.

Technorati tags: saas, saascon

GP and FC: SaaSing it up

(if you missed my previous post, that rhymes with "jazzing it up")

G

P and Fred Chong gave their typical, very informative session about SaaS Architecture this afternoon. My only complaint? Too short. Attention, SaaScon: 45 minutes is not long enough for substantive technical sessions! It's fine if the business track sessions are only 45 minutes, but any technical session worth its salt will need more time than that. Still, GP and Fred powered through, leaving us hungry for more.

Along the way, GP asked an interesting question: How related are SOA and SaaS?

My opinion: of course they are related, but there are some fundamental differences. Similarities? Obviously, they both have a service-enabled interface. Both have a great need for scalability. You may need to be able to provide some sort of metering and billing for each.

But the differences are big, too: SaaS implies multi-tenancy, and that's a big deal. That means writing something that's configurable/customizable for different tenants; for SOA, you don't have multiple tenants. Well implemented SaaS also means defining meta-data so that your customers can have customized data (or some other mechanism for running on different schemas)--SOA doesn't require this.

So it feels as though SaaS has all of SOA's needs, along with a whole lot more complexity.


Back to GP & Fred's talk: they talked about the their three headed beast of SaaS: Scalability, Configurability, and Multi-Tenant Efficiency. These guys have done a ton of work on this (see their white papers), and it is really good content.

By the way, one other thing I'd like to call out about their session: I think, in their 45 minutes, they said the word "Microsoft" exactly once. This wasn't a thinly-disguised commercial presentation: this wasn't a commercial at all. This was an honest, platform-independent discussion of architecture for SaaS. It wasn't specific to the platform, nor did it contain veiled references to the platform--the concepts could be implemented on any platform. I commend them for knowing the difference between a commercial opportunity and a technical session, and I congratulate them for doing it right. An earlier talk from Buell Duncan from IBM contained way too much IBM, and it came across as a commercial.

Back to SaaS: I'm still looking for the best way to exploit grid computing as a means to provide the scalability solution to that three-headed beast--and it looks like my opportunity is coming.

In October, Microsoft is going to release a CTP of a reference architecture of a SaaS application. This will be a living, breathing set of code, implementing a production-quality SaaS application. I can't wait for this. When I talked to GP after their session, he made it clear that they're hoping to release a final version of this code early next year so that people can branch, add their own improvements, etc.

It will be a great opportunity to show how SaaS can take advantage of grid computing to provide the tool to attack the scalability head of the three-headed monster.

Technorati tags: grid computing, saas, saascon

How do you say that?

I

learned an important lesson during Bill McNee's (founder and CEO of Saugatuck Technology) talk this morning: a new pronunciation of SaaS!

I know lots of people who use SAS Software, and it's always bugged me that SaaS tends to be pronounced the same way (rhymes with "pass").

One possible alternative would be to make it rhyme with "moss" (such as the Haas Pavilion on the Berkeley campus). But "soss" doesn't have any zing to it and just doesn't seem likely to fly.

But Bill McNee had a different take on it, and I loved it. When he says it, it rhymes with "jazz." So it kind of sounds like "sazz." It's perfect: it's jazzy, it's snazzy, and it doesn't conflict with SAS.

From now on, I'm using it. Thanks, Bill!

By the way, there are another couple of acronyms that need pronunciations, and I've been bold enough to step up and offer some:
HTML: hot meal
OPML: oapmeal (sounds like oatmeal)

Any others?

Technorati tags: saas, saascon

Wait. I thought O-S-S spelled "Panacea"

J

ust found the Grid Computing blog by Nikita Ivanov (I'm not sure how I missed that before), and I stumbled on this old (but still topical) post about Open Source Software (he refers to it as "OS" in the quote). He's talking about an event he had just attended with CxOs of major corporations (like Wal*Mart):

OS was practically laughed at on this event... Explanation was pretty simple: any serious mid- to large-business would use OS product only if it would have IBM-style support for it. Period, end of story. In order to get IBM-style support you need to pay - and pay a lot, to the point where free licensing in many cases stops making economic sense. This is rather obvious and blunt explanation but the one that fits reality: companies pay > $100K for Linux installation/maintenance/support from major vendors such IBM, HP and Novell...

I don't have anything huge to add to that--it's a view I've long espoused. Open Source Software has some great success stories, and some great technical successes. But it's not a panacea, and, on an enterprise scale, isn't as free as it sounds.

I've heard people here at SaaScon refer to SaaS and Open Source as if they are somehow inextricably linked, and it's a bit annoying. SaaS can be implemented on any OS on any platform--free or expensive, open source or not.

(More posts on SaaScon to come, including an exciting new pronunciaton of SaaS!)

Technorati tags: open source, oss

SaaScon 2006: Keynote 1

T

oday and tomorrow find me at SaaScon: The Business to Business event for Software as a Service.

The morning started off with Paul Gillin introducing Tim Chou (formerly of Oracle). Tim is a polished speaker and he was a wise choice for the opening keynote. He has an easygoing manner, and he delivered his entire talk with no notes and no slides. He just spoke--it was a very compelling delivery...

...and yet, I found myself quite uncompelled by his talk. Interesting, because I agreed with his general point: Software as a Service is succeeding today, will continue to succeed, and will change the software business top-to-bottom.

My first issue with Tim's talk was his take on customization: customization is dead. He gave the example of a dial tone: no one was ever offered a "custom" dial tone, so everyone is happy using the same dial tone. He then gave an anecdote of a customer they analyzed at Oracle: the customer had added 2,000 customizations to their software, but in two years had only actually used 2 of them.

Frankly, I found his examples to be almost condescending to enterprise software vendors. The idea that "because we all use the same dial tone, we can all use identical software" is just silly. Customization is definitely a part of life. If, as an enterprise software vendor, you pretend that you won't need to customize for your customers, you are kidding yourself and doomed to fail. Personal software is a different matter entirely, but this is a "Business to Business event."

I much prefer Gianpaolo Carraro and Fred Chong's take on customization: architect your system so that customization can be handled in a standard way so that you can customize without losing the benefits of shared hosting.

I had another disagreement with Tim's talk: the old "83% of organization's software IT budget goes to supporting existing software" saw. I've heard this before (with different numbers, but all of them large and over 70). The implication seems to be that 17% of a budget on new software isn't enough. But it always makes me think: how much should an IT department be spending on new software every year? Fifty percent? A hundred percent? Can you imagine buying all new software every year, constantly training your users on new software?

Moreover, moving to SaaS may change this number, but it's not as meaningful as you might think: it means that, rather than paying money on managing, you're paying more annual money for license and the cost of managing is built in to that license. And if you are able to get that software cheaper (because your SaaS provider is doing a good job of taking advantage of economies of scale and passing that savings on to you), that doesn't mean that the you now have more budget to spend on new software: it means that you now have a smaller budget. The part of your budget that was paying to maintain that software is now built into your license cost (and, ideally, it's smaller).

There seems to be this myth among companies in SaaS discussions that if we can lower software cost through SaaS, customers will be able to spend more money on software.

Wrong. Customers like SaaS when it saves them money. It doesn't free up huge IT budgets--it helps them decrease their IT budget.

Anyway, I'm glad to be at the conference, and I'm exciting to see the sessions (especially GP and Fred's talk later today)

Technorati tags: SaaS, saascon

Excel Services Is Cruising the .NET Waters; Excel 2007 Missed the Boat

I

have blogged pretty extensively about my experiences writing .NET code for Excel Services (the server-based version of Excel that will be part of
SharePoint Server 2007). To sum it up: it's awesome. It is a fantastic idea, and the seamless way they've incorporated .NET programmability (with attribute-based model) is elegant and functional.

I've been writing .NET code behind spreadsheets pretty much since they released VSTO in 2003, and this is a tremendous improvement. To make a .NET method available in Excel Services, you simply add a [UdfClass] attribute to your class, and a [UdfMethod] attribute to your method. That's it. Deploy the DLL (or EXE) to the SharePoint server, and you can now use that method from within Excel cells in Excel Services.

This is, well, amazingly simple. I was (frankly) astounded at how easy it was--and I've been using VBA and VSTO from the get-go. Finally, a very simple way to do real programming behind a spreadsheet. VSTO was great, too, but you needed an add-in for Visual Studio, and you had to deal with all sorts of .NET permissions restrictions and deployment difficulties. This, on the other hand, was quick, painless, and elegant.

The only hitch? When working with Excel Services, you build your spreadsheets in Excel 2007 (naturally). And Excel 2007 doesn't know anything about tyour User Defined Function. So when you're working in Excel, your function calls look like this:

See all of those #NAME? values? That's because Excel doesn't recognize the function.

I asked Shahar Prish, Excel Services developer and blogger extraordinaire, about this, and he promised to write some posts about how to make that same UDF available in Excel. And, in the last few weeks, he's kept his promise.

As he explained here and here, there is a "simple" way to make your .NET method available in Excel:

COM.

That's right. COM. The technology of the 90s. And, it's a typical COM pain in the ass. I'm borrowing liberally from Shahar's post for the code below. You simply:

Decorate your class like so...

Add register and unregister methods...

Use regasm to register your COM server

Tell Excel to enable your Automation Server as an add in

Whew!

And, get this, that's only if you have zero-dimensional inputs or outputs. If you have 1 or 2 dimensional inputs (my first real UDF had 10 inputs, including a 2-dimensional array), it's even more complicated.

What does all of this mean? Well, I understand that Excel Services and Excel are two different products, and that one was born in the .NET era and one came of age as the earth was cooling. Let's face it: Excel has a legacy of code older than college freshmen.

And yet, Microsoft has been pushing .NET on all of us for 6 years. They've had lots of time to improve Excel's integration with it. Forcing developers in 2006 to go through all of that COM work is, well, enough of a pain that people just won't do it.

Shahar, by the way, is a great guy and did a great job with his posts. I continue to be impressed with that team and with their product.

I only hope that the rest of the Office teams catch up with them.

Photo credit: matthew_hull

Technorati tags: excel 2007, excel services

New Feature Roundup!

T

he last few weeks have been crazy busy and, as usual, blogging has, for the most part, fallen by the wayside. I even let the Digipede Network 1.3 product release slip by with nary a mention!

John and I were at GridWorld (he was giving talks and receiving awards, I was learning about data grids and talking to partners) when the press release went out.

There are some very cool features in this release--here are a few (and why I think they're cool). Note that this isn't a comprehensive list; shoot me an e-mail if you're interested in one of those.

Task Progress Reporting: The Digipede Framework now allows distributed applications to report progress information back to the server during tasks--so those long running tasks can give feedback on how they're going

Windows CCS Integration: Want to use the Digipede Network to run jobs on those hot new CCS nodes, but make sure that you're not stealing cycles from the cluster's important jobs? No problem. The Digipede Agent can now monitor the CCS job scheduler and stop using the node whenever it's being used by CCS.

Major API Additions: This list is long! Now you have programmatic access to the pools on the system and to all jobs on the system. There are many new helper methods that make grid-enabling your .NET applications (or, as Greg Nawrocki would say, "virtualizing your applications") easier than ever.

Improved Debugging: Speaking of .NET applications, those are easier than ever to debug, thanks to some new methods on the DigipedeClient and JobTemplate.

AsyncResult pattern: In addition to the asynchronous pattern that we featured in previous releases (XXXXAsync), we're now supporting the BeginXXXX/EndXXXX pattern.

Monitor system from Workbench: Digipede Workbench is a job submission tool, not a system administration tool. Still, you might want to start Workbench in the morning and look at all the jobs that ran last night, right? Maybe attach it to a running job and follow its progress, or see which tasks ran on which machine? Bingo. It's in there.

Everyone here did a great job on this release, but special thanks go out to Rob, Mark, and Jeff.

No time to celebrate, though...we've got product plans to finish for upcoming releases. And evermore publicity, too. Look for a post soon that details some exciting upcoming events: MSDN webcasts, magazine articles, and more!

Thanks to the Web2.0 Logo Creator by Alex P for the logo.

Update 2006-09-20 9:50pm: At Rob's suggestion, I clarified the description of the async pattern.

Technorati tags: grid computing, digipede

Grid Computing in Action!

F

or people who want to see some "grid computing in action," here's a quick video:

The video below shows a grid computing race: on the left, you'll see a calculation of a portion of a Mandelbrot set in real time on my machine. On the right, you'll see the exact same code executing on a testbed of fivenine machines.

Update 2006-09-20 10:24: Changing video to one with narration.


It doesn't take a rocket scientist to see that fivenine machines are much, much faster than one.

How easy was this? I simply wrote a class that could calculate a portion of the Mandelbrot set. On the left side, I started a thread that called that class 25 times in succession (iterating over the coordinates for this graph). On the right side, I iterated over the coordinates and created 25 instances of my class, then shipped those off to the Digipede Network.

Before my local thread can even complete one calculation, the grid has returned a bunch of calcs (the black squares take the longest--that's why the bright ones get returned so much faster). In the end, the 59 machines finished in 15 seconds, and my local machine took over a minute.

I may produce a bunch of these videos. It's pretty quick and easy with SnagIt and YouTube (and, yes, as soon as I get into the SoapBox beta, I'll use it).

Technorati tags: grid computing, digipede

Winner, winner, chicken dinner!

Well, that's what the craps dealers in Vegas might say.

It was a good day for us at Grid World 2006.

John gave a talk, and it was SRO. Apparently, lots of people are wondering how it's possible to offer a grid solution that SMBs can buy, too.

Then, there was an Awards Ceremony for GridToday's first annual Readers' and Editors' Choice Awards. It was not a very glamorous event--Joan and Melissa Rivers' calendars must have been full--but it was rewarding, nonetheless.

The Digipede Network received an Editors' Choice Award for "Best price / performance middleware solution for a grid implementation." Well, we actually shared that award with United Devices, but let's not quibble over details. It's great to be recognized, especially with a company who has been doing this for so much longer than we have, and especially when so many other companies were considered for this award. (And, by the way, if anyone is actually doing price/performance benchmarks, I'll put our solution up against absolutely anything in the market. Go ahead--make my day).

I attended plenty of interesting sessions (great data grid sessions from Tangosol and GemStone), too. More later about the synergy between compute grids and data grids...

Technorati tags: grid computing, digipede, gridworld2006

Preparing for a release? Get out of town!

R

eleasing software wreaks havoc on my blogging schedule! I've been meaning to post all week to follow up on my last post about concurrency, but with the imminent release of the Digipede Network 1.3, I simply haven't had time.

Here's a synopsis of what I would have said, had this not been the week before release: Go read Joe Landman (here and here) for some really interesting stuff on the future of concurrency, from a software AND hardware perspective...

As the Director of Products (and product manager for the Digipede Network), release time means something different to me than it ever has before (as a developer or manager of developers). Now, I'm not fixing bugs (thankfully, there have been few bugs in the last couple of weeks). Instead, I'm fixated on the myriad details that surround the software: API references (for our ever-growing .NET and COM APIs), user documentation, administrator documentation, ensuring that the test suites are covering everything, monitoring the bug database (we use Gemini) to make sure nothing's popped up, etc. Add to that one of our busiest sales months ever, and it leads to some thinly stretched employees and some long nights.

In addition to releasing software next week, John Powers and I are headed to GridWorld in Washington, D.C. John's giving a talk which sounds like it's going to be terrific; I'm just attending. There are a bunch of exciting sessions (in fact, I don't think I'm even going to have time in my schedule to see John's talk!).

If you're attending next week (or if you're just in DC), and you're interested in talking grid with me, feel free to give me a call (510-816-7551) or send me an e-mail (dan AT digipede DOT net).

Photo credit: chelle

Technorati tags: grid computing, digipede