In yesterday’s batch of links I commented on James Urquhart’s video “Clarifying Internal Cloud versus Private Cloud.”

James Urquhart on Internal/External vs Private/Public clouds. While I agree that Private is not always equal to Internal, James’ [re]definition of increasingly accepted terminology just serves to muddy the waters by introducing the existence of unified control systems as a defining characteristic.

James called me on my comment, asking what I see the difference being.

Just as I finished typing the above words, Savvis’ Benson Schliesser jumped into the mix with his post “Cloud: Private vs Public, Internal vs External, Oh My!” adding an interesting dimension to the dialogue.

First off, my thoughts on James’ video. In the video he is making two points:

• Internal/External is an issue of “ownership”
• Private/Public is an issue of “control”

To illustrate the latter point, he provides an example of a unifying control system creating a “private cloud” from internal and external resources. This example is what I was primarily responding to, as it seems to label as “private cloud” an idea more commonly called “hybrid cloud,” primarily by virtue of the unified control system. This masks the “less private” (e.g. hybrid) aspects of the cloud and as a result (IMO) “muddies the waters.”

The notion of private vs public cloud remains central to the broader discussion around cloud computing, at least from an enterprise perspective. Right now, there a number of terms that seem to be used more or less synonymously to identify the same concept: private vs public, internal vs external, on-premise vs off-premise.

Most of the time, when I hear folks trying to subdivide this world even further, it comes as part of an attempt to sell “public” cloud offerings as “private” ones, to put it bluntly.

To extend James’ example, if I deploy a tool that allows me to “control” virtual resources from Amazon EC2 and GoGrid, either directly or via a broker, without having to utilize either offerings’ native tools, does that mean I’ve created a “private” cloud based on these “public” cloud services? Amazon would like you to think so. Most enterprises would not.

I define “private cloud” as a system exhibiting cloud characteristics, but operated for/owned by/contolled by a single organization. As mentioned in previous posts, I believe tenancy (an attribute that didn’t even figure into your list, Benson) is a key determinant of “privacy.” By tenancy I’m referring to who the cloud is “operated for.”

Ownership seems less important — if the system is physically owned by a third party but operated for my exclusive use (maybe even by me), that’s private. And control seems too hard to pin down — do I control EC2 since they give me an API? If you control a system for my exclusive use, but do so on my behalf (i.e. I control you, e.g. contractually), who is in control? (In other words, is “control” a transitive property?)

So Benson, to the point of Bryan’s quote, off-premise private cloud works fine for me. Shared private cloud, not so much. (The private cloud is shared among its users, yes, but not with other customers.) Shared clouds with better security measures and “enterprise grade” QoS are  perhaps “enterprise-grade public clouds” in my book (or maybe “virtually private” clouds).

In the trenches, of course, you’re absolutely right though. There are more than just a couple of defining attributes of cloud computing. And, in the same vein, we will measure each of the attributes with more than a single bit.

In the end, customers are smart, and the labels we put on our offerings will mean less and less as the market matures and educates itself.

The Intalio acquisition of Webtide, the company behind the popular open source Jetty web container, seems interesting and perhaps a bit under-reported, having been drowned out in the news from the big VMworld and JBoss conferences taking place. There are certainly folks in a better position to provide in-depth analysis (any thoughts @billynewport, @johnrrymer or @mjasay?), but it strikes me that there is an interesting strategic back-story here, if not interesting implications for the Java cloud platform space.

I’ve come across Intalio from time-to-time over the years, the first time back in 2003 on a BPM market research project for Plumtree. At the time they were a pure-play BPM player with interesting technology. Now they bill themselves as “The Private Enterprise Cloud Company” (ahem) and offer a curious array of products ranging from a hardware “cloud appliance” to BPM to CRM.

For those not familiar with Jetty, it is one of the more popular web containers for Java, second only to Tomcat in installed base (though hard numbers are elusive). The web container is a key component of the stack for Java web apps and Jetty has been baked into a number of Java cloud offerings including Google App Engine.

What makes this interesting for cloud-watchers is the fact that SpringSource, just acquired by VMware, has a controlling influence over Tomcat through its prior acquisition of Covalent. With VMware talking up an aggressive push into the Java PaaS market, this puts them at odds with Google over GAE (which uses Jetty), Stax (which uses Tomcat) and others*. And there are have already been rumblings within the Tomcat community accusing SS of holding back features/fixes for the open source product in favor of their commercial offerings.

So, by my calculations, Jetty becomes a bit of a hot commodity in all this, making the timing of Webtide’s sale to Intalio curious. Hopefully they got a great deal; they’d seemingly have had much to gain by waiting for more favorable market conditions but I guess that’s usually the case.

Also potentially quite the shrewd move on the part of Intalio, who has inserted themselves into the middle of a quickly growing and high-visibility segment of the cloud market. (I know none of the details, it could in fact be a horrible move

Aside: There’s an interesting side story about the recent spate of acquisitions of commercial open source companies in the Java tools space. Cloud Foundry to SpringSource, EHcache to Terracotta, and now Jetty/Webtide to Intalio. These were all very small shops. Is the long economic winter taking its toll on open source developers?

—–
* disclosure: including, to a limited extent, my employer – Appistry – whose CloudIQ Platform is a self-hosted PaaS offering supporting both Java and .NET

Thanks in part to an aggressive pro-cloud stance taken by the Obama administration, the federal government is positioning itself to be an aggressive adopter of cloud computing and cloud technologies, as evidenced by the recent GSA RFQ. As a result of the government’s push towards the clouds, there has been a strong desire on the part of agency IT leadership and practitioners to hear from real cloud users, providers, and technology suppliers. A number of civilian and defense CIOs and CTOs have taken “study tours” to visit firms like Google and Amazon in an effort to transfer back best practices and lessons learned back to their agencies.

In response to this need, and to help facilitate the sharing of cloud success stories with government, last week Appistry hosted a “Success in the Clouds” breakfast seminar in Washington, D.C.

I provided the “warm-up act” providing an overview of a variety of cloud computing topics and trends before inviting up Mike Rains from FedEx to speak. Mike shared FedEx’s real-world experiences as an Appistry customer and early adopter of private clouds for mission-critical applications, with much of his time spent addressing direct audience questions.

Here are my slides.

One of my take-aways from the event was a reminder of just how broad the cloud discussion is: At one of the breaks I was approached by an attendee asking whether I’ve seen customers using cloud computing as a way to reduce software licensing costs. I responded yes, as we’ve often seen a shift to cloud happen in parallel with a move to open source to accomplish just this. Her response startled me at first, “You mean to products like Open Office?”

To a server-side guy like myself, pushing desktop apps into the cloud is sometimes not quite as interesting as all the datacenter, application-tier and middleware magic that it takes to make that happen, but to end-user organizations the shift away from desktop applications and to web-delivered apps can provide tremendous economic value. It was a good reminder to be conscious of the many perspectives we bring to cloud.

John Foley of InformationWeek published a nice bit of investigative reporting in his recent post Amazon Web Services Secrets Revealed. The post tries to get at growth estimates of Amazon’s Web Services business from a number of angles and provides evidence of some impressive growth rates.

One data point cited in the post is some nice research done by Guy Rosen, posted to the InfiBase blog. Using his company’s tools he’s looked at the top 500,000 Web sites and determined that while only a small number of them are hosted on EC2, Amazon (a) hosts a disproportionate number of the large sites and (b) has experienced dramatic growth.

The actual estimate Guy provides for AWS hosted sites is very small on a percentage basis (0.28%) but it must be kept in mind that the hosting business is extremely fragmented and in fact a market share as low as 3.25% is all it takes to make it to the top of the “top hosts” list, according to data from the Netcraft Web site.

I find it interesting to see the AWS business measured this way because it supports a perspective I’ve held for some time: While some believe cloud computing results in the eventual dominance of just a handful of “big computers” (e.g. data centers, the list of which always includes Google, Amazon, Microsoft and whatever company employs the person telling this story), it’s always been my opinion that there will be many cloud providers and in fact, at some point in time, every hosting provider becomes a cloud provider in some sense of the word.

Cue up the arguments about “what really defines cloud computing” if you will, but this strikes me as an inevitable market shift.

At the highest level, what separates a hosting provider from a cloud provider are technology and business models. (And the former enables the latter.) The core technology required to launch an infrastructure-oriented cloud hosting business, e.g. one that provides access to virtual machine instances via an on-demand pricing models, is readily available today and the complex integration work that’s currently required to offer a complete solution will eventually give way to packaged tools.

Consider the fact that Plesk, one of the leading providers of “hosting automation systems” (i.e. control panel software) is owned by Parallels, a leading[1] provider of desktop and server qwwavirtualization software and emerging cloud computing player, and it’s easy to see the beginnings of the convergence of the hosting and cloud worlds.

Bonus Links:

• John’s article and the InfiBase data reminded my of an earlier post by Ruv, Amazon reserves the right to host your applications, which looked at the EC2 “reserved instances” pricing model and how it made the company’s offering price competitive with VPS hosting plans.
• In researching this article, I came across Krish’s recent CloudAve piece, Handful Of Monopoly Infrastructure Players – A Shortsighted Idea, which I’m not sure how I missed. Great article that compliments my own with a more “philosophical” and macro-economic perspective.

[1] A forum post found via a quick Google search suggests Plesk/Parallels holds some 30% market share relative to competitors like cPanel and H-Sphere; not hard data, but consistent with what I’ve seen out in the market.

Responding to an informative and well-researched–albeit a bit short sighted–study by McKinsey, a post earlier this week by Rajen Sheth on the Official Google Enterprise Blog helped to clarify for the world exactly what Google means when it talks about cloud computing. The post, a bit of a rail against the idea of private clouds, builds the argument for public clouds in part on the idea that Google and a few other companies have a lock on advancements in hardware infrastructure, software infrastructure, applications and innovation.

The fact is, as much as it might like to think so, if there is one area that Google does NOT have a monopoly in it is innovation, and I for one am irked by the implication that the enterprise will hand over the "keys to the kingdom" because it’s not able to innovate. Actually, enterprise IT organizations have access to many of the same innovative technologies that Google claims give advantage to public clouds.

That is the idea at the very heart of the private cloud "movement": that, through the aggressive adoption of innovative technologies like virtualization, commoditization, cloud platforms and eventually cloud-oriented software architectures, enterprises can achieve many of the advantages of public clouds without their disadvantages, which include concerns about security, visibility, control and lock-in.

So let’s look at Google’s arguments against private clouds in the order that they were presented in the original post, and discuss how enterprises can utilize the same or similar ideas in their private cloud initiatives.

Hardware Infrastructure

Google leads off its argument with a focus on the hardware it uses, arguing that its use of commodity-grade servers it uses are cheaper than the typical "enterprise-grade" server used in corporate data centers. No argument there. (Side note: I used this slide back in ‘05 to illustrate a similar point.)

What he seems to miss though is THE POINT of commoditization–that this is not cutting-edge proprietary technology only available to Google. The single-CPU server with SATA storage presented as an example of the lower-cost hardware platform that Google uses is available to any enterprise from a very large number of vendors at extremely low price points.

Rajen makes the great point that what enables Google to use very inexpensive, and inherently unreliable, servers is the fact that they’re "building reliability into the software" so that when the hardware does fail (inevitable with commodity boxes) the work continues on on another server.

Sounds very cool, but off-limits to the rank-and-file enterprise? Actually, no. This idea is one the key themes upon which Appistry was founded, and it lives on in our CloudIQ Engine product. Since this kind of software-based reliability (what we call "application-level fault tolerance") can be heady stuff, we built it into the platform itself, so that the developer doesn’t need to think about it.

Perhaps my colleague Guerry said it best in his very first blog post, "Cross-pollination, Beehives, the Game of Life, and You":

Did we actually write our software by observing bees? No, not directly; however, do stop and consider what a beehive teaches us. Individual bees, each important to the hive community, accomplish what needs to be done without central control, and with no single point of failure. Well, there is the queen, but let’s not quibble. If a bee dies, another picks up the work, and the hive continues by self-organizing, and self-healing.

The energy efficiency of Google’s servers and data centers is mentioned as a key advantage of public clouds, and this is an area that they have provided some leadership on. But the comparisons they like to make are to the "typical server" and the "typical data center" not to a data center that is pursuing a private cloud strategy. How do their servers compare to the best-in-class available from vendors like Rackable, who supplies high-density, energy efficient servers to the likes of Amazon, Yahoo, and also sells to, you guessed it, the enterprise?

The climax of the hardware-infrastructure-argument-against-private-clouds is the obligatory reference to Gartner analyst Darryl Plummer’s 2006 proclamation that enterprises spend some 80% of their budgets "keeping the lights on" versus driving change and growth in their businesses, followed by an assertion that:

The reality is that most businesses don’t gain a competitive advantage from maintaining their own data centers.

Perhaps, Google, but it’s about the applications, not the real estate. Many enterprises do in fact gain a competitive advantage by maintaining their own mission-critical applications which operate against proprietary data sources, are integrated to other back-end systems, and are tied to enterprise user repositories, all of which simply can’t and won’t be moved to the cloud immediately because of the issues mentioned above.

Software Infrastructure

The next area covered in the Google post is software infrastructure. While Google’s custom database, app server and web server is mentioned in this section, the bulk of the section focuses not so much on the quality and characteristics of the software infrastructure Google provides, but on the as-a-service model through which they provide it.

What’s obviously missing in this equation are the huge portfolios of existing applications that exist within today’s enterprises, none of which were written to use BigTable. This is the reality of cloud computing for the enterprise. It will evolve from the adoption and application of cloud-like technologies within the enterprise, first applied to today’s enterprise workloads. Over time, enterprises will adopt more cloud-friendly approaches to application architecture, but it won’t happen in a big bang and it certainly can happen behind the enterprise firewall.

That’s the guiding philosophy behind our product line, Appistry CloudIQ Platform. Our CloudIQ Manager is a tool used by infrastructure and operations teams to manage untouched applications (e.g. the existing apps) in a cloud environment — private, public or hybrid. And, CloudIQ Engine is a tool used by applications teams looking to create cloud-native applications with an inherent agility and ability to scale. In this way, CloudIQ Manager serves the role the of a private cloud “on-ramp” technology, and CloudIQ Engine is more of an advanced tool used to address specific needs.

Applications

The discussion around applications in the Google post focused on the virtues of application outsourcing, with email being used as the primary example. No knocks against Gmail–it rocks, but having just come from a panel of CIOs discussing this very issue, the challenges are not simply in justifying the cost of hosted email. For those CIOs the bigger issues were legal and compliance ones related to discovery and data retention. Corporate control over the data from a legal perspective is a huge issue as well: is Google willing or able to fight a 3rd party subpoena of my data to the same extent that I am?

But what if those organizations willing to adopt less feature-rich email and collaboration environments could take a page out of the Gmail playbook and deploy agile, private cloud-based email systems for 1/2 the cost of today’s systems, versus 1/3 the cost that Gmail can offer but without the accompanying legal and control issues. My guess is they would jump at it.

Innovation

As mentioned before, this is the one that got me going. Contrary to the opinion expressed in the post, today’s enterprise is better positioned to drive innovation through IT than ever before, thanks in part to technologies like those being developed to enable private clouds. Yet clearly the enterprise has more sophisticated requirements than the consumer user of SaaS services.

Conclusion

Rajen’s post ends by posing the question:

As companies weigh private data centers vs. scalable clouds, they should ask a simple question: can I find the same economics, ease of maintenance, and pace of innovation that is inherent in the cloud?

Unfortunately Rajesh, enterprises must also ask themselves the complicated questions: Can public clouds provide the security, visibility, control, compliance, support for existing applications, and legal protections required to even be considered for my applications?

Please don’t take any of this to mean that Sam Charrington or Appistry is a against the use of public cloud services by the enterprise or anyone else. Nothing can be further from the truth. WE ARE HUGE FANS!

That said, we strongly believe that enterprises must be empowered to choose the right cloud for the right job, and the model that we believe will emerge is one of a hybrid or federated cloud strategy centered of the private cloud but extending into the public clouds as the application use-case allows. Clearly we are not alone in holding this view. We continue to see strong end-user movement in this direction. Furthermore, both Forrester and Gartner have recently come out with proclamations of support for private clouds as the future of the enterprise data center.

So please, please, please keep up the good work, Google. The industry–no, the world(!)–needs more innovation around all flavors of cloud computing. But don’t discount the efforts of IT organizations to balance the constant drive for innovation with the unique requirements of the enterprise. And don’t discount the result of those efforts, the emergence of the private cloud.

Yesterday, the Cloud Computing Interoperability Forum (CCIF) was forced to pre-announce the Monday launch of a (now) widely-anticipated Cloud Computing Manifesto — a call to action for the worldwide cloud community to get involved and embrace the principles of an open cloud.  The pre-announcement was precipitated by a blog post by Microsoft Azure product manager Steve Martin, who voiced his disapproval at the way the Manifesto was being developed.

As you can imagine, there’s a sordid back story. You can some of the background and as well as dive into the still-raging discussion on the CCIF Mailing List on Google Groups. Rather than get into the antics, I think it’s important to look at what is really at issue here, and what needs to happen once the CCIF releases the Manifesto to the wild on Monday.

If you’re a regular here, you know that I’m biased. I am a co-founder of CloudCamp and the CCIF and have worked hard along with Reuven Cohen, Dave Nielsen, Jesse Silver and Sara Dornsife to get these organizations off the ground and make them successful. In addition, my company Appistry has been a staunch supporter of these efforts in word, deeds and dollars.

You’ve Got to Start Somewhere

At the heart of this issue seems to be the idea that, in order to be “valid,” and to advance the dialogue in the community, the Manifesto must be developed from the ground up in a public forum. I disagree with this.

Don’t get me wrong. I am in whole hearted agreement with the open process that is being called for, as I think all of us at CCIF are. But what I think is missing from the dialogue is simply the fact that “you’ve got to start somewhere.”

From the beginning the Open Cloud Manifesto was meant to simply be that starting place. Something solid and tangible for the community to respond to.

To me, it’s equivalent to how the open source community has successfully worked for years: someone gets an itch, scratches it themselves (or with the help of a few others), and then turns it over to the community. That’s what’s happening here.

Pushing the Ball Forward

So what needs to happen now? As of this moment, the so-called “secret” manifesto is now public. It is now in the hands of the community. We’ve seen initial “throw the baby out with the bath water” reactions, but I think we can do better as a community.

So now I’m pleased to announce my own Manifesto, the Open Cloud Manifesto Manifesto:

• Read it before you judge it
• Fix it once you’ve read it
• Keep pushing forward

The industry needs this.  Someone had to take the lead and it just so happens that — this time — it was Ruv and the CCIF that did it. 

The fact is, it’s out there now and this is the time for the community to make it better. Our customers don’t care about how the sausage is made.  They just want some good tasting Jimmy Dean (or Morningstar for vegetarians like me).

Two weeks ago Appistry announced our cloud computing market predictions for 2009. The predictions were well received, save a few haters over at Oracle .

To elaborate on our predictions we held a Webinar last week called “Forecast for 2009: Cloudy!” We wanted to try something new-and-different for the Webinar so we decided to run it like a panel discussion. The format not only turned out great, but was a lot of fun to do. In fact, it was such a good time that we’re planning to make it a tradition!

I moderated the panel of Appistry executives, which consisted of:

• Kevin Haar, CEO
• Bob Lozano, Chief Strategist
• Michael Groner, Chief Architect

Each of them discussed one of the predictions and I peppered them with my own questions and audience questions throughout.

The recording has been posted and I encourage you to take a look to learn more about our thoughts on the quickly-evolving cloud computing market. There is an on-line version, pdf slides, an mp3 podcast, and even an iPhone version for you to download.

BONUS LINK: In a separate but related note, I was quoted in a recent article in the San Jose Mercury News entitled “Forecast for computing: cloudy.” Much of the nuance in my assessment of the market was not captured in that quote, but I stand behind the statement: SaaS aside, the real cloud opportunity for the enterprise is in private clouds.

This has come up a lot recently, most recently when Coté tweeted:

@cote: Is the cloud categorization getting down to "apps, platform, and infrastructure"? I keep hearing that divide-up: http://tinyurl.com/4ghj7x

This taxonomy is far from perfect, but I think it does a good job of separating out the different types of clouds out there. Michael Sheehan from GoGrid was one of the first to publicly illustrate the cloud taxonomy with a pyramid (and I think he’s trying to apply for a trademark on it . (Image to the right stolen from Michael!)

As provider of a cloud platform, I am frequently called on to differentiate between cloud platforms and cloud infrastructure. Anyone who has used both Amazon EC2 and Google App Engine groks the difference between these two concepts, so I tend to use those offerings to illustrate the difference.

It’s really all about level of abstraction. And this translates directly to the end-user experience. With cloud infrastructure like Amazon EC2, the central theme is the virtual server. The user is (sometimes painfully) aware of the number and type of servers/virtual machines that they have running and management is done by individually logging in to each server, e.g. via SSH. Note that the user doesn’t know things like the brand and physical location of the servers, nor does the user expect complete or exclusive control of the machine. Those are all things that contribute to making the relationship cloud-like.

On the other hand, consider the end-user experience with App Engine. The central theme is really the application itself.  The user develops their application and with a single invocation of a command-line tool, packages and deploys the application to the cloud. The platform takes care of actually deploying the application to the right machines, based on policy, utilization or whatever. The user never logs into an individual server. The don’t even know at a given point in time how many machines their application is deployed out to. The platform takes care of all of that, plus provides APIs and services to the applications themselves. (Much of this is true for Appistry EAF as well, but the model is very different from App Engine.)

I’ve got some slides I use to elaborate on the distinction between the layers. Please take a look and let me know what you think:

Stephen O’Grady wrote up a nice post back in November called Cloud Types: Fabric vs Instance, where he described the Platform and Infrastructure layers as Fabric and Instance respectively. (Fabric has a nice ring to it, for obvious reasons He has a nice description of each that is worth checking out.