How cloud computing kills clustering

Cloud computing's benefits, including improved disaster recovery and increased connectivity flexibility, have surpassed what clustering has to offer. Cloud is the new cluster.

The cloud has become the new computing cluster. Clustering -- the ability to have two or more computers operate in lockstep for highly available systems -- has been around since the earliest mainframe days. But a new take on clustering that leverages virtualization tools is becoming more appealing, particularly as enterprise IT shops gain more experience using virtual servers and as cloud computing vendors add more high-availability features to their products.

[The cloud]...opens up this protection to a whole new set of companies.

Bob Williamson, VP with SteelEye Technology

Clustering used to be a very specialized discipline. You needed to set up nearly identical computers and run special operating system versions to keep them synchronized. Special networking adapters were also required to move data between two PCs at very high speeds.

But networks have gotten faster, and a combination of services that were previously only prevalent in customized clustered configurations -- including high availability, virtual storage management and near-term server failover -- are now available in the virtual world. Many can even substitute for disaster recovery (DR) applications.

So as cloud and its manageable application delivery grows more popular, and as IT shops gain more expertise in delivering virtualized applications, this knowledge can be leveraged into delivering clustering options.

Improved disaster recovery hurts clustering
How has this happened? Virtual machines (VMs) are easily portable and replicated across the cloud, so you can quickly get a secondary site up and running when the primary server has failed.

"We have seen disaster recovery protection now available to a whole class of customers that couldn't do it before," said Bob Williamson, an executive VP with specialized virtualization vendor SteelEye Technology Inc. "In the past, you needed to buy another physical server and have it ready if the primary machine went down. [The cloud] lowers the entry cost for deploying wider-area disaster recovery and opens up this protection to a whole new set of companies that haven't been able to consider it before."

In the past couple of years, the major virtualization vendors have strengthened their ability to provide more capable DR and business continuity. Citrix purchased VMlogix, which allows for more capable automated setup and teardown of machine configurations and automation tools that can be used with multiple hypervisors.

VMware, on the other hand, released vFabric and vCloud Director. The former combines the Spring Java development environment (which VMware manages for the open source community) for a lightweight app server with load balancing and other infrastructure needs, so that you can move things between various cloud instances. The latter can be useful for managing multiple VMs in the creation of a virtual data center and set up preconfigured app services and infrastructure.

Other vendors have stepped in with their tools to help automate the provisioning of new VMs and make DR easier. Novell's Cloud Manager, for example, has automated template-based provisioning processes that allow users to request, approve and deploy new workloads using a single Web-based portal. And its Forge appliance makes it easier to rapidly recover workloads with one-click failover and flexible restore options.

Novell's Forge appliance
Figure 1: Novell's Forge appliance can recover workloads with a single click, and this dashboard displays the status of your VMs. (Click to enlarge)

These offerings aren't appropriate for transaction processing applications, however, where immediate failover is required to handle things like online payments processing or airline reservations.

The cloud has become the new computing cluster.

"There are still times when you need more precise clustering, such as when you can't afford to lose a single transaction and have to restart this transaction on the new machine after a failover," said Carl Drisko, an executive and data center evangelist at Novell. "If your virtual machine goes down, anything that is being processed in memory is going to be lost."

"If you are using some sort of transaction processing system where you need to preserve the state of the server, then you are going to need some special purpose clustering solution," said Ken Oestreich, Cassatt's director of product management. Cassatt supplies automation tools for managing virtualized sessions, but high-availability virtualized applications can work for a less demanding cloud-based line of business applications.

Combining the cloud and clustering
One of the issues with earlier custom clustering solutions is that they require identical hardware and operating system versions for each physical machine that's part of the cluster. The cloud is more forgiving and flexible, not to mention less expensive.

Another issue is that many older-style clusters required high-speed links to tie everything together. Cloud-based offerings are less demanding and can make do with long-latency connections, even across typical Internet lines. VM6 Software has a high-availability Hyper-V-based offering called VMex that can turn two PCs into a cluster without the need for specialized hardware, even using ordinary Ethernet connections for less demanding applications. When the first Hyper-V machine fails, all VMs will be brought up on the second machine automatically.

VM6 Software's VMex
Figure 2: Creating a storage partition using VM6 Software's VMex. (Click to enlarge)

As these "almost clustering" services increase in popularity, look for the increasing sophistication of third-party monitoring vendors to help provide a complete offering.

This means clouds and clustering will become more interrelated and complementary for IT managers. While the two technologies have come from different heritages and infrastructures, they are now merging to provide a powerful tool for managing complex enterprise workloads.

David Strom is an expert on network and Internet technologies and has written and spoken on topics such as VOIP, convergence, email, network management, Internet applications, wireless and Web services for more than 20 years. He has had several editorial management positions for both print and online properties in the enthusiast, gaming, IT, network, channel and electronics industries.

Dig Deeper on High availability and disaster recovery