Problem solve Get help with specific problems with your technologies, process and projects.

Amazon's EC2 and the open source cloud projects

Discover the benefits of the open cloud computing solution, such as Amazon's EC2, and how open source cloud environments like Eucalyptus can provide support for EC2 web services and applications.

If cloud computing is considered "platform as a service," then there are "platforms" that may be uniquely flexible enough to support both development and deployment of cloud applications. With a little inventive mixing and matching by developers, it's even possible to support both public and private clouds.

Amazon's Elastic Compute Cloud (EC2) is the most generalized and well-known of the cloud computing service offerings. It's thoroughly documented, with a good developer program and easily executed development, and it offers comprehensive support for narrow- or wide-ranging deployment of applications. Pricing is pay-as-you-go for the ability to host Amazon Machine Images (AMI), which consist of an application and their associated runtime libraries. There is a per-hour charge and a data exchange charge, both of which are reasonable, and there is no monthly minimum. Any additional charges are for "reserved," as opposed to on-demand, service. Persistent data storage is also available at additional cost per gigabyte.

Any programming language can be used for application development, and deployment is supported on x86 platforms running Linux, Solaris and, for an extra charge, Windows. The AMI can make development easy by mirroring the development/test system. EC2 applications can use other Amazon web services like Simple Storage Service, Simple Queue Service, and SimpleDB, but this is not required. EC2 is also supported by key IT vendors: Oracle and IBM have partnered with Amazon for some cloud offerings, and their collaboration will enhance the utility of EC2 applications for enterprises.

Since EC2 can host virtually any application, it can also use any protocol or web service/service-oriented architecture (SOA) technology to integrate with companion applications or components in the enterprise. There is a standard EC2 linkage capability available, and in addition, the Eucalyptus open source cloud project mentioned below provides an EC2-compatible cloud framework. Eucalyptus can be used with any suitable data center for a private cloud implementation that offers the same overall structure and management as Amazon's EC2. For those looking to develop extensive EC2 projects, Eucalyptus can be a major benefit.

Open source projects like Eucalyptus, Nimbus and Hadoop offer an open-source version of a cloud computing architecture suitable for both public and private applications. Each of these cloud environments can be used either to augment EC2 or as a stand-alone private cloud architecture.

For those developing or deploying on Amazon's EC2, the Eucalyptus (Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems) platform may be the most interesting. A project of the University of California Santa Barbara, Eucalyptus' goal is to replicate the EC2 interfaces and functionality. This shows in the fact that Eucalyptus is the most EC2-like of the open source cloud architectures. There are some differences in the way that things such as Amazon's Availability Zones are implemented; these, however, seem to have more impact on planning or on translating deployment assumptions from a Eucalyptus cloud to EC2 than on development. Eucalyptus' biggest benefits are the ability to create a testbed for local development of EC2 projects, along with being able to integrate a private cloud with EC2-hosted applications.

Nimbus is another cloud platform offering similar capabilities to EC2. The focus here, however, is to provide support for EC2 web services so that EC2 S client applications can be used with Nimbus clouds. Nimbus offers an EC2 front-end that allows web services used for EC2 management to run on Nimbus clouds; all of EC2's features, however, are not supported. Nimbus also includes an EC2 back-end interface that, in allowing EC2 images to load, runs EC2 as an extension of Nimbus. This feature is evolving, however, so check on its latest capabilities before using it. In fact, for Nimbus/EC2 relationships in general, it's important to be sure that the features you need are available before moving ahead with your cloud strategy. Nimbus can also be used with Web Services Resource Framework (WSRF), a popular and generic cloud standard for client access. For applications that require speed in cloud creation and application integration, Nimbus is an excellent choice.

Cloud and grid computing are closely related concepts, and Hadoop is an open source cloud package that pushes into the grid space. Designed for what some characterize as "data-crunching," Hadoop includes its own distributed, scalable file system known as Hadoop Distributed File System (HDFS). It also supports the Google MapReduce programming model, which allows for the distributing of suitable functional problems over large complexes of processing nodes. This essentially means that a large problem is dissected, or mapped, to multiple nodes that all report their results for reduction into a single solution. Its design makes Hadoop especially suited to analytical operations on large databases, and implementations of large distributed spreadsheet-like structures, such as Google's BigTable, are also available in Hadoop. Cloudera, a Hadoop investor, has made available a commercial distribution of Hadoop that can be hosted on Amazon EC2, with a Red Hat-esque offer of support for the Cloudera distribution available for a fee.

The combination of Amazon EC2 and any of these open source cloud platforms can provide an extraordinary head start in getting a cloud computing application developed and deployed. Its flexibilityThe flexibility of the combination also allows for a much simpler method of executionversatile set of execution options, either as a public cloud service or a hybrid of a private cloud implementation with an "overflow" or supplementary set of public cloud features. Even planning and testing a small project on an open source platform is an excellent way to build experience and skill in cloud computing, which could then be transferred to a real business project.

font color="red">Tom Nolle, is president of CIMI Corporation, a strategic consulting firm specializing in telecommunications and data communications since 1982. He is a member of the IEEE, ACM, Telemanagement Forum, and the IPsphere Forum, and is the publisher of Netwatcher, a journal in advanced telecommunications strategy>


Dig Deeper on Open source cloud computing