Cloud computing provider Rackspace is teaming up with the National Aeronautics and Space Administration (NASA) to launch OpenStack, an open source cloud platform that it claims will better serve the growing cloud market than commercial open source companies or proprietary solutions.
We don't believe there is a real second alternative in the open source world today.
Chris Kemp, NASA CTO
The OpenStack project combines Rackspace's cloud storage technology with NASA's virtual server deployment and orchestration engine to make a single distribution that can be deployed anywhere from a single computer to an environment with hundreds of thousands of CPUs. There is no formal partnership between publically traded Rackspace and the civilian federal agency, but each has formidable (and apparently simpatico) cloud computing installations.
One thing Rackspace and NASA are clear on is the non-commercial nature of OpenStack.
"We are both in a very good position. We don't have the inherent conflict of interest some open source companies might have. What we're really trying to do is drive ubiquity and adoption. We aren't going to be in a position where we are holding back people because we want to charge them for it," said Jonathan Bryce, one of the creators of Mosso and a founder of the Rackspace Cloud service.
Rackspace said it had experienced growing pains in trying to make its internal cloud handle the flood of customers, many of whom were already Rackspace hosting customers wanting to switch to the cloud. Bryce said that NASA's own Nebula cloud project paralleled Rackspace's, even down to the problems they faced.
"We ran into the NASA guys about six weeks ago and they'd been through the same experience we had," Bryce said. Scalability and management headaches were the biggest issue.
Solving scalability in the cloud
Both said the issue with most of the cloud platform software out there was that it was usually architected around a central master component, like a database server, that orchestrated and controlled how virtual servers where used on a cluster of servers. On a relatively small scale, these could mimic the operation of a large cloud environment, like Amazon Web Services (AWS), but that didn't translate to tens and hundreds of thousands of nodes.
"We saw there was a focus on functionality, but they didn't meet our needs on a scale front," Bryce said of existing open source cloud and automation technologies.
He said NASA's answer was to distribute the control components more explicitly and make pools of semi-autonomous environments governed by "cluster controllers," loosely coupled by message queuing and linked to "public API servers" that handled incoming requests.
The project documentation compares this control system to the domain name system (DNS) server that governs how Internet traffic is routed. The whole thing, known as the Nova fabric controller, is now in developer preview and will be fully open sourced under the Apache license in the fall.
Why NASA went open source
NASA CTO Chris Kemp said the impetus for releasing Nova as an open source project was that there were no good open source alternatives for researchers or those with heavy-duty functional computing needs. Kemp said NASA found the open source technologies available at the start to be limited.
"We were using Eucalyptus at first," said Kemp, but they began to have issues with scaling the software to really large numbers of nodes without suffering performance problems. Kemp said another issue he faced was strict standards for federal computing that dictated he be able to account for physical location and usage of resources; those, too, were lacking or immature in open source projects or too closely aligned to enterprise, not scientific, computing needs.
"We don't believe there is a real second alternative in the open source world today," he said.
Kemp said that just as Rackspace found his work on the compute side valuable, NASA found what it was looking for in cloud storage in Rackspace's Object Storage technology, which will also be released to the public domain in the fall.
"We have billions of objects, we have exabytes of data out there, and we needed something really different from [standard storage virtualization]," he said.
Kemp is also firm on the need for a cloud computing platform that doesn't have a commercially driven goal. He said that both Rackspace and NASA were sharing the technology because it was incidental to each organization's main goals but valuable to each to empower users.
NASA pursues space research; Rackspace sells commodity servers. Spreading the cloud around doesn't cost them anything, but it might bring them customers and developers looking for a truly non-commercial option.
One thing is certain, said Kemp: "Neither one of us want to be in the cloud software market."
Carl Brooks is the Technology Writer for SearchCloudComputing.com. Contact him at firstname.lastname@example.org.
Dig deeper on Open source cloud computing