This content is part of the Essential Guide: The right systems management tool frees up time for the important work
Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Chef and Puppet lead the charge in open source cloud automation

IT pros increasingly turn to Chef and Puppet for open source cloud automation and orchestration. But other options, such as TOSCA, are also worth exploring.

Deploying software isn't easy, and it's more complicated in the age of service-oriented architectures, web services, microservices and the cloud. Virtualization presents new challenges, with applications and servers in new roles. Basic software operations tools are transforming into automation and orchestration tools, and open source software leads the way -- especially Chef and Puppet.

The software tools used today to deploy applications derive from early tools that simplified deployment by allowing developers to build operations guidelines into the applications themselves. This concept, called DevOps, embraces the goal of accurate and speedy deployment, but the varying approaches to getting there quickly prove divisive.

The Chef and Puppet divide

The simplest way to automate deployment is through scripts. Deploying software manually means issuing commands using the operating system or cloud management system's tools, and scripts are a flexible way to write the commands as a file that can be quickly executed. Two popular open source automation tools are Chef and Puppet.

Chef and Puppet seem to be growing closer together, and some new DevOps tools are straddling the declarative/ imperative line.

The Chef model is considered imperative or prescriptive because it's procedural -- it explicitly describes how to deploy and connect cloud application components. Chef creates "recipes" and "cookbooks" that are built in a programmer-friendly way. Each deployment step can be described independently, and bringing those independent recipes together creates a repeatable application deployment process. Since every operations step can be described in a recipe, there is nothing that can be deployed manually that Chef can't automate.

That same programmer-friendly model is a negative for some users. To them, the logical way to deploy an application is to describe in a model what the end-state of the deployment would look like. The open source DevOps tool would then process the models to create that state. This process, often called declarative or intent-based DevOps, is the basis for Puppet, another popular tool.

If Chef is programmer-friendly, Puppet's origins lie with operations people, who are more familiar with describing what they want than the detailed steps in getting there. Like a high-level programming language that makes programming simple, Puppet's models can simplify deployment descriptions. Furthermore, because an end-state approach can be used to describe any step in application lifecycle management (ALM), Puppet can naturally accommodate a complete ALM function, which Puppet fans like.

The challenge with Puppet is that you can deploy only that which can be modeled. As new challenges in deployment arise, the model-building process begins to resemble imperative script development, and, in fact, cloud evolution is beginning to bridge the divide. Chef and Puppet seem to be growing closer together, and some new DevOps tools are straddling the declarative/imperative line. However, the leading candidates for new deployment and configuration management all support a declarative model; none are purely imperative.

Options in cloud

The cloud is also forcing an evolution of deployment and configuration management tools. With cloud deployment dealing with two levels of virtualization, any deployment and configuration automation becomes more complicated, and DevOps is forced to confront both deployment and lifecycle management. This is reflected in a new cloud term: orchestration. This reflects the need for arranging complex steps into a unified composition.

Orchestration bridges the declarative and imperative models, but declarative orchestration appears to be the favored approach; describing the current and desired state of something is a good way to approach lifecycle management. Two new purely declarative alternatives -- CFEngine and Juju -- continue to gain popularity alongside Chef and Puppet. These tools emphasize model-building and provide policy and library support. In addition, tools that support both declarative and imperative orchestration -- Ansible and SaltStack -- now challenge the polarized approach of the past.

The tools mentioned so far have evolved from DevOps and traditional deployment. What about something specifically for the cloud? The international standards group Organization for the Advancement of Structured Information Standards developed a declarative-model approach called Topology and Orchestration Specification for Cloud Applications. TOSCA addresses both the definition of end-state application deployment and the specific modeling of virtual resources and pools. TOSCA also integrates management definitions to support lifecycle automation.

What's the best cloud orchestration approach for you? Here are some tips:

  • Tools that support the imperative model, including Chef, are the most powerful and support virtually any kind of application on any kind of cloud. But they're harder for nonprogrammers to learn.
  • If you plan to rely mostly on packaged applications in the cloud, check to see if the vendor has a preferred orchestration approach. If so, give serious consideration to that approach for your applications.
  • TOSCA is the cloud's own model, but it's also the least mature. If you can't make it work in the present, consider declarative cloud orchestration tools; the transition to TOSCA will be easier.
  • If you are already familiar with a DevOps tool for data center deployments, don't toss it aside just because you're moving to the cloud. All DevOps tools are becoming more cloud-friendly.

Varieties of open source packages -- and even some commercial ones -- support TOSCA, even though it is a relative newcomer to orchestration. Some popular cloud orchestration tools are based on TOSCA, including Cloudify, OpenStack Heat, Alien4Cloud, as well as ARIA and OpenTOSCA, which are both reference implementations of the specification. TOSCA is also being embraced for network functions virtualization and software-defined networking. Many would argue its success in these areas means it's the future of orchestration.

Orchestration is the key to the future of applications. It's a form of DevOps that can simultaneously cope with agile applications and agile virtual resources. No matter what your approach to orchestration, review and revise the details as both applications and virtualization evolve.

Next Steps

Build an orchestration strategy for cloud apps

Learn the difference between DevOps and orchestration

What to avoid when employing DevOps

Dig Deeper on Cloud automation and orchestration