This content is part of the Essential Guide: An OpenStack management guide for the enterprise

Minimize risks, downtime during an OpenStack update

To keep an OpenStack deployment running smoothly and securely, updates and bug fixes are critical. But to perform those updates, IT teams have to do much more than flip a switch.

The OpenStack platform consists of about 30 different modules, each of which has fairly complex features and requirements. The OpenStack development team is also open-source-based, which can result in uneven testing, documentation and code quality.

This makes it essential that IT teams regularly perform an OpenStack update to avoid operational instability. In many ways, it's the same challenge as keeping an operating system like Windows up to date with bug fixes and at the best level of security possible.

However, the difference between Windows and OpenStack is OpenStack is still very much evolving, with some of its modules at much lower maturity levels than others. Major OpenStack releases come every six months, compared with the two- to four-year cycle at Microsoft. And OpenStack's lack of maturity and ever-evolving feature set can complicate minor updates between major releases.

Until recently, the preferred method for performing an OpenStack update was through the command-line interface (CLI), which is fine for an individual server, but inefficient for large clusters of nodes. With large clusters, the chances of errors and update failures increase dramatically.

Best practices for performing an OpenStack update

In an ideal OpenStack update, IT would deactivate all nodes, apply fixes and then reboot the whole configuration -- but this requires a significant amount of downtime. Careful analysis of updates before implementing them can offer an alternative.

Look for updates that don't carry dependencies to other modules and don't materially change stored data structures. Bug fixes are the most common OpenStack updates and can be applied on a rolling basis throughout all nodes.

If an OpenStack update affects cross-module interactions, IT teams need to update both modules together. The problem, however, is any node might interact with any other node. The safest way to apply patches is to quiesce all nodes. However, if the cross-module update involves features that can be turned off, it may be possible to reboot the system safely, even though other nodes are running down-level. Then, when the cluster is updated, turn the feature back on.

In general, it's best to update one OpenStack module at a time, and then determine if the cluster is stable and error-free. Compartmentalization allows for easier debugging when a bug fix goes wrong.

Always get update code packages from known and safe sources. This is true for instance and container images and the apps they contain, as well as OpenStack code. As OpenStack clusters scale out and link to the public cloud, identifying and recovering from a security hack can be time-consuming.

Options for automating an OpenStack update

Lessons from Windows are not lost on the OpenStack community. It makes sense, for example, to automate the OpenStack update process, since this can minimize downtime and risk. Tools to enable this come with some OpenStack distributions.

Red Hat's OpenStack Platform director is one example. This package handles all major upgrades, as well as minor updates. Red Hat tries to avoid feature changes in minor releases that could, for example, affect APIs with cross-module implications.

Other vendors are addressing the automation issue, too. Platform9 automates OpenStack updates and Hewlett Packard Enterprise, Dell Technologies and IBM are also players. Other software vendors include Stratoscale, NephoScale and Mirantis. Given the complexity of manually updating OpenStack through the CLI, admins should use one of these packages to control updates with less risk. They aren't perfect yet, since cross-module dependencies can still complicate the process, but they should help with major upgrades and releases.

Next Steps

Compare Swift vs. Ceph for OpenStack storage

How to customize the OpenStack Horizon dashboard

Overcome OpenStack scalability challenges

Dig Deeper on Building and maintaining a hybrid cloud