Companies new to the cloud often cite security as a primary concern, but cloud veterans recognize there are other thorny challenges. Managing performance across heterogeneous platforms and controlling costs can trip up even experienced cloud users.
Because fewer businesses question whether they should place corporate information in the public cloud, adoption is growing steadily. Now comes the hard work.
There is a great deal of confusion about what the term cloud management means, and chaos reigns supreme. To manage cloud deployments effectively, IT departments need to understand how cloud systems management differs from traditional systems management, sift through a cornucopia of tools coming from a variety of suppliers and choose the right management platform.
Public cloud has invaded the data center: 79% of enterprises have at least one public cloud service in production, while 35% have more than four, according to Torsten Volk, managing research director at market research firm Enterprise Management Associates (EMA). Cloud adoption is growing at a healthy rate. In 2017, organizations plan to move 19% of their remaining data center applications into the public cloud, Volk said.
As the cloud has become central to business operations, IT concerns have shifted. Controlling public cloud costs became the No. 1 complaint for medium-sized businesses and enterprises in 2017, according to EMA. In many cases, businesses look at their cloud deployments and find waste, unpredictable monthly bills and vendor lock-in, Volk said.
Dealing with management silos
Part of the problem is that different teams across an organization have jurisdiction over different public cloud platforms. In some cases, business units have total control. DevOps groups often manage their own cloud services. IT support is scattershot. Some organizations divide teams by platform, having one group run Amazon Web Services (AWS) deployments and another handle Microsoft Azure. In other companies, the IT staff monitors all cloud applications.
"Cloud management systems have been deployed in a siloed manner," said Mindy Cancila, a research vice president at Gartner. As a result, a hodgepodge of applications, management tools and business processes are strewn across an organization, creating inefficiencies.
Businesses looking for help rectifying this problem can feel overwhelmed. Cloud systems management has become an amorphous phrase. As data center technology evolved, a wide variety of management tools emerged. Different products provide various functions that sometimes overlap and sometimes have no common thread.
Traditionally, system and network management focused on two items: performance and availability. Tools from vendors -- such as BMC Software, CA Technologies, Hewlett-Packard Enterprise, IBM and Microsoft -- monitored how quickly server, network and storage systems processed information. Historical data allowed organizations to identify trouble spots.
Interest in such capabilities has faded. Virtualization and cloud dramatically changed how computer infrastructures process information. Modern workloads are no longer tied to specific hardware; they dynamically move among different systems. Today, traffic patterns vary so greatly that monitoring a physical server's network traffic is of limited value.
Nevertheless, legacy management vendors have tweaked their offerings to provide insight into private and public clouds. Businesses that have used these cloud systems management tools in the past are most interested. The user interface is often consistent, and the customer understands how to interact with the vendor.
In addition, private cloud platforms emerged from suppliers like Abiquo, Flexiant, Embotics and Egenera. These tools focus on specific hardware but gradually have been enhanced to support a wider range of platforms, including public cloud systems.
Public cloud exacerbates management complexity
Public cloud vendors -- such as AWS, Microsoft and Google -- offer other monitoring options. These cloud providers developed tools to manage their own services. Their tools offer migration and monitoring functions but are platform-specific, which increasingly is a significant limitation.
Also, the public cloud has given rise to new approaches to system management. More and more, businesses have to justify cloud systems management costs. The notion that public cloud is less expensive than alternatives has not held true. Businesses have kept loose reins on system pricing, and they often discover that these services cost more than anticipated. In response, financial management tools emerged from suppliers, like Cloudability, Cloudyn and Digital Fuel.
Finally, new products designed specifically for cloud systems management have emerged. Cisco's CloudCenter (formerly CliQr), CloudBolt Software, DivvyCloud, RightScale and Scalr have been at the forefront of that movement.
The rapid pace of change
The various cloud systems management tools promise a broad suite of capabilities. First, technicians spend a lot of time bouncing from screen to screen to manage computing infrastructure. With some products, IT techs work from a single interface of consolidated cost and performance data.
Today, corporations release software at a faster pace than ever before; new deployments are ready with the push of a few buttons.
"One gaming company that we work with made 100 to 125 changes per month a few years ago," said Peter Scott, COO at DivvyCloud. "Now, the company makes thousands of changes per hour."
Cloud systems management tools support governance frameworks designed to decrease the risk of noncompliance -- and possible security holes. Here, management tools ensure that individual systems work with consistent feature sets.
Such functionality appealed to World Wide Technology, a St. Louis-based systems integrator. Each week, 500 developers work with customers that run more than 35,000 systems. At the end of 2015, the company was having trouble managing its cloud software in a coherent fashion. "We wanted to make sure that new workloads were created in a consistent fashion," said Jim Melton, cloud automation practice lead at World Wide Technology. The company considered its options, selected Cisco's CliQr, and now proactively manages new releases.
Cloud systems management tools also enable companies to control costs. In a growing number of cases, DevOps staff will spin up a test system, run the test and leave it running, according to Michael Crandell, CEO of RightScale. These tools identify unnecessary expenses so that firms can take the unused systems down to reduce cloud costs.
As a result of the benefits, corporations are buying these tools in growing numbers. "Interest in cloud management tools is very high right now," Gartner's Cancila said.
Embrace the chaos
Reining in the chaos requires a lot of work. First, companies must get a handle on how many cloud applications they have. Usually, more cloud applications are running than IT expects, so conducting an inventory is a good place to start.
To manage cloud computing in a more cohesive fashion, the IT department needs oversight authority. "We are seeing the CIO step in and take charge of the cloud management selections," DivvyCloud's Scott said.
Once in charge, IT must work collaboratively. The department needs backing from the top to put consistent business processes in place, but purchasing approval often still comes from business leaders and corporate groups, such as the legal and finance departments.
Cloud systems management tools need to keep up with ever-changing features supported by new APIs. For instance, these tools may work with established interfaces, but not new cloud services, such as Amazon Lambda or Azure Functions.
Signs of consolidation
With so many new companies breaking into the cloud management scene over the last few years, the market has been volatile. "The market quickly moved into acquisition mode," Gartner's Cancila said. In March 2016, Cisco made a big splash purchasing CliQr for $260 million. In January 2017, Hewlett-Packard Enterprise purchased Cloud Cruiser.
Further consolidation may occur. VMware's Cross Cloud Services plans to deliver management functionality across vSphere, AWS and IBM Bluemix. EMA's Volk expects Microsoft to enhance its yet-to-be-released AzureStack so it works with third-party systems and cloud services. These two heavy hitters will squeeze other vendors, even the established ones like IBM and Cisco, Volk said.
Corporations are moving to the cloud in growing numbers. Now, they need to develop disciplined management functions for these platforms. Given the dynamic landscape and many options, identifying the right cloud systems management tools is often a struggle.
Take a second look at your cloud infrastructure management tools
Overcome multicloud configuration management challenges
See what hybrid cloud management tools can do for you
- Monitoring and Management with Docker and Containers –Sysdig
- VDI Monitoring and Management Tools –TechTarget
- 7 Requirements for Monitoring Cloud Apps and Infrastructure –New Relic
- How Continuous Monitoring is Revolutionizing Risk Management –BitSight