This content is part of the Essential Guide: Essential guide to cloud management

Cloud cost management refines, but plenty of room for improvement

Managing cloud costs offers different challenges than on-premises budgeting. While users and providers are getting better, it's still a maturing space.

Controlling costs and deciphering IT spending behavior with cloud computing has come a long way, but there's still plenty of room for improvement and lingering questions about who to trust to help manage spending.

The pay-as-you-go pricing model of public cloud represents a significant shift for IT shops, and it requires a higher degree of vigilance to control cost. Providers and users are more nuanced in their understanding of cloud cost management, but it's still, in many ways, a maturing component of cloud computing.

Sticker shock was common in early cloud consumption, and oversized bills remain an issue, particularly for those newer to a platform. Sometimes, it's hidden costs; however, as IT pros are quicker to take advantage of the myriad services available from a provider, they also may neglect to take into account just how much they ultimately consume.

IT departments traditionally have had tight control over spending, but cloud is so flexible and on-demand that users can get their hands on it without first needing IT's approval. This runs counter to how many IT departments think about paying and allocating resources, so often the adjustment is as much about a mindset shift as it is about the technology, said Owen Rogers, research director at 451 Research.

"It's almost as if the technology has moved really fast, but the actual practicality of implementing it is still an issue," Rogers said.

Rogers, who leads 451's digital economics unit, developed the Cloud Price Index in part to help customers better understand the true cost of running workloads in various public clouds. In the early years of the market, cloud providers didn't address enterprise demands to have deeper insight into who consumes resources and how much various departments spend, but that is improving, he said.

Providers still must do a better job of showback and chargeback to tag resources to particular users, as well as provide more automation to control consumption and approvals, Rogers said.

Third-party versus native: Whom do you trust?

When it comes to using market leader Amazon Web Services (AWS), cloud cost management is usually not difficult -- when starting with a small environment, said Brett Gillett, AWS practice lead at Softchoice, an IT consulting firm based in Toronto. Problems arise as environments expand and become more complex, particularly when customers start using consolidated billing to link their various AWS accounts.

"It's like a double-edged sword," Gillett said. "AWS is a very transparent business, so they will tell you everything going on in your account -- and that's fantastic, but how do you manage all that information you're getting?"

How well customers handle their billing often has do with their internal policies and procedures, and one of the best ways to manage costs is to properly tag resources, so the bill can be dissected for greater visibility as needed, he added.

Billing on AWS was "horrific" up until a year or two ago, with the only way to estimate costs being through third parties, said Chris Moyer, vice president of technology for ACI Information Group and a TechTarget contributor. Even with the AWS Cost Explorer analytics tool, it may not be clear why costs might spike in a given month, so customers have to use their own analytics as well.

"It's great to have tools like these to analyze your bill in detail and estimate upcoming costs, but for anything specific to a company, the best bet will still be third-party tools," Moyer said.

Microsoft recently released an open source billing portal that provides insights across multiple Azure subscriptions. It's yet another example of cloud providers trying to provide greater visibility into consumers' usage, but it also highlights a constant debate in public cloud about whether to rely solely on native tooling or to turn to third-party options.

YSTA Services, a Fort Worth, Texas, company that owns several smaller services, uses Azure for disaster recovery and business insights. It's seen marked improvement in reporting since switching to Azure Resource Manager, but it still relies on a cost monitor from Spiceworks, said Brett Gilbert, YSTA Services' CTO.

Gilbert hasn't used the new Azure billing portal yet, but he said it can be difficult in Azure to compare costs with previous weeks or months.

"There are a lot of variables involved that can increase your costs, and you might not be overly aware of [them]," Gilbert said. "It becomes quite complex when you have an environment in Azure and the number of components we're using."

Azure still needs to improve its overall cost analysis functionality with its native tooling, but having a third-party tool is helpful, regardless of what is done to shore up the platform, Gilbert said.

"It's nice to have a third party outside that has an unbiased view into it that's strictly reporting on the metrics," he said.

Dozens of third-party vendors offer cloud cost management and monitoring tools, including Cloud Cruiser, Cloudability and Cloudyn. Those services can be helpful for monitoring workloads on multiple platforms, especially since it can be difficult to weed through the different billing metrics and discounts the various cloud providers offer. There's also extra complexity tacked on when taking into account performance, data transfer fees and the cost of the ever-expanding list of higher-level services available on these platforms.

Every core technology in the data center -- from client server to database to virtualization -- once it really starts to spike, the next obvious question is, 'How do I reduce cost?' It's incumbent upon platform providers to answer that question.
Dave Bartolettiprincipal analyst, Forrester Research

Customers want to see usage and billing information presented easily with multiple reports so they can quickly understand, and the preferred option is to get it directly from the provider, said Dave Bartoletti, principal analyst at Forrester Research.

There's no harm in trying one of these third-party tools because they're relatively inexpensive to start with, but those providers are always going to struggle to optimize across clouds compared to what the cloud providers could potentially offer natively, Bartoletti said.

It may sound counterintuitive, but cloud vendors often have a vested interest in providing tools to help customers save money on their platforms. Most customers aren't necessarily looking to reduce their overall cloud spending, so it's imperative these providers offer as much transparency as possible with pricing so users feel comfortable coming back for more, Bartoletti said.

"They know full well [their] cloud bill is going up because they're using more and more cloud every year," he said. "They just want to make sure it's going up reasonably."

Room for improvement in cloud billing

Of course, the providers themselves have to get better as demand grows, Bartoletti added.

"Every core technology in the data center -- from client server to database to virtualization -- once it really starts to spike, the next obvious question is, 'How do I reduce cost?'" Bartoletti said. "It's incumbent upon platform providers to answer that question."

Customers still want help in forecasting future usage so they can have better capacity management, Bartoletti said. Capacity management used to be more static and was typically something IT shops did a couple times a year to align the budgeting with server and network capacity, but that is changing, and enterprises are going to need help as they move from forward-looking budgeting to more challenging quota-based and project-based budgeting.

"Most IT teams are not cost management experts and never had to be," Bartoletti said. "The cloud is forcing large enterprise purchasing vendor management teams to be more closely aligned with project teams, product teams and to work toward more continuous budgeting."

Trevor Jones is a news writer with TechTarget's data center and virtualization media group. Contact him at [email protected].

Next Steps

Explore leading AWS cost management tools

How to manage cloud costs when working with multiple vendors

Optimize hybrid cloud with resource and cost management strategies

Dig Deeper on High availability and disaster recovery