Assessing cloud infrastructure provider performance in 2015

Evaluating a cloud provider's reliability isn't easy, but there are resources and tools that can help. Before kicking off the New Year, consider how your provider performed in 2015.

With the end of 2015 just around the corner, it's the perfect time to haul out some metrics and measure your achievements this year. For those using a cloud infrastructure provider, it's a good time to evaluate their reliability and see if it's worth sticking with them in 2016.

Assessing the reliability of cloud providers, even the biggest ones, can be difficult because independent data sources are few and far between. Fortunately, there are some options.

For example, data from CloudHarmony, a company based in Laguna Beach, Calif. that monitors cloud infrastructure provider uptime and was acquired by Gartner in 2015, provides insight into the performance of four leading cloud providers: Amazon Web Services (AWS), Microsoft Azure, Google and Rackspace.

Focused primarily on cloud provider outages, CloudHarmony's site displays rolling analytics for dozens of cloud vendors, with up to 30 days displayed at a time. According to data from early December, here's how those "big four" providers stack up:

  • For compute, there was a wide reliability range. According to CloudHarmony, Rackspace was the only company of the four to log 100 percent availability over the 30-day period.  Google Compute came close, with only several seconds of downtime over the same period. Amazon Elastic Compute Cloud (EC2) logged about two minutes of downtime. Microsoft Azure had a total of more than two and a half hours of downtime.
  • For storage, results were similar. Amazon Simple Storage Service (S3), Google Cloud Storage and Rackspace Cloud Files all logged 100 percent availability. Microsoft Azure Cloud Storage, however, was at 99.952 percent with more than 27 minutes of downtime.

While these 30-day assessments can provide a snapshot into a cloud provider's performance, it's also important to consider the outages or downtime they had throughout the rest of the year.

Other perspectives on cloud provider reliability   

With a slightly different perspective, Nasuni, an enterprise storage company based in Natick, Mass. that uses cloud providers, releases periodic reports on cloud performance. The company offered stats earlier this year as part of its biennial State of Cloud Storage Report, which tested the write, read and delete speeds, as well as the availability and scalability of public cloud storage providers.

It's clear Microsoft Azure Blob Storage and Amazon S3 are here to stay.
Andres RodriguezCEO of Nasuni

This year, Nasuni narrowed its test to three cloud storage services: Microsoft Azure's Blob Storage, Amazon S3 and Google Cloud Storage.

Microsoft Azure's Blob Storage came out on top, with Amazon S3 a very close second. Google Cloud Storage lagged behind the two. Each of the three providers was tested for availability based on response time to a single write, read and delete process at 60-second intervals over a 30-day period. Amazon and Microsoft were nearly tied in the availability metric, with Amazon's average response time of 0.1 seconds barely edging out Microsoft's 0.14 seconds. Google was about five times slower than the others. In addition, Amazon's response time was significantly more consistent.

"We've always said that public cloud storage is a commodity play similar to the disk drive market, and that only two or three players would ultimately survive and thrive," said Andres Rodriguez, CEO of Nasuni. "It's clear Microsoft Azure Blob Storage and Amazon S3 are here to stay -- their performance, reliability and scalability has not only significantly improved over the five years we've been testing, but they're far outpacing everyone else."

However, Google Cloud Storage shows promise, and may yet catch up to Amazon and Azure, he continued. "They have a ways to go, but unlike many others, Google not only has more than 15 years of experience running its own cloud for its own business, but it's also got vast financial and technical resources to invest," said Rodriguez.

Other events in 2015 showed the big cloud infrastructure providers maintaining generally high -- but not perfect -- records.

While there were multiple AWS service outages in 2015, most customers didn't see a significant impact on their applications or end users,  said Jason McKay, Chief Technology Officer at Logicworks, an AWS Premier and Managed Services Partner based in New York.

"We had one incident with the DynamoDB outage in September where some of our customers could not reach the AWS console, but there was zero impact from this incident or any other incident on customer applications," McKay said.

Furthermore, AWS has provided tools that enable large businesses to span multiple geographic zones for virtually every service. Therefore, with the proper automation and orchestration in place, businesses can maintain uptime, even if AWS were to experience disruption, McKay continued.

And, as the cloud ecosystem continues to evolve, traditional metrics such as uptime and availability are "table stakes," said Duncan McGregor, vice president, engineering and operations at Cogeco Peer 1, an IT and network services provider based in Montreal.

"In today's environment, it is essential and assumed that your chosen provider will offer compelling reliability standards," McGregor said. "Simply put, and similar to other trends we see across the industry, today's cloud metrics have moved beyond historic reporting and have the potential to assist organizations in making data driven decisions to maximize performance and efficiency."

Next Steps

Four ways to boost cloud network performance

Key considerations for selecting a cloud provider

Cloud vendors, users collaborate to minimize outage disruption

Dig Deeper on Public cloud and other cloud deployment models