This content is part of the Essential Guide: A machine learning and AI guide for enterprises in the cloud
Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Cloud machine learning still an answer in search of a question

Cloud machine learning is still a talking point in IT circles as vendors try to make it more palatable to mainstream enterprises.

A year ago Eric Schmidt, chairman of Google's parent company Alphabet, predicted on stage at Google's first cloud conference that machine learning would be the basis for every major IPO within five years.

It was a bold statement, since few enterprises without data scientists had embraced the technology. It also signaled how Google planned to use artificial intelligence to wedge its way in the public cloud market.

Fast forward a year later and cloud machine learning and the "democratization of AI" was one of the most talked about subjects at the same conference -- on stage and on the convention floor.

"Definitely machine learning is a theme that's interesting to me," said Jeremy Pollock, product manager at Mashery, an API management company. "Every single pitch, if not every single presentation, had some aspect of machine learning."

Like many of the 10,000 or so attendees at the Google Cloud Next conference held in San Francisco earlier this month, Pollock was intrigued by the potential of cloud machine learning services, but still unsure how his business unit can use them -- or whether the attention is simply that IT pros have glommed on to the latest hype.

How would someone come at this fresh or even begin to figure out how to solve a problem with machine learning or ... classify what kinds of problems it could solve?
Rob HarropCEO, Skipjaq

"I'm not quite sure it's 'AI for the masses,'" Pollack said. "I suspect that in practice it takes a lot of thinking about what types of questions you want to answer and what problems you want to solve and whether machine learning is a good fit."

And therein lie some of the hurdles ahead for public cloud providers which continue to place huge bets on the technology. Amazon Web Services (AWS), Google and Microsoft Azure have all rushed to make it easier for customers to use services that could potentially anchor massive amounts of data to their platforms, but questions remain about how easily enterprises can adapt to these methods and whether the public cloud is even the best place to do so.

Enterprises, please

Public cloud providers approach artificial intelligence from multiple angles. Some appeal to companies that want to build complex systems, while others aim to ease enterprises' onramp with packaged software. The latter has been in the spotlight lately, through machine learning algorithm and modeling suites, and APIs for uses such as speech and visual recognition.

Data-driven machine learning is very complicated and it's easy to make mistakes throughout the process, said Vivian Zhang, founder and CTO of the NYC Data Science Academy. Companies may feel incredibly stressed as they approach these techniques, which is where cloud providers come in.

"Cross validation, how they can do modeling, how they can automatically tune the model to reach top performance -- those are top priorities," Zhang said. "That's why I see AWS, Azure and Google moving to make package machine learning tools available to enterprises."

In some ways, public cloud providers are well-suited to abstract much of the underlying work that can take hundreds of hours to train. Each of the three major vendors has years of experience with machine learning through other, more prominent parts of their business, whether it's Amazon's retail business, or Google's search engines, or Microsoft's Office suite and Xbox.

"That kind of machine learning, and that scale and complexity, is always going to be a service that somebody big has to deliver," said Anand Krishnan, executive vice president and general manager of cloud at Canonical, the London-based company behind Ubuntu.

The push into machine learning services is also part of a race by cloud providers to go beyond commodity infrastructure as a service and offer as many services as possible, said Charlie Li, chief cloud officer at Capgemini, a global company with U.S. headquarters in New York.

"It encourages more enterprises to move their workloads to the public cloud," Li said. "So whether it's machine learning or IoT, these just happen to be the latest set of services that people demand, and more and more these services become table stakes."

It's too early to say one vendor is far ahead in this market, and much of the innovation is yet to come. But, unquestionably, machine learning has gained traction, particularly in the media and retail industries that depend on analytics to gain an edge, Krishnan said.

"It's absolutely hot, but it's not going into widespread production in the next three to six months," he said. "It takes time to latch on, but there's potentially much more to come. It was academic two years ago, but it's far more mainstream today."

So you've got machine learning -- now what?

Part of the challenge for these providers as they push machine learning services is that they may have an answer to a question enterprises don't yet ask. Familiarity has improved and vendors have pushed to provide more real-life use cases, but clearly there needs to be more education.

"How would someone come at this fresh or even begin to figure out how to solve a problem with machine learning, or, more importantly, classify what kinds of problems it could solve?" asked Rob Harrop, CEO at Skipjaq, a Richmond, England, startup that built a performance optimization service on top of AWS that incorporates machine learning. "There's a big gap in that they don't know what's possible."

Skipjaq uses machine learning as part of its service, but the company downplays its role in their product because of customer fatigue with the term, Harrop said.

Machine learning products, such as some of those tied to IBM Watson, are intended to solve specific problems, but for the most part, such efforts by public cloud providers are still nascent.

Capgemini customers have started to test simple functionality, mostly using machine learning to automate tasks, such as when to shut down a server or to incorporate Amazon's Alexa into operations, Li said.

Complexity and questions in the public cloud

Public cloud providers have pursued more advanced users even as they court enterprise novices. They've added GPU-powered virtual machines tailored to deep learning and embraced open source projects such as TensorFlow and MXNet. There are also a growing number of startups that build services on top of the public cloud with machine learning baked in.

Qubit, a London-based marketing analytics vendor, moved some of its workloads from AWS to Google to build its own machine learning platform with Dataflow, Pub/Sub and BigQuery. The hardest part of incorporating machine learning models at scale in the cloud is to get the right data pipelines in place, which means taking advantage of higher-level services that represent a huge leap for a company still on legacy systems, said Alex Olivier, technical product manager at Qubit.

"If you look at large enterprise legacy companies, when they talk about cloud they're worrying about lifting and shifting," Olivier said. "It is going to be smaller companies like ourselves who will go and use cloud offering the way they are designed with native APIs."

But depending on the type of services customers use, they may view some prepackaged services as nonstarters. Those tools make it easy to point an API at data or add a bit of AI into existing code for beginners, but they don't go far enough for companies that truly want to change their business model around machine learning, Li said.

"[To do that] you probably need to have a data science team that probably needs the ability to create its own algorithms to tweak and customize and the public cloud providers today still don't have enough robust, full customization for what you need," Li said.

Another major deterrent is cost, at least for those with enough capacity in-house. Machine learning can cost four to five times more in the public cloud than it can on premises, especially when that data is stored in the cloud, Zhang said. That's because of the level of computations needed to train models and because it can take hundreds of hours for models to converge.

Nuance Communications, a Burlington, Mass., company that makes products such as Dragon speech recognition software, has invested heavily in machine learning and deep learning. The company has moved about a quarter of its workloads to Azure and plans to move at least half by the time it's done.

Nuance uses Azure to deliver some of its services that depend on machine learning, but it still keeps the actual processing in its own data centers. Decisions about which workloads reside where often come down to storage requirements, said Joe Petro, senior vice president of engineering at Nuance.

"If you're trying to store and process terabytes of information, the numbers can get pretty extreme pretty quickly and really cause you to pause," Petro said. "But if you've put an algorithm out there and cycle information through it and it's all about network and compute and modest amounts of storage, it can make sense."

Security is a concern, too, especially without traditional firewalls to ensure companies don't use data sets and algorithms as their competition.

Much of the disillusionment with machine learning comes when companies ask about how they get their data to the cloud for processing, Skipjaq's Harrop said. So it's no coincidence that there was so much emphasis on secure data transfers at the Google conference, such as new tools for data preparation and integration and a Data Loss Prevention API to classify and redact sensitive data.

"[Enterprises] get all excited, but for it to work you need a lot of data in the right place and in the right shape and if you do have a lot of data it becomes a target for security breaches," he said.

Despite the drawbacks, providers will likely continue to mature their products as they try to make cloud machine learning more palatable across the board. Most enterprises still do benchmark calculations, but once those services become more stable and predictable, customers will likely bring more workloads to the cloud, Zhang said.

"It's interesting to see a company like Google or Facebook leading the trend of cloud computing and really high-level machine learning AI, but the other side of the market is still on the infant side," she said. "They don't know how to pool data easily and they're still living on Excel sheets.

"Looking ahead the next one to two years, they'll be coming from starting to walk to starting to run," Zhang said.

Trevor Jones is a news writer with SearchCloudComputing and SearchAWS. Contact him at [email protected].

Next Steps

Google rolls out new higher-level cloud services

Get started with serverless computing in the cloud

How machine learning can boost IT productivity

Key criteria for evaluating machine learning platforms

Dig Deeper on Big data, machine learning and AI