An unlikely partnership between two tech heavyweights symbolizes how cloud vendors prioritize machine learning and deep learning for the future of their platforms.
Amazon Web Services (AWS) and Microsoft Azure are the two most popular public cloud providers, with the latter trying to encroach on the former's sizable market lead. But in a surprise move, the pair has put aside their rivalry to create Gluon, an open source deep learning library intended to automate certain processes and make machine learning more approachable to developers.
Both of these companies, as well Google, IBM and others, see huge potential for machine learning in the cloud and deep learning applications built on their respective platforms. But these techniques are predominantly confined to the likes of data scientists, because typical developers lack the skills to build and train the models that underlie these applications.
Cloud providers sell virtual machines based on graphics processing units to attract data scientists to build applications in the cloud, and they offer APIs for prebuilt models for image recognition and natural language processing. However, efforts to find a middle ground that provides some customization, but is not too burdensome for developers, are still in their embryonic stages.
Arnal Dayaratnaanalyst, IDC
"Deep learning isn't for the faint of heart, and it's different than traditional machine learning, with many more different layers in a neural network," said Mike Gualtieri, an analyst with Forrester Research. "What they're trying to do is make that easier where a lot of the things you'd have to know are behind the scenes."
This is not the first time Amazon and Microsoft have partnered. Over the summer, the pair revealed plans to integrate their respective voice assistants, Alexa and Cortana. But this may be the first time they've spearheaded an open source project directly related to a cloud market in which they're vigorously fighting each other.
"[Microsoft and AWS] aim to really get early mover advantage by radically simplifying the learning and training process for a machine learning model," said Arnal Dayaratna, an analyst with IDC. "They see this as an absolutely critical moment in the development of technologies to empower developers to deliver machine learning applications."
Lack of TensorFlow support
Google Cloud Platform is seen as a distant third in the cloud market behind Azure and AWS, but has already embraced machine learning in the cloud as a core tenet to differentiate its platform and tethered its future cloud success to the prospects of big data and artificial intelligence. Google relies heavily on these capabilities internally -- as do Microsoft and Amazon -- and its TensorFlow open source library for machine learning garners much of the early mind share in this space.
Gluon is based on an open source framework developed by Amazon, called Apache MXNet, and will work with the Microsoft Cognitive Toolkit in a future release. Amazon said Gluon eventually will work with other frameworks, but did not provide further details.
That could be a problem for Gluon, particularly as companies such as H20.ai release tools that integrate multiple frameworks, such as TensorFlow and Caffe.
"They have this fantasy that if they ignore TensorFlow, maybe they can create momentum for them and not TensorFlow, which I find mindboggling," said Alexander Linden, an analyst with Gartner. "They are dwarfed by Google, especially when it comes to all the models being developed in TensorFlow."
Still, TensorFlow remains too complex for many developers, so a more digestible machine learning option like Gluon makes sense, despite its early exclusions, Linden said.
Machine learning in the cloud has limitations
Amidst all the attention public cloud providers have placed on machine learning and deep learning, industry observers said it's still not an ideal environment for most of these workloads, especially for more complex workloads. Linden estimated 85% of machine learning workloads run by data scientists are happening on premises, either because of cost, data gravity or skill sets.
"If I just want to get my toes wet, I would of course do this first in the public cloud," Linden said. "But when I become really serious about things and need more compute power on a sustainable basis, then the public cloud becomes too expensive."
Trevor Jones is a senior news writer with SearchCloudComputing and SearchAWS. Contact him at [email protected].
Here's how to beat data science struggles: Azure Machine Learning services
Compare machine learning services from top providers
Take a closer look at Azure's cognitive services portfolio
Databricks simplifies machine learning frameworks