AWS re:Invent 2020 is a marathon, not a sprint.
In the past, the one-week format of AWS' annual user conference often felt overstuffed and overwhelming. But the steady drip of this year's virtual show presents different challenges for attendees. Without the all-consuming focus of being there in-person, IT pros and industry observers are balancing their re:Invent participation with all the other concerns of their day-to-day lives.
While the first week's re:Invent keynotes centered on AWS' overall strategy, the second week of re:Invent featured the third and fourth keynotes focused on AI and a look behind the curtain into how AWS works, respectively. For those that were too busy with work, family or all the rest of the craziness that is 2020, here's a recap of those talks and the services announced in week two.
Keynotes hit on AI, redundancy and sustainability
Tuesday's keynote was machine learning's first dedicated slot at re:Invent, which makes sense given its outsized role in other keynotes in recent years.
And while the session put AI in the spotlight, there weren't any major divergences from the existing suite of services. Like most of the dozens of services and feature updates announced over the first two weeks, the focus was really about iterating on what's already available on AWS.
AWS' machine learning platform Amazon SageMaker got the most attention, with new features intended to simplify and accelerate the building and deployment of machine learning models. SageMaker is a great example of AWS' -- not always successful -- strategy of releasing services quickly. The idea is that even if they don't have exceptional functionality from the start, it's important to be fast, then solicit user feedback to upgrade as needed. First announced at re:Invent 2017, AWS has steadily expanded the tooling of the platform, including nine new features this year.
AWS, like cloud rivals Microsoft and Google, has invested heavily in AI capabilities. The cloud's scalability and varying levels of abstraction make it a logical fit for this growing field. It also represents a potential gold mine, because machine learning requires massive amounts of compute and storage.
And the AI enhancements weren't restricted to SageMaker. Amazon Redshift and Amazon EMR, two of AWS' older services, now have capabilities for building machine learning models and applications. Redshift saw several other updates, which is notable given the formidable competition AWS now faces in the cloud data warehousing market.
The second keynote addressed AWS' own processes -- its data center architectures and sustainability efforts. In the past, this talk has typically been held the first full night of re:Invent. It lays the groundwork for a conference that constantly reinforces the massive scale of the platform and the absurd lengths its engineers go to keep the power on for users. It also gives executives a chance to take not-so-subtle jabs at competitors for not matching their efforts -- at least in the areas AWS spotlights.
The setup may have been different this year, but the delivery was much the same. Peter DeSantis, senior vice president of AWS global infrastructure and customer support, discussed the "everything fails" mindset that has led Amazon to data center redundancies that include duplicate power lines that are independent down to the server rack, multiple backup generators for each power supply, and replicated, custom switchgears for when the power fails.
The focus is on reducing the blast radius and complexity, which has resulted in a 99.99997% uptime for the data centers that use this model, DeSantis said.
The keynote also hit on AWS' sustainability efforts. Those smaller switchgears, known as an uninterrupted power supply, or UPS, rely on different, more efficient batteries than typical UPS systems use. This and other hardware initiatives DeSantis highlighted dovetail with Amazon's pledge to use 100% renewable energy by 2030.
This year, Amazon procured 4.1 GW of new wind and solar farms and increased its renewable energy procurement by 300% over projects announced the previous year, DeSantis said. Amazon now projects it will meet its carbon pledge by 2025.
And finally, though not part of a keynote, another big strategic announcement came this week from Teresa Carlson, vice president, AWS worldwide public sector. By 2025, AWS will provide free cloud computing skills training to 29 million people, she said.
"We have to make sure everyone around the world has access to those training programs, no matter what social status they are, economic background or educational level," Carlson said.
There wasn't any formal announcement beyond Carlson's session. However, she did say this initiative will be an expansion of the existing AWS Educate program, and there will be some degree of coordination with the World Economic Forum.
Service and feature updates
The second week of re:Invent didn't have quite the onslaught of product announcements as the first week, but there were still more than two dozen new service or feature updates that were highlighted. Here's a quick recap of what you need to know. As is increasingly the case with re:Invent rollouts, many of these services are in preview, so they might not be available for you to use until later in 2021.
- Amazon SageMaker Data Wrangler is intended to dramatically reduce the time it takes to combine and prepare data for machine learning by eliminating some of the previously required coding steps. A user selects a data source and SageMaker transforms the data so it's ready for model training.
- Amazon SageMaker distributed training reduces the time to train deep learning models and data sets by extending those workloads across multiple GPU instances. Organizations will need to assess if the time savings is worth the cost of running more instances.
- Amazon SageMaker Clarify aims to reduce bias in machine learning models and add transparency. As our story on this feature discussed, it's certainly a timely rollout. It will be worth watching to see how successful it is in addressing a serious need in the industry.
- Amazon SageMaker JumpStart provides prebuilt, customizable CloudFormation templates so organizations new to machine learning can use SageMaker for common use cases.
- Amazon SageMaker Feature Store is a repository service intended to maintain consistency during real-time inferences.
- Amazon SageMaker Edge Manager is intended to address the security, performance and management of models that rely on IoT and other edge devices.
- Machine learning capabilities were added to graph database service Amazon Neptune to assist with predictions.
- With Redshift ML, users can now create, train and deploy machine learning models in the data warehousing service via SQL commands. This is a bit of catch-up for AWS. Microsoft announced a similar capability this year with Azure SQL Managed Instances, while Google BigQuery ML was first announced in 2018.
- Some longtime AWS observers may still associate Amazon Elastic MapReduce (EMR) with Hadoop, but Amazon EMR Studio is an IDE for data scientists and engineers to build apps written in R, Python, Scala or PySpark, using the Amazon EMR runtime for Apache Spark.
- Amazon Kendra, a machine learning-backed search service, can now adapt to how users interact with search results to provide more relevant results to their queries. It also added support for Google Drive.
- Amazon Lookout for Metrics is an anomaly detection service for time series data. It uses machine learning and can connect with AWS data stores and select third-party SaaS apps. Two other Lookout offerings were announced last week -- for Equipment and for Vision. While those services are tailored to factory production, Lookout for Metrics is for applications with heavy end user interaction, such as retail, gaming and marketing.
- Other Redshift updates include support for federated querying for MySQL in Aurora and Amazon Relational Database Service, self-tuning capabilities, compute-optimized nodes, and support for native JSON and semi-structured data processing. Users can also now share data across clusters and move clusters to other Availability Zones.
- Amazon Braket, a quantum computing service, added support for the PennyLane open source framework and increased the number of qubits its circuit simulators can support, from 34 to 50.
- Amazon HealthLake is, as the name implies, a data lake service for organizations that work in healthcare. AWS has slowly pushed into more industry verticals in recent years, and this HIPAA-eligible service is the latest example of that.
- AWS Audit Manager is intended for organizations in highly regulated industries. It continuously audits data to assess risk and compliance.
- For Amazon EC2, AWS added more network performance metrics, as well as G4ad instances, which are designed for graphics processing.
- And in networking updates, AWS added Amazon VPC Reachability Analyzer to test connections between resources. In addition, custom routing was added to Global Accelerator, while AWS Transit Gateway Connect is intended to improve the integration process with SD-WAN appliances.