pressmaster - Fotolia
Managing data is a vital process for any IT professional, but keeping up with the latest innovations in database technology is a challenge. One major change is Google ditching MapReduce for data analytics. "We seem to be moving away from MapReduce … it's very primitive and very batch-oriented in terms of how it's architected, and it typically has to be paired with something else," said David Linthicum.
Linthicum reminisces about the past of database technology in addition to discussing the recent changes with Mike Olson, chief strategy officer at Cloudera. Other issues for discussion include:
1. What has the new generation of database technologies led to?
"Some of the work that Google did in the early 2000s on large-scale storage and large-scale data processing morphed into an open source project called Hadoop. It has really been the basis for the movement toward big data across the industry," Olson said.
Both Linthicum and Olson agree that data management is an evolving science. What impact have predecessors had on database technology? (1:20-5:00)
2. Linthicum questions whether a new trend is to use someone else's hardware and move it into the cloud.
"We see most new companies building out their infrastructure in the cloud," Olson said. "No matter whether you are doing it internally on your own hardware or in the public cloud, you want stuff like elasticity and virtualization, so cloudy concepts belong in both those places." (5:01-6:55)
3. What's new at Cloudera and what can we expect? Olson talks about Cloudera adding Apache Spark. Is Apache Spark the successor to MapReduce? With Google shifting away from MapReduce, what's next? Olson compared the shift from MapReduce to the automobile evolution. "You can sort of think of this MapReduce to spark transition as manual transmission to automatic transmission," Olson said. (7:00-15:30)
4. With many different open standard options available for cloud, there is battling within the industry. What is the impact on cloud customers? Is there an end to the fighting in sight? Olson doesn't think so. "Because the stakes are so high and because there is so much budget available, the violence is going to go on for a while," Olson said. How can customers find the best open source providers? (16:00-24:30)
Analyzing the growing open source market
Who comes out on top in the cloud competition?
Major providers battle for Hadoop as a service space
Dig deeper on Big data and cloud business intelligence