Mathias Rosenthal - Fotolia

Open source central to Pivotal big data strategy

Pivotal's big data tools are being open sourced, as the company tries to gain traction in a market where transparency and portability are paramount.

Pivotal has made another bet on open source by pulling back the curtain on its big data tools along with partnerships with other industry heavyweights in an attempt to shore up Hadoop.

As part of the latest version of Pivotal's Big Data Suite, the company's core big data tools -- including Pivotal HD, HAWQ, Greenplum Database and GemFire -- will be open sourced this year. The move is one of several Pivotal has made to strengthen its position in the big data market, including feature upgrades and additions to the suite, a new partnership with Hortonworks and the formation of a coalition aimed at standardizing the core of Hadoop.

CS Stars LLC, a Chicago-based provider of risk management software analytics and data services, recently signed up to use the Pivotal Big Data Suite as part of a move from on-premises data aggregation to the cloud. Because the company's client base is diverse and global, it's important to have technology in place that allows it to be flexible, said Paul Marushka, CS Stars president.

"Open source is very important to us," Marushka said. "We think it will drive industry standardization and prevent vendor lock-in by allowing us to work on a variety of platforms."

Another key benefit of open source is its transparency to understand the software quality and minimizing risks that come from being tethered to one proprietary software system, Marushka said.

With big data tools in particular, open source is critical for many consumers, said Nik Rouda, senior analyst for Enterprise Strategy Group, based in Milford, Mass. Hortonworks and Cloudera for Hadoop and MongoDB and Cassandra for NoSQL are some examples of popular big data open source efforts.

"It's not for religion's sake," Rouda said. "It's about being pragmatic about vendors giving visibility and the different ways to do things so they don't get stuck in a dead end."

Last year Pivotal, which was spun out of EMC and VMware in 2013, generated more than $100 million in big data software bookings, with more than $40 million of that through subscriptions to its Big Data Suite, the company said.

If Pivotal wants to expand those sales, it's smart to shake some of the concerns about proprietary labels and embrace open source, while also providing a single platform with a range of services that enterprises can rely on, Rouda said.

"Pivotal is making a play to be that vendor that's trusted enough to have the different add-ons to what they're doing, but isn't so far separate that it's a risk to go with them," Rouda said.

One-stop big data shopping

Several other additions have been included in the latest version of Big Data Suite, including tools for distributed frameworks, a server for key-store and data structure and a message queue for applications. The suite is also now available on Pivotal Cloud Foundry, the company's open platform as a service offering.

In a December survey of more than 350 IT pros by Enterprise Management Associates, based in Boulder, Colo., 60% of respondents used multiple platforms for big data projects, said John Myers, managing research director for the firm. In addition, respondents were split over whether they would prefer a trusted platform from one vendor or an open source tool to build something themselves. This move allows Pivotal to sell to both groups, he added.

"They are giving people the ability to make multiple choices because that's what they're doing already," Myers said. "It's not just one way of doing things."

Big data often isn't driven by traditional IT, with many analytics applications coming from lines of business or data scientists who aren't necessarily thinking about operational requirements. Pivotal also is trying to bring some of that control back to address issues around management, support and future-proofing, Rouda said.

"They're making sure this is a smart decision for the business as a whole and not just a neat piece of technology that solves a clever problem," Rouda said.

Hadoop at its core

In conjunction with open sourcing the Pivotal big data tools, the company said it was among the leaders of the new Open Data Platform initiative, which is aimed at promoting the Apache Hadoop ecosystem and standardizing its core. The other members of the project are GE, Hortonworks, IBM, Infosys, SAS, Capgemini, CenturyLink, EMC, Teradata, Splunk, Verizon and VMware.

The nascent group will spend the next several months working out the bylaws and overall construct of the initiative, as well as working to bring other vendors on board.

Trevor Jones is the news writer for SearchCloudComputing. You can reach him at [email protected].

Dig Deeper on Managed Kubernetes and container services