When some IT pros think of big data, the idea can seem larger than life. Take those big data infographics, for...
instance, that show googols of storage, quintillions of bytes and billions of users. For some organizations, big data strategies just seem too big to handle.
But developing a successful big data strategy -- including one for cloud -- is far from impossible. IT teams just need to think about big data growth in pieces, which is a sensible approach. A great deal of big data processing will take place outside an enterprise's data center, so it's important to put a clear strategy in place before investing in more data center gear and code.
Baby steps toward big data in the cloud
The first basic step in developing a big data strategy is deciding on an end goal. Because the industry changes so fast, that goal doesn't need to be completely inclusive, but does need to capture the business' broad-brush objectives.
When formulating a strategic direction for a big data project, consider which data should go to the cloud and which should stay in-house. Also consider whether you want to build an analytics team, and, if so, whether you want to rent or write your own code.
Choosing whether to host big data on premises or in the cloud is a major decision for any IT shop. There will be conservative users who want to keep the analytics in-house, while others will be adamant on cloud. And, in this case, it's probably best to go with the latter. The cloud is agile enough to support an enterprise big data project, even as strategies and needs change. If critical data must be kept in-house, then take a hybrid cloud approach, and ensure data security by encrypting both data at rest and in transit, where possible.
Mistakes are bound to be made and it's embarrassing to have to explain the purchase of unnecessary and expensive servers, or the wrong code licenses. Renting compute resources through infrastructure as a service or software as a service is a better option when big data strategies are still somewhat fluid. The cloud is often the more economic choice for big data in the long run, as well.
Finding the low-hanging fruit in big data techniques
Once the strategic direction is settled, it's time to get down to implementation. Most likely, some first-candidate big data projects surfaced during strategy discussions, so it's a case of determining which to do first. Often, the best choice is to aim for something small that can be supported by external resources.
Human resources applications, for example, are good initial candidates for big data in the cloud. Most HR departments are small and many are looking for a makeover. There are also many tools available for HR-related big data projects. In one recent example, Xerox used big data analytics to reduce employee turnover in its call centers by 20 percent.
Marketing is another good candidate for getting started with big data in the cloud. There's a huge amount of data in a marketing department's hands, both from internal sources and, more recently, from external sources such as social media and ad capture systems. But this is a bigger project than HR; it's tougher to separate hype from reality when it comes to advertising and social media, but the potential payback is good. Determine a few areas that will add value to the sales process and tie them down first.
After HR and marketing implementations, extend your big data techniques to other business units. Early use cases for big data in the cloud include reducing production waste to optimize build cycles, predicting demand from customers and tuning up a delivery fleet. There are a lot of projects to choose from.
Take the baby-step approach with each of these new projects. Many users need to be sold on the idea of big data. And remember that success tends to breed more success; an effective project will attract support for future big data efforts.
Big data is really a set of smaller, interlocking projects. In the long term, interconnecting an analytics structure to provide even better insights is clearly an objective -- but the "crawl, walk, run" model is prudent.
Using Hadoop for big data in the cloud
Choosing the best cloud platform for big data needs
Gaining business value through big data