Essential Guide

Guide to big data analytics tools, trends and best practices

A comprehensive collection of articles, videos and more, hand-picked by our editors

Hadoop

Hadoop is a free, Java-based programming framework that supports the processing of large data sets in a distributed computing environment. It is part of the Apache project sponsored by the Apache Software Foundation.

Hadoop is a free, Java-based programming framework that supports the processing of large data sets in a distributed computing environment. It is part of the Apache project sponsored by the Apache Software Foundation.

Hadoop makes it possible to run applications on systems with thousands of nodes involving thousands of terabytes. Its distributed file system facilitates rapid data transfer rates among nodes and allows the system to continue operating uninterrupted in case of a node failure. This approach lowers the risk of catastrophic system failure, even if a significant number of nodes become inoperative.

Hadoop was inspired by Google's MapReduce, a software framework in which an application is broken down into numerous small parts. Any of these parts (also called fragments or blocks) can be run on any node in the cluster. Doug Cutting, Hadoop's creator, named the framework after his child's stuffed toy elephant. The current Apache Hadoop ecosystem consists of the Hadoop kernel, MapReduce, the Hadoop distributed file system (HDFS) and a number of related projects such as Apache Hive, HBase and Zookeeper.

The Hadoop framework is used by major players including Google, Yahoo and IBM, largely for applications involving search engines and advertising. The preferred operating systems are Windows and Linux but Hadoop can also work with BSD and OS X.

This was first published in August 2010

Continue Reading About Hadoop

PRO+

Content

Find more PRO+ content and other member only offers, here.

Essential Guide

An architect's guide: How to use big data

5 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

-ADS BY GOOGLE

File Extensions and File Formats

Powered by:

SearchServerVirtualization

SearchVMware

SearchVirtualDesktop

  • Virtual desktop security guide

    To secure virtual desktops, consider antivirus, certificates and network vulnerabilities. Just remember, VDI doesn't always ...

  • Guide to low-cost desktop virtualization

    In this guide, learn to virtualize desktops without spending more than you would when deploying PCs, and what VDI vendors are ...

  • VDI pilot project guide

    A VDI pilot project should start with a VDI project plan. Know what pitfalls to avoid and test product options to achieve a ...

SearchAWS

SearchDataCenter

SearchWindowsServer

SearchSOA

SearchCRM

Close