Big Data is something like a carpenter’s toolbox. All of the fun happens when you start building houses.

Everyone is talking Big Data. But Big Data is as hard to imagine as the web was 20 years ago. Big Data is driven by smart tools, cloud architectures, cheap processing, cheap storage, greater access to statistics and information, and the search for new ways to gain productivity.

The concept of Big Data supports an array of conflicting notions.  A search of “Big Data” results in a range of topics from analytics to large scale data integrations to novel correlations. The Big Data tent includes all of these somewhat contradictory ideas.

That’s because Big Data is really about the tools and techniques that can be applied to vast piles of information. At this point, the subject is something like a carpenter’s toolbox. All of the fun happens when you start building houses.

Here are the broad categories of things that Big Data tools are being applied to:

1.     Big Data means More Data That Fits 

Analytics are nothing new. The eighties saw data begin to drive processes as organizations embraced Total Quality and Statistical Process Control. The first wave of the information age saw the democratization of chart making and lots of stuff turned into pie charts and bar charts. The underlying sources of information always had problems making custom reports.

To this day, people spend tons of time assembling unique reports by hand. The tools of Big Data include the ability to develop repeatable processes from disparate databases that live under your roof. From this perspective, Big Data is more of the same old number crunching. It’s just faster, easier, and repeatable.

2.     Big Data Means Consuming The Social Graph and More Data that Doesn’t Fit

Some questions have remained impossible to imagine answering. Collecting and analyzing the data required to actually manage a market based employment branding campaign on a job by job basis of 5,000 jobs (which is how you ought to do it) was beyond comprehension. Now it’s possible for a non-programmer to assemble the data in a way that leads to process automation.

Huge chunks of Data that are bigger than the storage capacities of old fashioned enterprise tools are popping out of the woodwork. As employers increasingly find it important to own a copy (or two) of all employees’ social data and be able to digest it, questions that involve cross-referencing in-house data with external data are becoming normal. With no database actually capable of holding all the data, organizations will use Big Data tools to have large chunks of data synch up.

This is particularly true in the Recruiting operation where it is useful to have (or have access to) massive data on the labor market and the candidate pool.

3.    Big Data Means Integrated Data, All Data Singing Like a Choir 

The holy grail in legacy Enterprise software solutions is a single integrated database that has all of the silos within the organization talking to and learning from each other. Part of the reason that HR has failed to demonstrate its alignment with business goals is that the holy grail never materialized. Making correlations across the organization has always required the involvement of the IT Department.

With Tools that behave like intelligent overlays, rich internal data sets can be combined, compared and contrasted to unearth gems of insight about what really drives performance both for humans and for the enterprise itself. Big Data solves the historic data logistics bottleneck and makes it possible for new questions to emerge.

4.    Big Data means Dis-Integrated Data, Pulling Pieces to Answer Your Question 

At the very same time, one-off questions can be unearthed by pulling micro chunks of data to focus on very specific situations. Big Data includes tools for effective navigation of huge data sets individually, or in combination. You can see the impact of a decision on all of sales, or on offices with less than three employees who all have an average of more than 10 years experience.

This is where much of the power of Big Data really exists. The discovery of high-leverage, unique opportunities to increase organizational performance is how major gains are tweaked out of functioning systems.

5.     Big Data Means More Insight Rediscovering the Obvious 

Expect a lot of grumbling as people get accustomed to using these new tools. Unearthing brilliant discoveries like people who live in the rain belt get to work later on rainy days than people who live in the Banana belt. Cross referencing external databases with internal data will produce some amazing stories.

6.    Big Data Means More Insight From the Tiny 

The Butterfly Effectis often used to explain Chaos Theory. In the Butterfly Effect, A Butterfly causes a hurricane. Small variations in inputs can cause large variations in outputs. 20th Century computing architecture was designed to save the very resources that are now in abundance. As a result, computing systems usually focus on the meaning of large trends in databases rather than tiny increments.

A great example of this is web server statistics analysis. While most contemporary tools focus on big chunks of traffic, the answer is sometimes a tiny thing. While it’s interesting to know that 15,000 of your website visitors came from Botswana, it’s probably more important to know that Bill Gates visited. Contemporary systems can’t do this but Big Data makes it possible.

7.    Big Data Means New Correlations That Drive Performance Lots of questions never get asked because it seems impossible to answer them. Big Data will create many opportunities to understand the connections between things that don’t seem to be connected. Think local gas price fluctuations, attendance of sophomore year classes and graduating GPA.

Where the information age brought standardization and proceduralization to all sorts of jobs, the next step is to restore curiosity. The possibilities of Big Data are mostly limited by the curiosity of the people who are handling it. We currently don’t have good tools and techniques for developing curiosity in people who have had transactional jobs.

In the coming years, much of the HR workforce will need training to make the shift from transaction processing to the development of good questions. It’s not an easy shift.

If you want to get up to speed on Big Data, here’s a good starting list.

• The “Big Five” IT trends of the next half decade: Mobile, social, cloud, consumerization, and big data
• Big Data as defined by the folks who coined the term and run the conference
• Data Are the Next State of the Economy
• Planning for Big Data: a CIO’s Guide (Free eBook from OReilly)
• Dion Hinchcliffe’s Archives
• Dwayne Lay’s Why the WSJ is Dead Wrong About Moneyball
• Maren Hogan’s Moneyball and the Blind Leading the Blind and The Implications of Big Data
• Josh Bersin’s What The Splunk IPO Says About Big Data