Over the last couple of weeks I have been very interested in the growth of Big Data. A few years ago Big Data was primarily found in academic and marketing writing, ie not in the main stream. This has changed with many commentators now discussing the merits that this new frontier has to offer.
For those not up to date what is Big Data?:
big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, curation, storage, search, sharing, analysis, and visualization.
In other words think the Human Genome program or where I have seen more mainstream commentary, analyzing the status updates from services such as Twitter or Facebook. There are lots of reasons why Big Data is important and understanding how to use it and leverage it will become critical for business success. The biggest issue we face that Big Data will help solve is the vast amount of data points we are generating through social networks, trends such as cloud computing and The Internet of Things.
While much of the discussion around Big Data is consumer based there have been a number of notable discussions about the use of Big Data inside the enterprise and unsurprisingly these discussions include the impact of Big Data on HR and how we can now tie employee data to other large datasets for predictive modelling and recruitment.
In her paper from April 2010, Privacy and Publicity in the Context of Big Data, danah boyd raises a number of pertinent points that I think deserve more thought an discussion in terms of their impact on business.
danah’s key points:
- Bigger Data are Not Always Better Data
- Not All Data are Created Equal
- What and Why are Different Questions
- Be Careful of Your Interpretations
- Just Because It is Accessible Doesn’t Mean Using It is Ethical
Each on of the above points have tremendous influence on how successful Big Data will be when used inside an organisation but I want to touch on two of her points that struck a chord with me. (However I would strongly suggest you go read her whole paper.)
danah’s first point of Bigger Data are Not Always Better Data, “Big Data is exciting, but quality matters more than quantity. And to know the quality, you must know the limits of your data.” At a basic level just because you can review all of the tweets and connections of your employees or candidates does not mean you have all of the information about these people as they might have different accounts under different pseudonyms some might be protected others not. Just because you have access to millions of datapoints does not mean you have the right data points.
danah’s final point is the one that deserves the most thought. Just because data is accessible doesn’t mean that using it is ethical. Just because a candidate or an employee tweets or puts a status update on Facebook should we really use that data in our analysis? To quote danah:
To get here, we’ve perverted “public” to mean “accessible by anyone under any conditions at any time and for any purpose.” We’ve stripped content out of context, labeled it data, and justified our actions by the fact that we had access to it in the first place. Alarm bells should be ringing because the cavalier attitudes around accessibility and Big Data raise serious ethical issues. What’s at stake is not whether or not something is possible, but what the unintended consequences of doing something are.
From here danah goes on to look at the concept of privacy and its many facets when it comes to information that has been placed in a public space. Recent case in point, Mark Zuckerberg’s sister and her Christmas photo. danah concludes that our obsession with Big Data has the ability to destabilise and change our social norms, I would say it already is, but this does not mean we need to remove the concept of privacy altogether.
Big Data is made of people. People producing data in a context. People producing data for a purpose. Just because it’s technically possible to do all sorts of things with that data doesn’t mean that it won’t have consequences for the people it’s made of.
There are great opportunities ahead for HR with adoption of “new” technologies such as Big Data and Cloud Computing however as we move towards this new world we need to be careful not to destabilise our workforce to a point where they disengage or worse still create a world that makes Orwell’s 1984 look like a kindergarten picnic.