Challenges With Data 1

Challenges with Data

A recent article in Harvard Business Review calls to attention some of the challenges inherent in analyzing Human Resources data.  The writer, Peter Cappelli, points out that analysis of HR data is hampered by a variety of factors including legal issues encountered when crossing international borders, lack of abundant data, and a sense that not much could be learned from modern “big data” analysis capabilities.

The problems that Mr. Cappelli identifies are valid, but not limited to HR alone.  Below are a few areas where I’ve encountered similar issues to the HR-specific ones described in the HBR article:

International Boundaries to Efficiency

Cultural differences abound in multinational organizations.  I’ve encountered similar challenges to the cultural differences described in the HBR article.  During a project earlier in my career I was working to develop a system to collect high-volume manufacturing data across a multinational organization.  The biggest problem encountered was not technical (we could pull data from our production machines and centrally analyze it with already-owned COTS software and minimal coding) and it was not related to skills.

Our challenge was the incongruent collection of laws across the multiple nations in which our organization operated.  Did you know that it is not legal in some European countries to collect manufacturing plant performance data that can be broken down according to shift?  And yet, how can we focus improvement efforts if we are forced to use general data that doesn’t identify areas ripe for improvement?  It’s like getting laser eye surgery but the surgeon is forced to use a spotlight instead of a laser.

In the end, we settled for localized progress instead of taking advantage of the synergies that would have occurred with a wider-reaching effort.  It was a necessary middle-of-the-road solution.

Lack of Data

Without data it is a challenge to determine the best improvement opportunity.  Sure, the people who are affected by problems can tell you what those problems are.  But systems can quickly become complex and it can be difficult to know where to deploy limited resources to achieve the best return.

It can be necessary to run through multiple iterations of data collection to find the best method.  I’ve found that before spending resources on an expensive solution, you should sometimes make a small time investment to collect everything available and analyze it for usefulness.  In a manufacturing environment this might mean collecting production data from every machine (even if it has to be manually collected on paper forms – gross).  This initial collection may reveal that some of the data sources are unreliable or otherwise not useful for larger-scale analysis.  Then you can focus your efforts on the indicators that are truly useful.

 “What Use is Analysis, Anyways? We Already Know Our Issues; Just Fix Them!” (AKA “We Don’t Have Data”)

This is a great question.  From my perspective, it highlights the differences in some of the tools commonly used to identify problem areas.  From the data analysis perspective, multi-level Pareto Charts tell a good story and go a long way towards identifying the root cause of the problem.

Without adequate data, however, a common (and useful) tool is the Cause & Effect Matrix.  This should be used as part of a group exercise to identify improvement opportunities by weighing the experiences of the individuals involved.  Though not as precise as a more data-driven analysis, it can yield great results (and great participation).

Regardless of the project in question, there is a strong likelihood that administrative (or cultural, or legal) boundaries will be crossed.  With these crossings there will surely be conflict that must be overcome.

What are some of the boundaries that you have encountered?  What techniques have proven successful for you, and which techniques have proven ineffective?  Please share some of your experiences below!