2021-02-04 HR Examiner article John Sumser The HR Data Department stock photo img cc0 by AdobeStock 235503318 edit 544x324px.gif

“What we’re talking about here is the emergence of a new set of analytics-driven data-based elements of the HR department.” - John Sumser


The HR Data Department

Part One


Over the next two to five years HR’s most important asset will be its data. AI and intelligent software tools are swarming the HR Technology stack. Because these tools are being introduced without much in the way of customer acceptance testing, they’re likely to spawn a new era in HR management. Let’s take a look at what I expect to see in more detail.


What we’re talking about here is the emergence of a new set of analytics-driven data-based elements of the HR department. Not only will intelligent tools change the scope and focus of the work of HR (as it will for the rest of the organization) ever-increasing volumes of data will push some parts of HR into roles that look very much like operations. HR will accomplish things that will improve workforce agility and productivity and move beyond a cost center. As both the keepers of the data and an integral part of the functions that data depends on, HR has an opportunity to move into a more strategic role.


As I mentioned earlier, the most important asset in the emerging HR Department is its data. This data can be used to make an important difference to the organization, including better automation, insight, productivity, and organizational safety while monitoring and intervening in a variety of settings.


While service delivery and execution is the principal job of the HR Department, it will evolve quickly to become a deep source of actionable insight into the company’s workforce and its adaptation to market circumstances. The transformation to data-centric operations will be driven, in part, by the move to a standard conversational interface with HR. Today’s chatbot overpopulation problem will be resolved with a single, department-wide utility. And that is what drives the completion of digital transformation in HR.


Elements of The Emerging HR Data Function


2020-10-15 HR Examiner graphic Emerging HR Data Function data and its uses graphic 544x329px.jpg

Click to enlarge




  • At the heart of the HR Department is its data. The data comes from three places: within HR, within the company, and outside the company. This section includes the responsibility for the cleanliness and quality of data, Each segment is exploding in volume and relevance.

  • Internal HR Data is the primary asset of the HR Department. It is the stuff that is collected, processed, analyzed, and disseminated by the company’s HR Technology stack.

  • External data is the foundation of new ways of seeing the company. From the sorts of employment branding feedback found on Glassdoor to predictive sourcing tools like EngageTalent, we’ve just started to understand how data from outside the company can shape internal experiences and operations.

  • Data from Operations and the Rest of the Business is where one gets the ability to see the actual work of the company. In order for HR to effectively show its impact on critical business operations, it needs data from the rest of the company.


For maximum utility, this data will be stored in a Data Lake. Data Lakes offer the capacity to store data in disparate forms in order to uncover their utility in later stage examination. Ddata lakes and data warehouses are both widely used for storing big data, but they don’t have the same function. A data lake is a vast pool of raw data, the purpose for which is not yet defined. A data warehouse is a repository for structured, filtered data that has already been processed for a specific purpose.


Uses of Data (Internal HR Processing)


Not all uses of data are the same. And some uses will change significantly.


  • Reporting: Once a measurement becomes routinely consumed, it is a report. Reports look at and visualize movement, activity, progress, or variance in a particular area. Many reports begin their lives as analytics.

  • Analytics involves a deep examination of a particular organizational dynamic. When an analytic becomes routine, it is a report.
    • Predictive analytics is a subset. This is the use of statistical tools to forecast outcomes in specific areas.

  • Data Science Lab: Most organizations over a certain size will have at least some of the functionality of a data science laboratory. The data science lab will allow the organization to model and understand the functioning and behavior of their organizations. Current offerings like Visier will inevitably morph to include model development by their clients.

  • AI Lab: AI comes to the organization in overt and covert ways. The primary function of the AI lab is to understand and manage the various forms of intelligence as they spread. The management and maintenance of intelligent tools happen here.


Interrogation of Data


There is a difference between the way that data is managed and developed and the way that it is delivered. The distinctions can be a simple format in the case of reporting.


2017-04-21 HRExaminer photo img sumser john bio pic IMG 3046 black and white full 200px.jpg

John Sumser is the Principal Analyst for HRExaminer.

  • Operational (History): Reporting is always backward-looking. The delivery of reports is a summary of what happened.

  • Real-time Surveillance: The surveillance category contains any real or near-real-time look at the behavior of the workforce and can range from trouble spotting to pulse engagement surveys.

  • Discovery and Automation: With intelligent tools, the HR team can begin to ask scenario-based questions (What if) and start to explore patterns for clues to enhance organizational performance.




Intelligent tools and data confer new and important responsibilities on the HR Department. These new missions are all data-centric. The main themes are:


  • Security: According to the Sierra-Cedar HR Systems Survey, 40% of HR Departments are responsible for the security of Personal Identifying Information (PII). That’s where HR’s responsibility begins. The degree to which HR produces proprietary and sensitive information expands with each new experiment in Data Science or AI. HR’s focus on the company’s overall security will focus on the human element as tools like KeenCorp deliver. HR is going to be responsible for continuing to protect the company in new ways.

  • Safety is related to security. The 21st Century organization uses its data to explore, predict, and report safety and well-being issues. Group trust, discrimination, bullying, harassment, and other behaviors can be detected in data and will be approached as health and safety concerns rather than compliance issues. HR will intervene before a victim reports much in the same way that industrial plants detect machine maintenance and potential failures. Privacy is also a safety issue and there will be acknowledgment and discussion that the ways we collect and use data have real consequences on people who are the subject of the data. We will see more governance issues involving who has access and how the data is used.

  • Ethics isn’t a rulemaking function. Ethics is a set of evolving questions that guide the company through complex decision making. External scrutiny, in the form of social media, reviews, journalism, customer reviews, news, and political visibility will continue to expand. Transparency becomes a constant because the only way to do things right consistently is to assume that you are doing them in public. As a result, ethics becomes a constant conversation that learns to take account of current circumstances and adjust.




  • Experiences: While it never rises to the level of the employee’s complete experience of work, much of HR’s work involves the delivery of experiences. As conversational tools become HR’s interface with the workforce, the department will be responsible for understanding and improving the employee experience of HR. This ranges from routing interactions about benefits and payroll to more complex involvement with performance management.

  • Personalization: Compensation, benefits, career path, performance evaluation, promotion, discipline, termination, and all of the other HR interactions are extremely personal. The more that data can be used to reduce the friction implicit in a relationship where employees feel vulnerable, the more HR can become the glue for the organization.

  • Products: While HR’s use of intelligent tools, data science, and people analytics is evolving, it will produce formal prioritization and budgeting processes to tackle what will be a huge backlog of projects. The best model for this sort of output is the product release process currently used in software development. HR will be developing services that have many characteristics of other subscription processes.


The HR Product Manager in
The Emerging HR Data Department


A shepherd for a dizzying array of tools and operational complexity in a landscape dotted with legal and cultural landmines.


Digital transformation has failed to deliver on either the digital or transformation parts of the promise. The reasons are many. In HR, the problem is largely caused by the way that functions are siloed and separated. Managing the value of the overall people-data asset, combined with the desire to deliver a uniform level of experience quality, will drive the emergence of a single conversational interface.


It’s the inherent benefits of a unified data asset, coupled with reduced costs and increased employee satisfaction that will drive the emergence of a new role: The HR Product Manager.


Currently, HR is managed as a series of loosely federated spheres of influence. Employees attempting to interact are often faced with extensive coordination in order to access services. One of the main roles the HR Generalist currently serves is as a human switchboard for the maze of HR services.


The HR Product Manager role will evolve from two longstanding threads: technology implementation and agile management processes. In agile development environments, the Product Manager is the person who, with dotted line relationships to everyone, is responsible for moving products forward. As HR increasingly becomes productized (you have to do this to deliver on the experience idea), the role of people who shepherd new ideas into the workforce takes shape.


HR Product Management will occupy a staff level function in the HR Department. The job entails gathering data, and insight from across functions in order to deliver new products, personalization, and/or experience. The primary function of the job is to turn digital transformation into a series of cross-silo, achievable projects while expanding the momentum of the projects that emerge from increasing measurement and analysis. These functions will be the drivers behind the evolution of employee experiences that deliver the entirety of HR to the employee.


The Consequences of ‘Columbusing’


Columbusing is urban slang for ‘discovering something that’s existed forever.’ The term is a reference to the fact that Columbus is credited with the discovery of a place that had been inhabited for thousands of years before his arrival. It’s a pejorative term that indicates a failure to understand history and the value of what’s come before.


Columbusing at the intersection of HR and Technology is increasingly common in the development of intelligent tools. It takes the form of applications that are developed without ever taking a look at the existing solution set. Bias and bias reduction come to mind. With decades of EEOC compliance, monitoring, and audits, HR Departments have learned how to manage bias at the regulatory level. Precious few of the new tools build on prior learning.


It’s similar in the various tools that measure culture. There are a solid 75 years of science and practice in the areas of cultural measurement and organizational effectiveness. Yet, few if any new intelligent tools vendors build on these historical foundations. You can also see the consequences of ‘columbusing’ in newly minted assessment tools.


It is unusual for HRTech vendors to open up their internal scientific processes to peer review even though the value they deliver is of the kind of science that needs public peer review.


The lack of transparency and review is another outcome of radically decentralized research and development (funding small projects or companies and saddling them with revenue demands). When venture financing dominates an industry’s R&D investment, lots of projects get started in places where earlier work is available. It’s unusual for founders to see this.


It’s worth doing research into the underlying science before installing a tool that is divorced from it. You risk the penalties associated with reinventing the wheel.


Employee Experience Demands Data Integration with Operations


Executed well, HR is mostly invisible to employees. The current emphasis on employee experience in HR is a shallow reflection of a much larger problem. The overall experience of the workplace, which is beyond the control of HR, is a critical component of an ever-engaging relationship with the workforce.


Operations is where real data about employee performance, sentiment, experience, accomplishment, and satisfaction is held. HR has always had a derivative view, expressed in secondhand reports of performance in reporting systems designed for merit plan distribution. Access to data from operations promises to unlock the longstanding promise that HR has a contribution to make in the area of employee performance.


While new performance management systems attempt to channel feedback about workplace value from a variety of sources into a channel for recognition and improvement, the measures are all second-hand. Operations data offers real-time performance appraisal among other things. Rather than measuring the equivalent of individualized surveys, real work data gives an objective context to the conversation.


There are all kinds of operational data from the entirety of digital communications to keystroke contents and rhythms. Contemporary Organizational Network Analysis tools are the beginning. Actual workplace data is the next step.


AI and the Focus on Business Outcomes


HR departments will be held accountable for their impact on significant business outcomes like revenue, profitability, productivity, project completion, and marketplace agility. Direct causality will always be difficult to demonstrate. That said, people analytics will become increasingly concerned with developing algorithms and models that allow HR projects to be prioritized by their actual value to the organization.


There are techniques (like Structured Equation Modeling) that already allow a disciplined data team to develop reasonable arguments about the impact of HR on the bottom line. As the department gets accustomed to using data from the rest of the business, the ability to see the relationship between HR and organizational health and accomplishment will continue to grow.


It appears that there will be a growing emphasis on the way that the business delivers value to stakeholders other than the shareholders. I recall an announcement from the Business Roundtable that said that business should focus on investing in employees, protect the environment, and deal ethically with their suppliers. (The idea that shareholder value was the most important objective is only 25 years old and was promulgated by this same group.)


The pressure to understand investment and return in the workforce will drive deeper research in academia and in organizations. It’s reasonable to imagine a fairly different landscape for results reporting that evolves over the next five years. Measures of employee equity, impact of diversity, suitability of tools for future adaptation will all be experimented with. Models of organizational process, measures of organizational intelligence, and work process effectiveness (from a human capital utilization perspective) are all predictable elements of this trend.


Modeling the Organizational Culture


There is no element of the intelligent tools revolution that is more guilty of ‘columbusing’ than the various tools that claim to understand, measure, and influence culture. It’s as if none of the vendors were aware of the work of Peter Senge, Edgar Schein, or the other major thinkers on the topic of organizational culture and anthropology in general. Instead, they present shallow conceptions of culture that are likely to produce errant output. Already, we hear concerns from CEOs about whether ‘sentiment analysis’ produces anything that’s useful.


We understand precious little about the deep function of our organizations. Much of what is currently considered a measure of culture are wisps of sentiment and response to very immediate stimulus. Engagement surveys and other assessments of employee sentiment help to understand the current mood of the workforce. Culture and its implications are something much deeper.


If current tools measure something like organizational weather, culture is the terrain and geological structure. While the weather varies with the seasons, the culture itself is a slower changing thing with rhythms of its own. It’s going to take effort and investment to use the various network analysis tools to see the topography clearly and understand what is below the surface.


A comprehensive picture of how the organizational actually functions requires some sort of structural framework as a point of departure. The existing work of academics like Senge and Schein provide a point of departure. In the absence of a conceptual framework, the early going will include some humorous disasters.


Complexity Science


There are a few more areas we need to explore to map out The HR Data Department and The HR Product Manager that I’ve been discussing. If you haven’t already done so, read the two linked posts above before moving forward.


People and their organizations are complex, adaptive systems. That means that their behavior is determined more by emergent phenomenon (which are difficult to predict) than by force of habit. Current intelligent tools default to a view that the future is just the past plus a little more; that tomorrow is just more yesterday.


If humans and their organizations were more like a game of chess, go, or whatever, contemporary intelligent tools would be able to predict the future with extreme accuracy. But, like the weather, ecosystems, and social political processes, organizational behavior exhibits many non-rational characteristics.


Complexity science has real potential to move the conversation along. Rarer than a data scientist, people who are good at building models of complex systems are a very rare breed. As the coming years start to show the shortcomings of contemporary models, we will turn our attention in the direction of complexity science.


Group Intelligence (Superminds)


There are established mechanisms for measuring the intelligence of groups. Most contemporary psychological assessments are centered on the individual. They produce results designed to judge and categorize people as a way of supplementing other information in the hiring process. The idea that groups could be assessed along the same lines, although it seems obvious in hindsight, was quite an insight.


Expect to see the idea of assessing group intelligence extended to include machines as members of human organizations. The basic notion is that group intelligence (or whichever aspect you decide to quantify) should be routinely measured and improved. The object of any addition to or rearrangement of a group ought to be improvement in process, flow, and intelligence.


Collective intelligence has been a quiet goal throughout the evolution of computing. From Douglas Englebart’s famed 1960s “mother of all demos” until now, the steady undercurrent is that we are building a global brain. This would make it reasonable to start to manage the intelligence of our workgroups, organizations, and communities.


Intelligence is just the first characteristic to measure in groups. Empathy, adaptability, cognitive load, performance tuning, and adoption variation will all be the subject of group and organization measurement. We will start to see the organization as something that has an awareness and identity composed of individuals, but is larger than and separate from them.


One way of thinking about measured intelligence is to use IQ measurement as a metaphor. A score of 100 is a normal IQ. A 70 IQ (low intelligence) is 30 points off the mean while a score of 130 (high intelligence) is 30 points above the mean.


Let’s say you have 100 people in your organization. An organizational IQ score of 100 might represent average utilization of your intelligence asset. A 70 organizational IQ score would then mean suboptimal utilization of talent. A score of 130 would mean the opposite. (This may be, but is not necessarily better. We also know the ‘geniuses’ who can’t seem to find their way back from lunch.)


Being able to measure the impact of a proposed new employee on these aspects of group intelligence (and other variables) might well be part of the solution to the dramatic failure rates in recruiting.


The Limits of Specialized AI


The current phase of intelligent tools can be thought of as Specialized Artificial Intelligence (SAI). This is not anything like the General Artificial Intelligence (GAI) envisioned by most people when they think about the topic.


The current crop of tools is great at discovering predictability and patterns in data. That single skill can deliver extraordinary benefit in situations where repetition is the essence of the problem. Large quantities of data about predictable processes is the heart of today’s SAI.


GAI, on the other hand, involves systems that can predict unanticipated changes.


SAI will be useful in focused arenas with large quantities of data. It will perform less and less well in situations that more closely resemble the way that knowledge work is done. Much of organizational life is devoted to ‘barely repeatable processes’ that are loosely lumped together in a department. For example, everyone knows that the Recruiting Department uses different methods to recruit different kinds of people. Process standardization is useful for reporting and management. But the actual quality of the result is determined in the places where the approach is non-standard. It’s hard to see how intelligent tools add value in these cases.


Watch for part two tomorrow.

Read previous post:
Can a workplace recover from discrimination or harassment or whistleblower drama?

What can HR do to recover from discrimination or harassment or whistleblower drama? Resident employment law attorney Heather Bussing reviews...