“Some of today’s tools (and all of tomorrow’s) do much more than record and report. They suggest, recommend, decide, evaluate, prescribe, filter, analyze, monitor, and learn. Era 1 tools could not hurt people. Era 2 tools can.” – John Sumser

AI Risks, Ethics, and Liability


HR software product liability

Those four little words make some people very uncomfortable. For the entire life of the software industry, no one ever really thought about product liability. The issues were all covered in the unreadable terms and conditions we were required to click through. Things are changing.

The first era of software stretched from ballistic tables and payroll to the latest in HR Forms completion. Over the course of 80 years, software recorded, collected, calculated, and reported data. GIGO (Garbage In, Garbage Out) was the primary principle. There could be no liability because machines simply reported what they were given.

In this second era, things are different. Some of today’s tools (and all of tomorrow’s) do much more than record and report. They suggest, recommend, decide, evaluate, prescribe, filter, analyze, monitor, and learn. Era 1 tools could not hurt people. Era 2 tools can. In a world where machines extract truth and insight, they (at least) share responsibility for the decisions they make. It may be that they have an exclusive right to the liability.

One industry leading CEO says, “we call it machine learning when we talk about it internally. We call it Artificial intelligence when we speak to the market.” For the purposes of this article, I’ll use the terms Machine Learning, Artificial Intelligence, and Big Data somewhat interchangeably. I am referring to our computers’ emerging ability to change their output based on insights derived from new data.

HR Enterprise Software tools will be at the forefront of the implementation of new product liability concerns. Increasingly, HR software recommends and directs the behavior of managers and employees. If the guidance or insight is damaging or wrong, software vendors will be unable to wriggle out from the consequences. Currently, the vast majority of recommendations provided by intelligent HR software are ‘self-correcting.’ They learn from their mistakes and correct the underlying world view. It is common to hear them described as tools that ‘get better with usage.’ Another way of saying that is that their error rate improves over time. One part of the liability issue is the question ‘who bears responsibility for the error rate?’

There is a more difficult dimension.

All cultures, organizational or otherwise, are defined by their biases. The essence of culture is its unique world view. Decisions and behaviors that support and expand the worldview are rewarded. Things that undermine or contradict the worldview receive negative feedback.

Since algorithmic decision making adjusts to the things that make a culture different, they tend to amplify the biases of the culture. Most of these machine learning tools are black boxes. The only way to see the bias is by examining the output. In other words, these new tools may create liability before it can be discovered and managed.

Intelligent machines (and wonderful theories about their near and long term potential) are at the heart of today’s pop culture. Everyone ‘knows’ that self-driving cars are just around the corner, that robots are going to take your job, and that pretty soon you’ll be talking to an intelligent assistant who knows where you put your car keys and can order more laundry detergent.

Yet, these programs and machines are not people who are generally governed by standards of reasonable care. They are property, which is governed by laws of strict product liability, warranty, and whether the design is fit for the intended use. The difference is much like an owner’s liability when their dog bites someone. The dog is a separate actor, the owner did not intend for the dog to harm anyone, but the owner is strictly liable because the dog is the owner’s property.

We are seeing companies develop and sell technology based on machine learning processes that the human designers do not fully understand nor control. These machines are giving recommendations and suggestions based on probabilities to employees, many of whom are ill-equipped to understand and effectively use the information. Often, we don’t know whether the information will be useful until we develop and test it over time.

But it’s not just data. It’s evidence. And the laws that will apply are not the ones that organizations have traditionally been operating under. There is a chain of potential liability that runs from the developer vendors through the sales chain to the organizations using the software and machines.

Basic Ethics Questions

HR has been slow to adopt AI and Machine learning but the numbers are growing.

Given that we are still in the early adopter stages, it’s a solid time to think about the ethics involved in using Machine Learning systems to manage, supervise, assess, train, deploy, or categorize Human Beings.

Ethics involves questions of right and wrong. Large institutions are usually concerned with optimization, efficiency, and innovation. They seek ways to maximize returns and minimize costs. They think of their employees as resources first and people second. Corporations have traditionally had some challenges with the very idea of ethics.

Here are the kinds of ethics questions that are going to occupy the conversation about ethics in HR Technology. The questions are standard. The answers may vary from place to place.

  • Who owns employee data?
    Who can sell or manipulate it? How much is an employee entitled to see? If something is wrong, what recourse does the employee have? If it is embedded in some machine learning scheme, what are the ownership variables?
  • If the system learns about itself through data about an employee, does the employee own the learning?
    Can she take it with her when she goes? Is she entitled to royalties if the data is sold for benchmarking or other purposes.?
  • Is it okay to use data that was built without a control group?
    How do we measure effectiveness in the example of experimental control? When machines improve their error rate while implemented in an organization are there extra employee protections required?
  • What is the line between manipulation and motivation?
    If chatbots increase emotional ties, is it okay to use them to increase engagement scores? What are the likely regulatory responses to overly manipulative work environments? Don’t people usually follow orders from authority figures? Doesn’t this apply to machines? Is there a limit to self-congratulatory positive feedback?
  • Are statistics more reliable than human decision-making?
    Where’s the proof? Is empathy a necessary part of decision making? Kindness? Humans are demonstrably more effective at handling novel and/or erratic inputs to decision making. Does that matter? How do you factor unmeasurables into the decision-making process?
  • How do you disagree with a machine’s decisions?
    Can you afford to be the person who is carping about decision quality? How do you get into a position to see bias (it won’t show up in individual recommendations, it’s systemic.)? Do we get stupid in the face of a machine recommendation? Are people predisposed to follow the instructions of an authority figure (consider Google Maps and one’s ability to argue with its recommendations.)
  • Who has the liability for machine recommendations?
    Who pays for damage caused by the machine? How do you handle mistakes? How do you monitor the quality of the algorithm’s performance? Is it ethical to use tools that are known to be imperfect on employees? Are there implicit human experiences that are interfered with when machines are the arbiters of personnel decisions?
  • How do you limit the data’s ability to influence the company?
    How to you turn it off and replace it? How do you know when you have too much influence from a single source? Are there tools that allow you
    to see the risk in machine led decisions?

In a nutshell, the ethics questions we will be grappling with are rooted inthe fact that we simply don’t understand in a sophisticated way:

  • How human beings work
  • How organizations work
  • How the human-machine interface will change these things.

We are going to learn more about each issue in an accelerated way. They are coming to employment decisions, and we will learn from their mistakes and successes. You can expect to discover new ways of thinking about employee safety as the risks at work shift from physical to mental and emotional.

Check back for part two on Thursday May 23, 2019.

Read the entire series

Read previous post:
Learn to Trust Your Gut

“For those looking to figure out solutions to the problems facing their respective organizations or careers, learn to develop, refine,...