graphic for The 2018 Index of Predictive Tools in HRTech: The Emergence of Intelligent Software

 

When 80% is success.

On November 27, 2017, in HRExaminer, by John Sumser

2017-11-27-hrexaminer-article-john-sumser-when-80-percent-is-success-photo-img-cc0-via-pexels-full-w-text-v2-544x360px.jpg

For many areas where AI is used, an 80% success rate is entirely reasonable. Automatic tagging of your photos? Perfect. Facial recognition of the people in your photos? Fantastic. Criminal tracking? Not so quick.


 
The industry standard measure of machine learning (and other AI related technologies) is 80%. If the forecast is successful 80% of the time, the tool is considered a success.

If you wrinkle your eyebrow at this, you’ll hear that humans are never 100% right/successful either; that 80% is ‘good enough’.

The fundamental benefit of any AI related technology is that it can predict. For many things, an 80% rate is entirely reasonable. Automatic tagging of your photos? Perfect. Facial recognition of the people in your photos? Fantastic. Criminal tracking? Not so quick.

The closer it gets to home and the more you depend on the results for safety, security, or guarantees, the less adequate a 20% error rate is. The closer the prediction comes to affecting the work, livelihood, hopes and aspirations of individual people, the more important complete accuracy becomes. An 80% success rate means that the machine is wrong one out of every five times. If you are making even the simplest HR decisions, a 20% error rate is reason for concern.

If the ‘smart system’ is making product or film recommendations, a 20% error rate is an inconvenience. Making one out of five merit increases badly is not as acceptable. Getting it right 80% of the time is unacceptable if the tool is the foundation of a project to decrease hiring bias.

At 80%, AI systems require a range of human beings to assist and overrule them. You can’t really install an HR tool that is 80% effective unless you clearly understand the rest of the things required to manage it.

 

graphic for The 2018 Index of Predictive Tools in HRTech: The Emergence of Intelligent Software


 
  • This makes sense. With technology rapidly progressing, I feel that some programs are adopted too quickly without analyzing the risks. Just as you pointed out, an 80% success rate fails when real people are involved.

  • Andrew Marritt

    1) The chances you will be successful with Russian Roulette are 83.3%. Is this good enough?
    2) Prediction without a loss function is pointless for decision making yet that is exactly what the majority providing analytics solutions are proposing (hint, the loss function is much more difficult to construct)

Page 1 of 11
Read previous post:
HRIntelligencer v1.26

Highlights: The bidding war for qualified AI practitioners, FLUREE, the latest project from SilkRoad, Why Artificial Intelligence is Different from...

Close