For many areas where AI is used, an 80% success rate is entirely reasonable. Automatic tagging of your photos? Perfect. Facial recognition of the people in your photos? Fantastic. Criminal tracking? Not so quick.
The industry standard measure of machine learning (and other AI related technologies) is 80%. If the forecast is successful 80% of the time, the tool is considered a success.
If you wrinkle your eyebrow at this, you’ll hear that humans are never 100% right/successful either; that 80% is ‘good enough’.
The fundamental benefit of any AI related technology is that it can predict. For many things, an 80% rate is entirely reasonable. Automatic tagging of your photos? Perfect. Facial recognition of the people in your photos? Fantastic. Criminal tracking? Not so quick.
The closer it gets to home and the more you depend on the results for safety, security, or guarantees, the less adequate a 20% error rate is. The closer the prediction comes to affecting the work, livelihood, hopes and aspirations of individual people, the more important complete accuracy becomes. An 80% success rate means that the machine is wrong one out of every five times. If you are making even the simplest HR decisions, a 20% error rate is reason for concern.
If the ‘smart system’ is making product or film recommendations, a 20% error rate is an inconvenience. Making one out of five merit increases badly is not as acceptable. Getting it right 80% of the time is unacceptable if the tool is the foundation of a project to decrease hiring bias.
At 80%, AI systems require a range of human beings to assist and overrule them. You can’t really install an HR tool that is 80% effective unless you clearly understand the rest of the things required to manage it.
