Today’s article focuses on the things that are likely to go wrong at a system level. Individual ‘agents’ (does that term mean anything anymore?) fail at predictable rates. All of them, without exception, break at some point. Scarier still, a collection of agents fails more often that its individual components. It’s a relatively solvable engineering problem.
Worrying about what could possibly go wrong is not a psychological condition. It is a part of adequate preparation for the future. Generative AI adoption is a journey. adequate preparation involves trying to predict what will happen. Again, this is an engineering problem. Bringing well understood engineering principles to bear is how we fully integrate a new technology into the toolkit.
Failures cascade. They do not happen in isolation. You get a flat tire, you have to stop and fix it. That makes you late to the meeting. You were going to present the big new plan. Now you have to rejuggle everyone’s calendar. One thing leads to another.
It’s not that all failure cascades have negative outcomes. Serendipity provides magic sometimes.You meet a business partner while waiting for your flat to be fixed. You stumble into the best restaurant by mistake.
Remember Eisenhower’s admonition …’Plans are worthless, but planning is everything’? Or the grittier, ‘No plan survives first contact with the enemy’? The value is in the thinking, not the document. Understanding probable failures and their consequence enables you to think clearly during the inevitable disaster.
Planning increases the likelihood of a positive outcome. It’s not a guarantee. It’s a way of improving your odds.
There’s a table at the end of this piece that lists AI Governance Features You Can See Coming. It describes the risk, explains why it matters, and offers suggested ways to approach solving the problem. If you squint and breathe slowly, everything you need is there. It is not anywhere close to comprehensive. Just a look at the biggest ones.
A key point I’ve missed making: AI speeds up and scales mistakes the organization already makes. AI governance turns out to be a blend of organizational governance and engineering. Most of today’s scary stories about AI are really failures of governance and engineering that were allowed (even encouraged) by the corporate culture of their makers.
The other thing AI does is expose the weaknesses. Every organization has places in their culture and bureaucracy that are underdeveloped. Sometimes, it’s ignorance. Sometimes, it’s willful. Sometimes, it’s a question of budgets and priorities. Sometimes, it’s sheer arrogance and hubris. Sometimes, it’s impossible to think or see out of the box.
In a way, AI flows like water. It penetrates to the lowest possible level. And, that’s where the risk really lives. But, that’s hard to see because it involves detailed data, data structures, deep understanding of individual jobs, and a systems oriented vision of potential consequence.
