There ought to be a warning sign at the entrance to every LLM: Infinite Rabbit Holes Ahead. I use analogies extensively and LLMs help me make sure that the comparisons are accurate. That’s what makes this look at governance so complete.
Someone told me that ‘governance is the rules of the road.’ I loved the analogy and decided to dig in to it. Behavior on today’s roads is a complex mix of laws, norms, infrastructure, and technology. Then, there are the ‘unwritten rules of the road. It’s the same gap between the way it’s supposed to be and the way that it is I’ve been talking about.
Using AI today is like driving without a license, seatbelts, brakes, roadsigns, pavement, or agreed upon roads. If it evolves like our traffic systems, rules will arrive after accidents happen. If it evolves the way Silicon Valley has, the unwritten rules of the developers will be the foundation of the actual rules. There’s a third way in which the organization saddles up and takes full responsibility for its use of intelligence.
And so, I set off to get the maximum out of my fresh new analogy. It helped me see how complex it will be to integrate humans and machines into our new organizations. Under current circumstances, the customer owns all of the risk once they customize the agents (and other human - machine integrations).
You have to consider the cases for one car and multiple cars both inside and outside the organization.
Each one of the boxes in the 2x2 matrix has an additional dimension: the percentage of responsibility divided between vendor and customer. For the most part, the vendor is shielded from responsibility for outcomes. You tailor it and the consequences are yours. But, the creator of the agent, commercial or internal, holds much of the data and tools required to troubleshoot, maintain, and improve the individual AI instance.
In the road analogy, both cars and drivers need to be registered and licensed. That’s the right starting point for AI governance: a registry of all AI uses in the organization. Who owns, operates, creates, maintains, improves, and troubleshoots are important things to determine and document of every AI instance. Full governance covers the authority, responsibility, and accountability for operations and outcomes of each registered agent and each sanctioned operator.
(I’m calling them AI instances because the universe of AI is larger than agents wrapped around an LLM engine).
Since AI instances can not be held accountable or liable, you have to tie the responsibility and accountability for outcomes directly to a human being. There may be organizational resources that get deployed to solve some of the areas beyond the expertise of the owner. But, without single points of authority and responsibility, accountability becomes a nebulous thing. While that’s likely to be how some organizations deal with the problem, its a recklessly open invitation to trouble.
You’d think that simple diagram would suffice. Unfortunately, I’ve never met a rabbit hole I didn’t at least peek down. The exploration went much further.
The next couple of illustrations are ridiculously complicated and extremely accurate. I started to explore road construction, social norms surrounding transportation, road design as governance, and more. The first illustration shows the many multiple layers of materials, activities, and norms that fully make up the rules of the road.
If you look closely, you’ll find a wealth of info and a ton of ridiculousness. I particularly enjoy the bike path to nowhere. And still, this is what it takes to create a road from scratch.
The second chart takes that illustration and revises it for use as model of AI governance. It really is this complicated. That said, I’ve been working to convey this complexity as simply as possible.
AI governance is the set of rules, roles, and records an organization uses to decide who/what may build, buy, test, monitor, deploy, change, maintain, and improve AI systems, who answers for the results, and how anyone would know when something went wrong.
It’s better to think of it as decision governance.
As you can see from the graphics, there is a book to be written. For now, the graphic represents a top level outline. It assumes that regulatory compliance is a point of departure. The reality is that AI governance will be nothing more than a set of binders that no one pays attention to unless it becomes ‘the way we do things around here’.