AI governance is one of those topics where everyone agrees it matters, nobody disagrees with any individual principle, and then most teams go back to work and do not actually change anything. The gap between “yes we should govern AI” and “here is how we govern AI” is bigger than it looks.

This is meant for the engineer or architect who has been told to “put governance around the AI stuff” and is trying to figure out what that actually means in practice. I am going to skip the slide-deck definitions where I can, and focus on the six things that, in my experience, actually move the needle.

What AI Governance Actually Is

Strip away the consulting language and AI governance is the set of policies, processes, and controls that determine who can use AI, for what, with what data, and with what oversight. It is the same kind of thing as data governance or change management – a framework that lets a company use a powerful tool without losing the plot.

It is not:

  • A single tool you buy.
  • A committee that meets once a quarter and approves things.
  • A long document nobody reads.
  • The legal team’s problem.

It is closer to a control plane. Decisions about AI usage flow through it. Risk is assessed against it. When something goes wrong, the response is shaped by it. If your governance program does not have a clear answer to “can the marketing team use Tool X for use case Y with customer data Z?” then it is not really governance yet – it is intent.

Why You Actually Need It

The honest version of this is that you need AI governance for the same reason you need any other governance: because the consequences of getting it wrong have gotten big enough that the cost of the process is worth it.

A few specific drivers, in rough order of how often I see them push organizations to act:

Regulatory pressure. The EU AI Act is in force. ISO/IEC 42001 is being adopted. Industry-specific rules are stacking on top. If you operate anywhere with meaningful AI regulation, the question is no longer whether to govern, it is whether your governance will survive an audit. Showing up with “we trust our employees” is no longer a credible posture.

Data exposure. The fastest way for a company’s confidential information to end up in a third-party model’s training set is for an employee to paste it into a public chatbot. Most data loss from AI has nothing to do with sophisticated attacks. It is a sales engineer asking a chatbot to “clean up this customer list” at 11pm.

Operational blast radius. This is the one I have felt most directly. AI agents that take actions on real systems can cause real outages. The MCP servers I have been building for network operations can theoretically reboot a switch, change a firewall rule, or push a config. Without governance around what those agents can do and when, you are one bad prompt away from a Sev 1.

Reputational risk. A hallucinated answer in a customer-facing system is a press release. A biased decision in a hiring or lending workflow is a lawsuit. The thing you thought was an internal experiment becomes a public conversation faster than you would expect.

Cost. This one is quieter but real. Without governance, AI spend grows in ways nobody is tracking. Different teams sign up for different platforms. Tokens get burned on use cases that do not justify them. The first AI bill that crosses someone’s desk and triggers a “wait, what are we even doing with this?” is almost always the moment governance gets prioritized.

None of these are hypothetical. All of them have happened, publicly, in the last eighteen months.

The Six Things You Actually Need

If you took everything in a typical AI governance framework and asked “what is the minimum viable set of controls that gives me real coverage?” you would land on something close to these six. They are the structure I use, and the structure I recommend.

  1. Ownership and Accountability
  2. Risk Tiering and Use Case Review
  3. Least Privilege Access
  4. Human in the Loop
  5. Audit and Observability
  6. Lifecycle Management

Order matters. You cannot do risk tiering well without ownership, you cannot enforce least privilege without risk tiers, and you cannot audit anything you have not enforced. Build them in order.

1. Ownership and Accountability

The first question every governance program has to answer is: when something goes wrong with an AI system, who is responsible? Not in a punitive sense. In an operational sense. Who gets the page? Who has the authority to pull the plug? Who signs off when it goes live again?

In practice, this means three roles per AI use case, and they should be different people:

  • Business owner. Owns the outcome. Defines what “working” means. Decides whether the use case still makes sense.
  • Technical owner. Owns the implementation. Picks the model, builds the integration, maintains the code.
  • Risk owner. Owns the boundary. Decides what data the system can see, what actions it can take, when it has to be reviewed.

For low-risk use cases the same person might fill two of these roles. For anything customer-facing or anything that can change state in a production system, separate them. The bias toward consolidating ownership is real and it always ends with no one feeling responsible at 2am.

Accountability also means a named decision-maker for going-live. “The committee approved it” is not accountability. “Priya approved it on 2026-04-12 after reviewing the risk assessment” is. Write down the name.

2. Risk Tiering and Use Case Review

Not every AI use case carries the same risk, and treating them identically is how governance programs become bottlenecks. The fix is a tiering model that says, up front, how much process each use case will go through.

A simple three-tier model that has worked for me:

Tier Examples Process
Low Internal summarization, code completion, meeting notes Self-service approval, standard guardrails
Medium Customer-facing chat (informational only), internal workflows that touch sensitive data Risk review, named owners, quarterly check-in
High Anything that takes autonomous action, customer-facing decisions, regulated data Full review, executive signoff, continuous monitoring

The point of the tiers is not to be precise – it is to be fast on the low-risk things so the governance team has time to be careful on the high-risk things. A program that takes six weeks to approve a meeting-notes summarizer will get bypassed, and then you will have no governance at all.

Use case review should happen before implementation, not after. Reviewing a built thing creates pressure to approve it because someone already spent the money. Reviewing a proposal lets you say “no, but here is how you could get to yes” without anyone feeling like their work is being thrown away.

What to actually ask in a review:

  • What problem does this solve, and how was it solved before?
  • What data does the system see? Where does that data come from? Where does it go?
  • What actions can the system take? On what systems? With what blast radius?
  • What is the failure mode? What does a wrong answer look like, and who is harmed by it?
  • How do you know it is working? How would you know if it stopped?

The last question is the one most teams have not thought about. If they cannot answer it, the use case is not ready to ship.

3. Least Privilege Access

The same principle from every other corner of security applies here, and it is more important than usual because of how easily an AI system can be coaxed into doing things outside its intended scope.

The thing to internalize: an AI agent’s effective permissions are the union of every permission of every credential it can use. If the agent runs as a service account with read access to ten databases, you have to assume any prompt could read from any of them. Prompt injection is real, and the model is not your authorization layer.

Concretely:

Scope the credentials, not just the model’s instructions. Telling the model “do not read from the HR database” is a suggestion. Not giving the service account access to the HR database is a control. The second one is the one that matters.

Per-use-case service accounts. Resist the temptation to share one big AI service account across teams. Each use case should have its own credential, scoped to exactly the data and actions it needs. When something goes wrong you will know which use case to revoke.

Time-bound and revocable. Credentials should expire by default. Long-lived AI credentials that nobody owns are how you end up with a former contractor’s API key still calling your production API in 2027.

No standing write access for experiments. If the system is in pilot, it gets read-only. Period. The friction of “I cannot make it do the thing in the demo” is the friction that prevents the demo from breaking production.

This is the area where the gap between “we said the right things in the policy” and “the credential actually has the right scope” is widest. Audit it directly. Look at the IAM policy. Do not trust the slide.

4. Human in the Loop

“Human in the loop” gets used loosely. It can mean anything from “a human approves every output” to “a human exists somewhere in the org and could in theory intervene.” The useful version is more specific.

A workable model has three positions a human can occupy:

+-------------------+   +-------------------+   +-------------------+
| Human in the loop |   | Human on the loop |   | Human out of the  |
|                   |   |                   |   | loop              |
| Human approves    |   | Human monitors;   |   | Fully autonomous; |
| each action       |   | system runs but   |   | review after the  |
| before it happens |   | human can stop it |   | fact only         |
+-------------------+   +-------------------+   +-------------------+

The risk tier determines which one you get. High-risk actions need a human in the loop. Medium-risk needs a human on the loop. Low-risk can be out of the loop with audit. Document which position each use case is in and what it would take to move it down a step.

The thing nobody warns you about: human-in-the-loop only works if the human is paying attention. If you ask a person to approve fifty AI decisions a day and forty-eight of them are obviously fine, by the second week they are rubber-stamping the whole queue including the two that should have been caught. This is well-documented in aviation and medicine, and it applies here too. Design the queue so that most items do not need human review, and the ones that do are genuinely worth the attention.

Practical implementations I have used:

  • Confirmation dialogs for destructive actions (the agent has to ask twice; the second call carries a confirmation token).
  • Approval workflows in chat (Webex Adaptive Cards, Slack interactive messages) where a named approver clicks yes/no.
  • Sampling for review (humans review a random 5% of low-risk outputs to catch drift).
  • Escalation triggers (the system itself flags when its confidence is low and a human is required).

5. Audit and Observability

If you cannot reconstruct what an AI system did and why, you do not have a governable system – you have a black box that you hope is behaving.

The minimum I expect for any non-trivial AI use case:

  • Every prompt and response logged, with timestamps and user/agent identity.
  • Every tool call logged, with arguments and results.
  • Every change to the system itself (model version, prompt template, tool set) versioned and dated.
  • Logs retained long enough to support an incident review – typically 90 days at minimum.
  • A way to search the logs by user, by use case, by time, by tool.

Most of this is unglamorous engineering. It is also the thing that turns a Sev 1 from a multi-day investigation into a thirty-minute query. After every incident I have ever worked, my first question has been “what was the system doing right before this broke?” and the value of the logging is exactly proportional to how fast I can answer that.

There is a privacy wrinkle worth flagging: prompts and responses can contain sensitive data. The logs themselves become a regulated asset. Encrypt them, scope access tightly, and figure out your retention policy before you have a discovery request.

Observability is more than logging. You want metrics:

  • Volume (how many calls per day, trending).
  • Latency (P50, P95, P99 of model response time).
  • Quality (when measurable – click-through, accept rate, downstream task success).
  • Cost (tokens, dollars, per use case).
  • Failure rate (errors, refusals, fallbacks).

If you cannot graph these for each use case, you do not have observability. You have logs.

6. Lifecycle Management

AI use cases are not products you ship once. They drift. Models get deprecated. Prompts that worked six months ago start to underperform as the data they reason over changes. Tools get added, vendors get acquired, the team that built it leaves.

Lifecycle management is the discipline of treating each AI use case as a thing with a beginning, a middle, and an end.

Beginning. Documented use case, approved owners, baseline metrics, signoff. The output of the use case review.

Middle. Periodic re-review (quarterly for high-risk, annually for everything else). Did the use case actually deliver what it was supposed to? Has anything changed – model, data, regulation, team – that invalidates the original approval? Are the metrics where you expected?

End. Deprecation. Every AI use case should have an exit plan. What replaces it? How do users find out? What happens to the data? How long are the logs kept? “We will figure it out when we get there” is the answer that leaves orphan systems running for years.

A pattern that works: tie use case re-review to the same cadence you use for any other system – access reviews, vendor reviews, security reviews. AI does not need its own special cycle. It needs to fit into the cycles you already have.

Best Practices, Briefly

A few things that have generalized across the programs I have seen:

Start with an inventory. Before you write a single policy, find out what AI is actually being used in your organization. The answer is always more than you think. You cannot govern what you cannot see.

Make the easy path the safe path. If using the approved AI tool is harder than pasting data into a random web chatbot, people will paste data into a random web chatbot. Sanctioned tools have to be at least as convenient as unsanctioned ones.

Default to small models. This is operational, not regulatory, but it matters: the bigger the model, the bigger the cost, the slower the response, and the more reasoning capacity you are giving the system. Pick the smallest model that does the job. You can always upgrade.

Write the runbook before you need it. “What do we do if the model says something racist in production?” is a question with a much better answer if you wrote it down in advance.

Govern the people who govern the AI. The team that runs the AI governance program needs its own oversight. Otherwise the people approving use cases become a single point of failure – both for risk and for bottlenecks.

Decouple the model from the use case. Build your applications so that swapping the underlying model is an afternoon, not a quarter. Models change fast. Lock-in compounds.

Train people on what to escalate. Most governance failures are not “the model did the wrong thing.” They are “the person using it did not know they were supposed to flag it.” A short, plain-language guide on what triggers escalation is worth more than a hundred-page policy.

Resources Worth Your Time

The AI governance space is noisy. A short list of things that have actually been useful to me, with my one-line take on each:

  • NIST AI Risk Management Framework (AI RMF 1.0). The most practical of the major frameworks. Use it as a checklist if nothing else. nist.gov/itl/ai-risk-management-framework
  • ISO/IEC 42001. The international standard for AI management systems. Heavier weight, but if you are pursuing certification, this is the bar.
  • OWASP Top 10 for LLM Applications. Concrete, technical, and updated regularly. If you build with LLMs, read this. owasp.org/www-project-top-10-for-large-language-model-applications
  • EU AI Act. Even if you do not operate in the EU, the structure of the risk tiers is worth understanding. It is shaping how other regulators think.
  • Google’s Secure AI Framework (SAIF). A useful complement to NIST, with more emphasis on the security side. safety.google/intl/en_us/cybersecurity-advancements/saif
  • Anthropic’s Responsible Scaling Policy. A worked example of how an AI company governs itself. Useful for thinking about what “good” looks like even if your scale is different.

A note on consultancies: most of the “AI governance frameworks” being sold right now are repackaged versions of the above. Read the source material first. You may not need the consultant.

Where to Start

If you have no governance today and are trying to figure out the first move, do this in order:

  1. Inventory. Get a list of every AI tool, model, and integration in use across the org. Include the unsanctioned ones if you can find them. This will take two weeks and will surprise you.

  2. Name the owners. For each item on the inventory, write down a business owner, a technical owner, and a risk owner. If you cannot, that item is either too small to matter or too unowned to keep. Decide which.

  3. Tier the use cases. Sort the inventory into low, medium, and high risk using the framework above. Apply the harder controls to the high-risk items first. Do not try to boil the ocean.

  4. Build the review process. A lightweight intake form, a small review group, a documented decision. This becomes the front door for every new use case.

  5. Add logging. For anything in the medium or high tier, get logging in place. Retain it. Make it searchable.

  6. Schedule the re-reviews. Put the next review date on every approved use case the day it is approved. Calendar it. Re-reviews that are not on a calendar do not happen.

That is a six-month plan if you are starting from zero, faster if you have some of the pieces already. The thing not to do is wait until you have a perfect framework before you start. The perfect framework does not exist, the imperfect one you start using next week will be better than the perfect one you launch in two years.

The One-Line Version

AI governance is the unsexy infrastructure that lets your organization use AI without it using you. Build it the same way you would build any other control plane: ownership first, then risk, then access, then oversight, then audit, then lifecycle. Skip a layer and the rest of it does not hold.

The work is not exciting. It is also the difference between a program that survives its first incident and one that does not.