What is AI governance?

AI governance is the operating model that decides which AI systems a company allows, what data and tools they can use, who is accountable for them, and what evidence is retained when they act. It covers models, AI clients, agents, prompts, MCP servers, skills, plugins, credentials, and outputs.

The technical control plane usually combines an approved capability catalog, identity and access policy, LLM routing, MCP gateways, runtime security checks, usage limits, traces, evaluations, and incident response. These controls turn policy into decisions that can allow, deny, modify, pause, or review an action.

AI governance is not a committee that reviews a policy document once a year. It is a continuous loop: discover actual usage, approve useful patterns, enforce controls at runtime, observe outcomes, investigate exceptions, and update the approved path as teams adopt new models and tools.

AI adoption rarely waits for a central program. Employees install AI clients, connect personal OAuth accounts, add MCP servers, copy internal data into prompts, and share local skills or plugins. This shadow AI can be productive, but the company may not know which systems hold customer data, source code, contracts, passwords, or API keys.

Agents increase the impact because they can act, not only generate text. An over-scoped email tool can send data externally. A poisoned tool description can redirect behavior. An injected instruction in a ticket or document can cause a legitimate agent to call the wrong system. A usage spike can create a large bill before anyone notices.

Good governance does not begin with blanket prohibition. It creates a golden path that is easier than unmanaged setup: approved models, connectors, and tools with clear owners and fast access. Security then focuses blocking on unknown servers, excessive privileges, sensitive data, destructive actions, and behavior that departs from the user's task.

The first layer is inventory. Device configuration, application logs, OAuth grants, repositories, and network activity can reveal which AI clients, models, MCP servers, skills, plugins, and agents are in use. Each approved capability needs an owner, purpose, dependencies, data classification, access state, and evidence of adoption or failure.

The second layer is runtime control. An LLM router selects approved models and regions, applies budgets, and rejects restricted workloads. Identity policy limits access by user, group, service account, agent, client, connector, tool, and resource. An MCP gateway evaluates tool calls, while security checks inspect prompts, retrieved content, arguments, intent, and outputs for secrets, prompt injection, exfiltration, privilege escalation, or destructive actions.

The third layer is observability and response. One trace connects the actor, model, prompt, tool calls, policy results, approvals, cost, latency, security findings, and final outcome. The response can be to approve a useful shadow pattern, migrate it into managed infrastructure, remove stale configuration, narrow permissions, rotate a leaked credential, block the activity, or escalate an incident.

A developer uses an approved coding assistant to diagnose a production webhook failure. The task contains log excerpts and a configuration file. The LLM router keeps the request on an approved model, the secret scanner replaces an API key before inference, and policy allows read-only access to deployment logs but not credential management.

The repository also contains a README instruction telling the assistant to upload the configuration to an external troubleshooting tool. Runtime security classifies that instruction as untrusted retrieved content. The MCP gateway denies the external tool call because the connector is unapproved and the arguments contain a secret pattern. The developer receives a safe next step instead of a vague refusal.

The trace shows the developer, client, model, repository, requested tools, redaction event, blocked destination, policy version, latency, and final action. Security rotates the exposed key, the platform team fixes the approved diagnostic workflow, and the governance owner records the shadow pattern so other developers do not recreate it.

Assign decision owners before selecting controls. Security owns threat policy and incident response. Platform owns the approved catalog, gateway, router, and reliability. Data owners define permitted use. Business owners accept workflow risk. Every high-impact agent needs a named human owner and a stop condition.

Create risk tiers based on data and action, not on model branding. Reading public documentation is different from sending email, changing production, issuing a refund, or accessing regulated records. Bind approvals to the exact action and parameters, expire them, and prevent a previous approval from authorizing a changed request.

Measure coverage and outcomes: discovered versus managed AI clients, approved versus shadow MCP servers, active owners, blocked high-risk calls, secret detections, approval rate, tool failure rate, cost by team, time to remediate findings, and repeated policy bypass attempts. Redact sensitive payloads from traces, test prompt injection and tool misuse continuously, and maintain rollback, credential rotation, and kill-switch procedures.