If AI agents are becoming organizational actors, then governance needs to move beyond principles and into operational structure.
In Camille Stewart Gloster’s upcoming book The Insider You Build, she explains that governance is not defined by policies or structures, but by whether it can actually influence system behavior at runtime. In an agentic environment, governance only exists where it can shape, constrain, and intervene in decisions as they happen.
Her simplified framework focuses on three capabilities:
- Authority design: defining what agents are allowed to do
- Runtime enforcement: ensuring governance operates during execution
- Attribution and learning: maintaining traceability and improving over time
Chinnaraju’s 2026 paper argues that effective AI agent governance requires organizations to treat agents as formal organizational actors rather than experimental tools.1
This begins with Agent Charters, which function as credentials for each agent. Think of these as an operating license or passport for every AI agent, clearly defining:
- Boundaries: what the agent is allowed to do, what data it can access
- Authority: where its authority stops
- Escalation: when it must escalate or be shut down
Without this formal documentation, agents operate in a grey zone, where authority is unclear and accountability cannot be enforced.
Ownership
Autonomy does not remove accountability but redistributes it. The paper identifies four critical functions:
- a sponsor responsible for purpose and alignment
- a steward managing the agent lifecycle
- a risk/compliance owner ensuring regulatory and ethical alignment
- and critically, an operations controller who can monitor and override the agent in real time
Together, these roles form a control loop where authority flows to the agent, but accountability flows back through monitoring. Embedding these roles into access control systems, workflows, and approval processes, supported by audit logs is essential to avoid informal or unclear responsibility.
Runtime Execution & Monitoring
Agents must be integrated into enterprise systems (e.g. ERP, CRM, transaction platforms), but crucially, they must also be observable and controllable in real time. If organizations enable execution without the ability to monitor behavior or intervene, they create what the paper defines as a structural governance failure, where operational capability exists without governance capacity.2
This means real-time monitoring, dynamic permissioning, escalation triggers, and the ability to intervene or override decisions as they happen. In other words, governance isn’t something you set upfront, it’s something you must enforce at the moment of decision.
Maturity-based Approach to Autonomy
To manage this risk, the paper proposes a maturity-based approach to autonomy, where authority is gradually increased across four stages:
- Start with assistive systems (recommendation only)
- Move to supervised execution
- Conditional bounded autonomy with escalation
- Only then consider full autonomy
The central principle is that autonomy should be earned through evidence of control and reliability, not granted upfront.
Scaling Governance
Finally, these elements are brought together in a staged implementation roadmap, moving from governance readiness, to pilot deployment, controlled scaling, and ultimately institutionalization. Progression through these stages is gated and requires demonstrating effective oversight, monitoring, and alignment before expanding agent use. At maturity, agents become embedded in core operations, subject to compliance cycles and board-level oversight.
Where Should Governance Sit?
As organizations begin to operationalize AI governance, the question emerges: Where should governance actually live in an agentic system? Emerging research points to three primary models:
- Centralized governance: a single control layer or supervisor oversees all agents
- Distributed governance: each agent governs its own behavior
- Hybrid models: combining central oversight with local controls
Caldeira and Banerjee’s comparative study (2025) highlights a critical trade-off. Distributed models improve transparency but reduce efficiency. Centralized models improve performance but reduce visibility. Interpretability and performance are often in tension. In practice, this isn't a one-size-fits-all answer, but depends on the organization’s size and industry compliance requirements. Most organizations will probably need hybrid approaches, balancing control, scalability, and explainability.
Governance Beyond Systems: The Leadership Question and Agent Risk Management (ARM)
Recent scrutiny of leadership within the AI industry also highlights a deeper, often overlooked dimension of governance: human accountability at the top. Investigations have raised uncomfortable but important questions about trust, transparency, and who ultimately holds power when the stakes are this high. When governance relies on the integrity of a few individuals, and that integrity is called into question, the entire system becomes fragile.
The lesson for organizations is clear: governance cannot depend on personalities or promises, it must be structurally embedded, independently verifiable, and resilient to leadership failure.
Effective governance requires a clear "separation of powers." While Business Units should own the agent, defining its purpose, managing its "repair," and reaping its rewards, Infosec might own the "kill switch." This ensures that when an agent drifts or is compromised, the decision to halt operations is made by those prioritized with safety, not just performance.
This is where the concept of Agent Risk Management (ARM) becomes essential. We must treat AI agents as part of our extended workforce. Just as we manage contractors and third-party vendors, we need a framework to manage the digital identities and behaviors of our AI actors.
At KnowBe4, we believe accountability must be designed into the system, not assumed. By treating agents as a critical branch of your workforce through Agent Risk Management (ARM), governance becomes independently verifiable and resilient to leadership failure. In a world where AI has the power to act, we must ensure that the human "on the loop" remains empowered, informed, and ultimately in control.
References:
Caldeira, V. & Banerjee, A (2025). Where Should Control Reside in Multi-Agent Language-Model Systems? Proceedings of the 6th International Conference on AI Research (ICAIR 2025) 6(1). https://papers.academic-conferences.org/index.php/icair/article/view/4157
Chinnaraju, A. (2026). When AI Agents Act: Governance, Accountability, and Strategic Risk in Autonomous Organizations, 12(12), https://doi.org/10.51244/IJRSI.2025.12120050
Damien, C. (2026). AI Hallucination Cases.
https://www.damiencharlotin.com/hallucinations/
MyBroadband (2026). Scandal erupts over South Africa’s new draft AI policy which used fake references generated by AI.
https://mybroadband.co.za/news/government/643743-scandal-erupts-over-south-africas-new-draft-ai-policy-which-used-fake-references-generated-by-ai.html
Stewart Gloster C. (2026). The Insider You Build. John Wiley & Sons Canada, Limited.
https://camillestewartgloster.com/theinsideryoubuilt
