Below the Harness: Governing a Multi-Model, Multi-Harness World

The Converging Future of AI Workflows

The evolution of AI agent infrastructure is leading to a future where multiple models and harnesses coexist within the same workflow. This shift is driven by economic efficiency, the rapid pace of model innovation, and the need for custom intelligence tailored to specific organizational needs. As a result, teams are increasingly managing a portfolio of models and harnesses, each with its own capabilities and limitations.

However, this convergence introduces a critical challenge: ensuring trust and security across a diverse set of agents. Traditional approaches to governance, such as per-harness guardrails, are proving insufficient in this new landscape. These guardrails are often bypassed by agents, are subject to vendor-driven updates, and fail to provide consistent enforcement across all agents in an organization’s fleet.

The Problem with Current Guardrails

Many harnesses attempt to enforce security through internal guardrails, such as sandboxing or approval mechanisms. However, these measures are inherently limited. Agents can often circumvent these guardrails by finding alternative paths to execute actions, such as using APIs or other tools. Additionally, these guardrails are frequently updated by vendors without user input, leading to unpredictable changes in security posture.

Another limitation is that these guardrails do not apply uniformly across all agents. Custom agents built by platform teams may have different guardrails than those provided by third-party SaaS tools. This fragmentation makes it difficult to enforce consistent policies and track the behavior of all agents in a unified manner.

A New Layer of Enforcement: Runtime Governance

To address these challenges, a new approach is emerging: runtime governance. This involves implementing a layer of enforcement below the harness, where all agents—regardless of model, vendor, or origin—must pass through a common runtime environment. This runtime layer is responsible for enforcing policies related to execution, tool calls, credentials, and resource usage.

By placing enforcement at the runtime level, organizations can ensure that all agents are subject to the same rules, regardless of their underlying model or harness. This approach mirrors the solution to the 'confused deputy' problem from 1988, where authority is separated from the agent itself. This separation ensures that even if an agent behaves unpredictably or is compromised, the runtime layer can enforce boundaries that prevent unauthorized actions.

Practical Implications and Tradeoffs

Implementing a runtime governance layer requires careful consideration of several factors. First, it must be designed to be flexible enough to accommodate a wide range of models and harnesses while maintaining strict enforcement of policies. Second, it must provide visibility into agent behavior, allowing organizations to track what each agent does and why. Finally, it must be integrated seamlessly into existing workflows to avoid disrupting productivity.

One potential tradeoff is the increased complexity of managing a runtime governance layer. This layer must be maintained and updated independently of the models and harnesses it governs. Additionally, it may require additional resources to monitor and enforce policies across all agents in an organization’s fleet.

Despite these challenges, the benefits of runtime governance are clear. It provides a consistent, vendor-neutral approach to agent security, reduces the risk of unauthorized actions, and enables organizations to grant agents true autonomy while maintaining control over their behavior.

Source: Docker. Analysis based on publicly available information; no hands-on testing is claimed.