Oversight Metrics

EU AI Act Article 14 requires human oversight that is effective, not decorative. Oversight metrics turn that requirement into numbers computed continuously from your existing governance data: how often humans intervene, how fast they respond, how often they override the agent, and how many approval requests expire with nobody watching. No additional instrumentation is required. The metrics are derived from policy evaluations and approval workflow records you already generate.

The Metrics

Metric	Definition	What it tells you
`evaluations.fail_rate`	Failed policy evaluations over decided (pass + fail)	How often agent activity violates policy
`approvals.requested`	Approval requests created in the window	Human intervention demand
`override_rate`	Denied over human-resolved (approved + denied)	When humans reviewed, how often they said no
`unattended_rate`	Expired over requested	Approvals that lapsed with no human decision
`avg_approval_latency_seconds`	Mean time from request to human decision	Oversight responsiveness
`defer_to_human_policies`	Enabled require-approval / require-human-review policies that apply to the agent	Whether defer-to-human criteria are defined at all
`review_overdue`	Whether the agent’s scheduled review date has passed	Review currency

The unattended rate is the most honest single measure of whether human oversight exists outside the policy document: the approval mechanism existed, and nobody decided in time. A stale-pending request counts as expired the moment its expiry passes, even before the background sweeper updates its status.

Per-Agent Scorecard

curl "https://api.meshai.dev/api/v1/agents/agt_abc123/oversight?start=2026-06-01T00:00:00Z" \
  -H "Authorization: Bearer msh_YOUR_API_KEY"

{
  "success": true,
  "data": {
    "agent_id": "agt_abc123",
    "period_start": "2026-06-01T00:00:00Z",
    "period_end": "2026-07-01T00:00:00Z",
    "risk_level": "high",
    "review_overdue": false,
    "defer_to_human_policies": 2,
    "evaluations": { "total": 1240, "passed": 1180, "failed": 60, "fail_rate": 0.0484 },
    "approvals": { "requested": 18, "approved": 12, "denied": 3, "expired": 2, "pending": 1 },
    "override_rate": 0.2,
    "unattended_rate": 0.1111,
    "avg_approval_latency_seconds": 340.5
  }
}

Tenant-Wide Summary

curl https://api.meshai.dev/api/v1/oversight/summary \
  -H "Authorization: Bearer msh_YOUR_API_KEY"

Returns the same metrics aggregated across all agents, plus agents_evaluated (distinct agents with policy evaluations in the window).

Window Semantics

Both endpoints accept start and end query parameters (ISO 8601). Default window is the last 30 days; the maximum range is 180 days.
Timestamps without a timezone offset are treated as UTC.
Historical windows are reproducible. Approval expiry is judged as of the window end, not the current time, so a report for a closed period returns the same numbers every time you run it. A request that lapsed after your window closed was still pending within it.
Requires the governance:read scope.

How Defer-to-Human Counting Works

defer_to_human_policies counts enabled policies of type require_approval or require_human_review whose conditions match the agent, using the same condition matcher the policy engine uses at evaluation time, including risk-tiered scoping. An unclassified agent never matches a policy scoped to specific risk levels, so classifying your agents is a prerequisite for risk-tiered oversight coverage.

Approvals: the human-in-the-loop workflow these metrics measure
Risk Classification: risk levels that scope defer-to-human policies
Incident Reporting: the evidence pack cites oversight activity in the incident window

Getting Started

Proxy (Zero-Code)

Framework Guides

Governance

EU AI Act Compliance

Billing

The Metrics

Per-Agent Scorecard

Tenant-Wide Summary

Window Semantics

How Defer-to-Human Counting Works

​The Metrics

​Per-Agent Scorecard

​Tenant-Wide Summary

​Window Semantics

​How Defer-to-Human Counting Works

​Related

The Metrics

Per-Agent Scorecard

Tenant-Wide Summary

Window Semantics

How Defer-to-Human Counting Works

Related