Skip to content
coverctl
Blog

Blog

Coverage feedback belongs in the agent loop

coverctl
Klar Labs

Most coverage tools answer a dashboard question: what was coverage after CI ran? That is useful for humans reading reports. It is the wrong loop for AI coding agents.

Agents edit, run tools, and decide whether to keep or revert a change before push. coverctl is built for that loop: per-domain policy in .coverctl.yaml, an MCP tool surface (check / suggest / debt), and local-first execution over native runners (pytest, nyc, cargo-llvm-cov, go test, …).

If your coverage system cannot be called mid-edit with a structured pass/fail the agent can act on, it is not agent-loop coverage governance — it is reporting.

See also: Quick start (AI agent) and MCP server.

What agents do with coverage output

coverctl
Klar Labs

An agent that receives free-form shell output will invent next steps. coverctl returns bounded JSON: passed, summary, domain rows, and on rejection a stable error_code plus remediation.

That boundary does two jobs:

  1. Actionability — the agent knows whether to write tests, lower ambition, or stop.
  2. Safety — dangerous runner flags (--rootdir, --require, …) are rejected before they reach a toolchain.

Read the contract: Rejection schema and Threat model.

From Codecov dashboards to agent-loop gates

coverctl
Klar Labs

Codecov and Coveralls excel at historical dashboards and PR badges. coverctl does not replace those products for leadership reporting — it fills the gap inside the edit loop.

A practical migration:

  1. Keep your existing CI upload for humans.
  2. Add .coverctl.yaml domains that match how you already think about risk (auth, payments, utils).
  3. Point the agent at coverctl mcp serve so check runs before commit.

Compare pages: vs Codecov, vs Coveralls, vs native commands.

Rejection schemas beat prompt hope

coverctl
Klar Labs

Prompt injection against coding agents is not theoretical. If an MCP tool forwards arbitrary flags to pytest or node, a hostile prompt can turn coverage into code execution.

coverctl’s answer is boring and effective: sanitize inputs, reject known-dangerous shapes with stable error codes, and document the schema so agents (and humans) can recover.

Details: Rejection schema. Eval corpus under internal/eval/scenarios/ gates every PR on this boundary.

The lethal trifecta and local coverage tools

coverctl
Klar Labs

Security write-ups about agent systems often cite a lethal combination: access to private data, exposure to untrusted content, and ability to communicate externally. coverctl is deliberately narrow: it runs locally, reads coverage profiles and policy you already have, and does not phone home by default.

Opt-in --mcp-telemetry emits JSONL to your stderr for your own pipelines — still no SaaS upload. Platform teams evaluating multi-repo rollout should start from the threat model and platform teams pages.