Evidence-Producing Controls

Status note: this project is conceptual. It is a proposed catalogue, not a completed one. No pattern here has been validated in a production estate by us.

The question

Most control frameworks specify what should be true. Far fewer specify how anyone would demonstrate it eighteen months later, to someone who was not present and has reason to be sceptical.

The gap is not usually intent. It is that the control was implemented as a practice rather than as a mechanism, and practices leave no trail. The question is whether the patterns that do leave a trail can be catalogued concretely enough to be applied at design time, when it is cheap, rather than during an audit, when it is not.

The approach

Assemble a catalogue where each entry is defined by the artefact it produces rather than by the intention it expresses. A candidate entry would specify:

• the obligation it is intended to satisfy;
• the mechanism, described concretely enough to implement;
• the artefact it emits, its retention, and where it lives;
• the failure mode — specifically, how the control could appear to be operating while producing nothing;
• the engineering cost of retrofitting it versus building it in.

Initial candidate areas: deployment provenance for model and configuration versions; scheduled evaluation that records results irrespective of whether anyone reads them; threshold breaches that raise tickets rather than notifications; override capture that requires a reason to proceed; and action-level logging that records effects and identities rather than only conversations.

What would make it useful

The catalogue would be worth something only if entries are specific enough to be implemented from, and honest about cost. A pattern library that recommends comprehensive instrumentation everywhere would be correct and useless.

The harder and more interesting part is the retrofit cost column — the difference between building a control that emits evidence and adding evidence emission to a control already in production. Our expectation is that the ratio is large enough to change design decisions, but that is an assumption we would want to test rather than assert.

Why we are interested

Governance failures are rarely failures of principle. Organisations generally have the policy. What they lack is a mechanism that would notice the policy being violated, and by the time that is discovered the systems are in production and the cheap moment has passed.

Related reading: Governance After Deployment.