Spec drift
Spec drift: what it is and how to catch it
Spec drift is the gap that opens when the code no longer does what its committed specs, design docs, ADRs or agent instruction files (AGENTS.md, CLAUDE.md) say it does.
What is spec drift?
A repository’s docs make claims about its code: NodeWorker calls
RepairScheduler, the domain never imports infrastructure, the parsers live in
core.parse. Spec drift is what happens when a pull request makes one of those claims
false and the document isn’t in the diff. Nothing fails. The tests pass, the review passes,
and from that merge on the document describes code that no longer exists.
Why it matters now
A stale doc used to cost a new contributor an afternoon. Now coding agents read
AGENTS.md, CLAUDE.md, specs and design docs as instructions, and they
don’t get suspicious. Told that NodeWorker calls RepairScheduler,
an agent will happily put back the dependency your last refactor removed. A stale spec is a set of
instructions for undoing your own work.
What it looks like
Three real cases from public repositories. Each links to the line and the commit.
A refactor moves the work; the doc still credits the old class
Ericsson’s ecChronos README says
NodeWorker “calls RepairScheduler.putConfigurations() to keep jobs up
to date”. Pull request #1786 moved
that call into a new class, SchemaRefresher. The refactor was fine; the README was not in
the diff. The walk-through shows both revisions.
The pattern agents are told to copy breaks its own rule
In agentbridge, AGENTS.md points
agents at MessageFormatter.kt as an exemplary pattern with “Zero UI
dependencies”. Since a refactor
in May, the file imports from the ui package. Every agent that follows the
instructions copies the dependency along with the pattern. Striff’s
check on pull request #1201
lists it as already broken on master.
The doc names a class that no longer exists
objectionary/eo’s Maven plugin README said
org.eolang.parser.Program runs the XSLT transformations and Pack holds the
list. Neither class existed any more. After Striff listed both names as out of date, the maintainer
replaced them with Pipeline and Canonical in
one commit.
How to catch it
By hand, read the doc with the code open and check only the sentences that make a claim about structure: every name it mentions still exists, every “A calls B” is still in A’s imports, every “must not” is still true, every file held up as an example still has the property it is praised for. It works once. It doesn’t survive the next fifty merges.
On every pull request, Striff does it for you, as a GitHub check beside your tests:
- It reads the docs you already have: specs, ADRs, READMEs,
AGENTS.md,CLAUDE.md. No rule files, no configuration. - It turns each checkable sentence into a rule and checks it against the code before and after the change, so it knows whether this pull request broke it or it was already broken.
- It confirms every break before showing it: an agent re-reads the sentence and the code twice, and anything not confirmed is withheld.
- It quotes the sentence and links the line, so the reviewer decides whether the code or the doc is wrong.
See a real check, the dashboard demo, or reports on open-source repositories.
FAQ
Is spec drift the same as documentation drift?
It is the same gap with higher stakes. Documentation drift misleads a human reader. Spec drift is that gap in a document something builds from: a coding agent reads the spec or AGENTS.md as instructions, so a stale sentence steers the next change.
Can AGENTS.md or CLAUDE.md drift?
Yes. They name classes, say where things live and say what must not depend on what, and every one of those sentences can go false on a refactor that never touches the file.
How do I detect spec drift in CI?
Turn the checkable sentences in your docs into rules and evaluate them on every pull request, at the base and at the head, so a rule the change broke is told apart from one that was already broken. Striff does this as a GitHub check, from the docs already in the repository, with nothing to configure.
Which sentences can be checked?
Claims about structure: that a class exists or has a member, where a type lives, what depends on what, and what must never depend on what. Statements of intent (“we prefer composition”) are not claims a parser can answer, and Striff counts them as couldn’t check rather than passing them.
What does Striff cost?
Free on public repositories. Private repositories start at $29 a month. See pricing.