Striff Engineering Product
The claims in your architecture docs, checked on every pull request
Design docs have a second reader now: coding agents build from them, and nothing in a pull request checks that the code still agrees with the prose. Striff turns the sentences already in a repository into rules and checks them at both revisions of every pull request. What that looks like on one real change, and how often it fires across 609 public ones.
On this page
Your architecture docs have a second reader now. For twenty years a README or a design note was read by a person, who noticed when a sentence had stopped being true and asked. Today coding agents read AGENTS.md, CLAUDE.md and the architecture notes before they write anything, take every sentence as fact, and build from it. A decision written down once steers thousands of generated lines. A sentence that has quietly gone stale steers them too.
Nothing in a pull request checks for that. The diff shows the code. The sentence lives in a different file, so the reviewer never sees the two side by side, and the merge goes through with the prose and the code disagreeing.
Striff closes that gap. It reads the documents already in your repository, turns the sentences that make a checkable claim about the code into rules, and checks each rule against the parsed code at the base and the head of every pull request. A language model does the translation from prose to rule. A program does the checking, and it can only report what it computed. There is no rule file to write and nothing to configure, because the rules are the ones your team already wrote down.
The result is a GitHub check run beside your tests with a table of the rules the change touched, each one quoting the sentence it came from and the line in the code that decided it. Here is what that looks like on one real pull request.
One change, one sentence, one verdict
Ericsson’s ecChronos schedules repairs for Apache Cassandra. Its core.impl README goes through the module class by class, and of NodeWorker it says: “Calls RepairScheduler.putConfigurations() to keep jobs up to date.”
Pull request #1786 moved that call into a new class, SchemaRefresher. After the change NodeWorker no longer references RepairScheduler at all. The refactor was fine. The README was not in the diff, so nobody reading the diff had a reason to open it, and the sentence has been wrong since the merge on 8 September.
Striff checked 28 rules from that repository’s documentation at both revisions. Twenty-seven held. One did not:
Ericsson/ecchronos #1786 · documented rules
RepairScheduler.putConfigurations() to keep jobs up to date core.impl/README.md:136 NodeWorker depends on RepairScheduler Broken by this PR The rule held before this pull request and is broken after it. ConnectionType enum (from utils) defines the three ecChronos control modes: connection/README.md:57 ConnectionType is in com.ericsson.bss.cassandra.ecchronos.utils.enums.connection General term resolved Held The rule held before this pull request and still holds after it. VnodeRepairTask / IncrementalRepairTask — Concrete RepairTask subclasses for vnode and incremental repair respectively. core.impl/README.md:65 VnodeRepairTask depends on RepairTask, and IncrementalRepairTask depends on RepairTask One sentence, two rules Held The rule held before this pull request and still holds after it. Left, the sentence as the maintainers wrote it, with its file and line. Middle, the rule it became, in plain English and as the formula that is evaluated. Right, the verdict on this pull request: whether the parsed code satisfies the formula before the change and after it.
A finding is only worth something if you can check it without trusting us, so every verdict is built from facts you can look up in the two public commits:
From the sentence to the verdict
The rule came out of the prose. Everything under it was computed from the parsed code at each revision, which is why the finding can name the line that moved and cannot invent one that did not.
The fix is one word in the README, and on the pull request that caused it that is a one-line follow-up commit, made while the author still remembers why the call moved. Left alone, the sentence keeps costing people time. The next contributor is sent to NodeWorker and reads a class that no longer does the job. A coding agent handed the same README has a worse option available: the quickest way to make the sentence true again is to give NodeWorker a scheduler and re-create the dependency the refactor just removed. A stale sentence is a set of instructions for undoing your last refactor.
What you get, and what you do not
Every rule Striff reads gets one of four verdicts. It held, so nothing in this change breaks it. It was violated, so this change broke it. It was already broken before this change, which is reported but never blamed on the author in front of you. Or it was restored, broken before and true after. A rule the parser cannot answer, because the code it names is generated, or lives where a source parse cannot see it, is left out rather than shown as passing. Striff never reports a pass it did not compute.
The mechanism is the reason the verdicts can be trusted. The model only proposes rules; a proposed rule that names a class the parsed code does not contain is dropped before evaluation, so the model cannot invent a violation by inventing a name. The program that evaluates the rules cannot add a finding. How every claim traces back to your docs or your code covers that in detail, and the rule language and its limits are written up by the founder.
How often it fires
Rarely, and that is the design. A check that fired on every pull request would be muted within a week. Across 609 public pull requests in 542 repositories, Striff read 7,161 rules out of those repositories’ own documentation. 5,674 of them could be answered from the parsed code; the rest named things a source parse cannot see and were left out. Of the answered rules:
7,161 rules read, 5,674 answered · 609 pull requests
Roughly two percent of answered rules accuse: 97 broken by the change under review, 53 already broken before it. Everything else held, which is what a correct document looks like. Accusations concentrate, too: a single repository accounts for half of them, so the rate per repository is far lower than the rate per rule.
The 53 already-broken rules are the quiet second result. Each is a sentence in a repository’s own documentation that was already false before the pull request. Whether the fix belongs in the code or in the sentence is the maintainer’s call, and now the maintainer knows there is a call to make.
Where the docs and the code disagree
The sweep was not small repositories. Nearly every one had over a thousand stars, and a third had over ten thousand. Five of the disagreements it found, each checked by hand against the default branch in September 2026, largest repository first:
- Apache DolphinScheduler★ 14.5k
docsThe contributor guide says to implement
org.apache.dolphinscheduler.registry.api.RegistryFactory.codeNo type by that name is declared anywhere in the repository. The name survives only in the guide, in two languages.
- BenchmarkDotNet★ 11.5k
docsThe exporters page documents the properties of
ISummaryStyle.codeThe type is
SummaryStyle, a class. There is no interface by the documented name. - Apache Pinot★ 6.1k
docs
DESIGN.mdsayspinot-sql-ddldepends only onpinot-spi,pinot-commonandcalcite-babel.codeThe module’s build and
MaterializedViewSchemaInfereralso pull inpinot-query-planner. - MCP C# SDK★ 4.5k
docsLine 258 of the Copilot instructions says to use
McpServerFactory.codeDeleted in December 2025 by the same agent that wrote the line. The line is still on
main; a fix is open. - Ericsson ecChronos★ 37
docsThe
core.implREADME saysNodeWorkercallsRepairScheduler.putConfigurations().codeThe call moved to
SchemaRefresherin #1786. The sentence did not move with it.
Each row links to the line in the document. The bar is the repository’s star count on a log scale. These are disagreements between a repository’s own documentation and its code, reported as such; whether the fix belongs in the sentence or in the code is the maintainer’s call.
Writing docs that get checked
You do not have to change how you write. ecChronos’s README was written for people, and it produced 28 rules. Sentences of a certain shape get checked, though, and it helps to know which.
Name real things: NodeWorker is checkable, and “the worker” only is if one class answers to it. Say where things live and what they use: “ConnectionType is in utils” and “NodeWorker calls RepairScheduler” are the placement and dependency claims a parser can answer at both revisions. Say what must not happen: “the core never imports from plugins” is a rule the moment it is written down, and it is the kind an agent in a hurry is most likely to break. Leave intent as prose; “we prefer composition” is not a claim about structure and nothing will pretend it is. And keep the agent files honest, because AGENTS.md and .github/copilot-instructions.md are read by the same process, so the rules you give your agents are checked against the code they write.
Where to start
Install the GitHub App and open your next pull request. Striff reads the docs you already have and checks the rules it finds in them, free on public repositories, with nothing to write and nothing to configure. For a public pull request in somebody else’s repository, the browser extension does the same in a tab beside Files changed.