Striff Engineering Product
Every Striff finding comes with a receipt
AI review comments are easy to write and hard to trust. What a Striff check on your pull request actually says, where each line of it comes from, and how to verify any of it yourself in under a minute.
On this page
Most developers have the same experience with AI code review. The comments sound right. Some of them are right. You cannot tell which without redoing the analysis yourself, so after a few weeks you stop reading them.
A better prompt does not fix that, because the problem is not how the model writes. It is that the model’s opinion is the finding, and there is nothing underneath it to check. So Striff works the other way round, with one rule we hold everything to:
Here is what that looks like from your side of the pull request.
What lands on your pull request
When you open a pull request in a repository with Striff installed, a check appears beside your CI. It has four parts, the same four you can see on the homepage:
- A review summary. A few sentences on what the change does structurally: what gained or lost a dependency, what public surface moved.
- Top review items. Only the things worth your attention, which in practice means places where the change contradicts your own documentation.
- Documented rules. Every rule from your docs that Striff could check, each with the sentence it came from and whether this change kept it or broke it.
- A diagram of the change. The classes the pull request touched, what they gained and lost, and how they connect.
None of it needs configuring. The rules come from the READMEs, ADRs, ARCHITECTURE.md and agent instruction files already in the repository.
Checking one claim, start to finish
Here is a real finding. Ericsson’s ecChronos documents its core.impl module class by class, and pull request #1786 made one of those sentences false. The check said:
NodeWorker no longer depends on RepairScheduler, which core.impl/README.md line 136 says it calls. It did at the base revision.You do not have to trust that. It is made of four things you can look up:
The receipt for one finding
Open the README at line 136 and the sentence is there. Open NodeWorker.java at the base commit and the call is on line 209. Open it at the head commit and it is gone. If any of those lookups disagreed with the check, the check would be wrong, and you would be able to prove it in a minute.
The language model did exactly one thing in that finding: it read line 136 and proposed the rule “NodeWorker depends on RepairScheduler”. Everything after that was computed. The model did not decide the rule was broken, and it could not have, because the verdict comes from parsing the code at both commits. The full story of this pull request is on the blog, including why a one-word README fix mattered.
Who decides what
A check is built in four steps. Two use a language model, and neither of those can put a finding in front of you:
Where the language model is allowed in
| Step | Done by | Can it add a finding? |
|---|---|---|
| Parse both revisions of your code | A parser, the way a compiler would | ✗ It produces the facts everything else is checked against. |
| Read your docs and propose rules | Language model | ✗ It only proposes. A rule that names a class or package your code does not contain is thrown away before it is ever checked, so the model cannot invent a violation by inventing a name. |
| Check each rule at the base and the head | A program | ✓ This is the only step that produces a verdict. Only a rule that held before the change and fails after it counts against the change. |
| Write the summary and the notes | Language model | ✗ It phrases what was computed. A sentence that asserts something the comparison did not find is dropped, and the computed statement is shown instead. |
The model proposes and the program decides. When the model gets something wrong, the mistake has to contradict the code or the docs it was given, which a program can catch. You do not have to.
This also means that when Striff is wrong, it is wrong the way ordinary software is wrong. A bad verdict traces back to a sentence, a rule and two parsed facts, so it can be reproduced, fixed and tested. There is no equivalent for “the model felt confident”.
What it will not tell you
Being true is not enough to earn a line in your check. The bar is one question: does the reviewer already know this from the diff or the diagram?
True, and still not a finding
| Candidate | True? | In your check? |
|---|---|---|
| "This pull request adds an import" | ✓ | ✗ It is a line of the diff. Restating the diff is not analysis. |
| "Coupling on this class went up by 4" | ✓ | ✗ The number is on the class in the diagram. On its own it implies no action. Why a coupling delta is not a finding. |
| "This package now depends on that one", when your docs say nothing about either | ✓ | ✗ Possibly exactly what you intended. With no rule your team wrote down, a tool can only guess, so the new edge is drawn on the diagram instead. |
| "This change made a sentence in your README false" | ✓ | ✓ It needs your docs and the code at both revisions. The diff contains neither, which is why nobody caught it. |
There is no battery of generic heuristics grading the shape of your code. A rule nobody on your team wrote down is a rule nobody on your team agreed to. If you want something checked, write it in your docs.
It only reports what it can verify
The other half of not making things up is not implying things. A pass/fail check can put “we looked and it is fine” and “we could not look” behind the same green tick, and a green tick is exactly what a reviewer trusts. Striff does not do that. It is a best-effort check: a rule it cannot answer from the code is left out of the results, never shown as passing. Every rule it does report has one of four outcomes:
Every rule it reports is one of four
Violated
Held before this change, fails after it. The only outcome blamed on the pull request.
Already broken
Broken in the code Striff checked, not by this change. Shown on its own line, never blamed on the author, and never passed off as a pass.
Held
This change keeps it: nothing in this pull request breaks it. It is not a claim about the rest of the codebase.
Restored
Broken before, true after: the change fixed something the docs promised.
A source parser sees less than a compiler: no generated methods, no annotations, no reflection. Say your docs mention Order.total() and Order is a Java record. The compiler generates total(), so the parser never sees it. "The method is missing" would be false, and "the rule held" would be a guess. So Striff says neither, and leaves that rule out.
That makes Striff best-effort by design. It can miss a rule a compiler would have answered. What it will not do is tell you a rule held when it could not see whether it did.
What you get out of it
Fewer comments, and every one of them checkable. Across 1,394 rule checks on 74 public pull requests, Striff found exactly one violation, and it was real. When a check that quiet says something, it is worth reading, and you can verify it faster than you could argue with it.
It also changes what your docs are for. A sentence in your README stops being a hope and becomes a rule, checked on every pull request, whether a person or a coding agent wrote the code. Here is how to write docs Striff can check, though it reads the docs you already have without any changes.
Try it on a pull request you know
The quickest test is a pull request whose history you already know, so you can judge every line of the check yourself.
- Your public repositories are free. Install the GitHub App and open a pull request. Rules, diagram and review notes, on every PR.
- Someone else’s public pull request. The free Chrome extension shows the same review in a tab beside Files changed, without installing anything on the repository.
- Private repositories start at $29 a month. See pricing.