Striff Engineering Data & research

AI didn't make engineering discipline obsolete. It made it the whole job.

Design doc, then plan, then the agent builds it piece by piece: the companies that make coding agents now tell you to work this way. We surveyed 335 open-source repositories: 63% already hand their agents docs to work from, and only 8% of those check that the code still matches.

10 min read

On this page
  1. The docs became the program
  2. Every practice got more load-bearing
  3. So we measured it ourselves
  4. The practices that scale themselves, and the one that doesn’t
  5. When the docs are the program, drift is a bug
  6. The bill comes due quietly
  7. So what do you actually do?

First published in September 2025. Rewritten in September 2026 with our own data: a survey of 335 open-source repositories.

Here is how a feature gets built at a growing number of companies now. Someone writes a design doc: what the feature does, where it lives, what it must not touch. The architecture doc already says how the system fits together. Then an agent turns the design into an implementation plan, a list of small, checkable tasks, and works through it one task at a time. The next feature starts the same way, from the same documents.

This isn’t a fringe workflow. It’s what the companies that build coding agents now tell you to do:

What the toolmakers now tell you

WhoWhat they say
GitHub, Spec Kit"Instead of coding first and writing docs later, in spec-driven development, you start with a (you guessed it) spec." It "becomes the source of truth your tools and AI agents use to generate, test, and validate code."
Anthropic, Claude Code"Explore first, then plan, then code." Have Claude write the spec to a file, then: "Once the spec is complete, start a fresh session to execute it."
Cursor"Most new features at Cursor now begin with Agent writing a plan."
OpenAI, CodexExecution plans are "thorough design documents, and 'living documents'", an approach that has "enabled Codex to work for more than seven hours from a single prompt."
Google, Conductor for Gemini CLI"Plan before you build." Specs and plans "live alongside your code in persistent Markdown files."
AWS, KiroEvery feature starts as requirements.md, design.md ("technical architecture, sequence diagrams, and implementation considerations") and tasks.md. Amazon's internal memo, per Reuters: "We're making Kiro our recommended AI-native development tool for Amazon."
Thoughtworks Technology RadarCommitting instruction files such as AGENTS.md to the repository: Adopt. Spec-driven development: "We've seen many developers adopt this style."

Sources: GitHub, Anthropic, Cursor, OpenAI, Google, Kiro and Reuters, Thoughtworks Radar Vol. 33.

Birgitta Böckeler put it most plainly on martinfowler.com: “The spec becomes the source of truth for the human and the AI.”

The docs became the program

Look at what that changes. For twenty years a design doc had one reader, the next engineer, and it was allowed to go stale because that engineer could ask someone. Now its main reader is an agent that implements whatever it says, every time, and asks nobody. The documents in your repository have become the instructions your code is built from.

The adoption curve is steep:

Docs for agents, in eighteen months

60,000+
open-source projects had adopted AGENTS.md by December 2025 (Linux Foundation)
137k
GitHub stars for Spec Kit, in thirteen months
287k
GitHub stars for Superpowers, a skills and planning kit for coding agents, in eleven months
68%
of 2,303 agent context files describe the project's architecture

Sources: Linux Foundation; github/spec-kit and obra/superpowers, star counts as of September 15, 2026 (OpenSpec has 68k and the BMAD method 53k); Agent READMEs, arXiv 2511.12884.

Every practice got more load-bearing

There’s a comforting story that goes with all this: AI writes cleaner code than most humans, so the old disciplines matter less now. I think it has it exactly backwards. Almost no engineering best practice exists to help you write code. They exist to help you change code later, safely, without holding the whole system in your head:

What each practice is really for

1

Tests

Not proof of correctness. Confidence to change something six months from now without fear.

2

Readable code & naming

Cheap onboarding for the next reader, who is now as likely to be an agent as a person.

3

Small PRs

Units of change a human can actually hold in their head and meaningfully judge.

4

Docs & ADRs

Shared memory, and now the instructions agents build from.

The common denominator: every practice manages the cost and risk of future change. None of them is about typing speed.

What AI changed is the volume of change. If best practices are the machinery for managing change, and AI multiplied change, every one of them became more load-bearing, not less. The industry data agrees:

What the industry data shows

+23%
pull requests merged on GitHub per month, year over year (Octoverse 2025)
+91%
PR review time on teams with high AI adoption, alongside 98% more PRs merged (Faros AI, 10,000+ developers)
−35%
how often new code connects to existing functions, since 2023 (GitClear, 623M changes)
3.8%
of changed lines are moved, refactored code in 2026, down from 13% in 2023 (GitClear)

Sources: GitHub Octoverse 2025; Faros AI, The AI Productivity Paradox (a vendor study; correlations across teams); GitClear, June 2026. And Google's 2025 DORA report: AI adoption "now improves software delivery throughput… However, it still increases delivery instability."

GitClear’s connectivity number is the one I keep coming back to. In their words: “New code is less and less woven into the existing codebase.” Refactoring, the work that keeps a codebase’s shape coherent, has collapsed to under 4% of changed lines. That isn’t a story about bad code line by line. It’s a story about system-level properties degrading while everyone’s attention stays at the line level.

So we measured it ourselves

When I first wrote this post, every number in it was somebody else’s. So this time we looked at the repositories this matters most for: 335 active open-source projects whose own documentation describes their architecture. For each one we read the last 30 merged pull requests, and the files, build and config on the default branch.

335 repositories that wrote their architecture down

30%
of 7,609 merged pull requests show an AI coding agent's involvement (dependency bots excluded)
63%
give their agents docs to work from: AGENTS.md or CLAUDE.md, spec and plan files, or custom agent workflows
22%
keep spec or plan files for agents to build from; 10% use a spec-driven toolkit such as Spec Kit, OpenSpec or Superpowers
8%
of the repos that give agents docs run any tool that checks an architecture rule

Our survey, September 2026. Active public repositories in Java, Python, C# and TypeScript whose docs state architectural rules; not a random sample of GitHub. AI involvement means a co-author trailer naming an AI tool, a pull request opened by a coding agent, or a "Generated with" footer; that is a floor, since tab completion and chat-assisted edits leave no trace. Architecture checks means ArchUnit, import-linter, dependency-cruiser, NetArchTest and similar, detected from default-branch build, config and CI files, with every hit checked by hand.

Put those together. In repositories that went to the trouble of writing down their architecture, AI agents are already in roughly one pull request in three. Repos with an AGENTS.md or CLAUDE.md see even more: 38% of their merged pull requests involve an agent, against 18% in repos without one. Nearly two-thirds hand their agents documents to build from. And of those, 92% run nothing that checks an architecture rule. The agents are building from the docs, and nothing checks that the code still matches them.

The practices that scale themselves, and the one that doesn’t

Most of the classic practices have a guardian that scales automatically. Style has linters. Correctness has tests and CI. Even readability has help now, since coding assistants are genuinely good at naming and idiom. Crank the volume up and these hold the line, because the enforcement is mechanical and per-file.

But look at what’s left unguarded:

Every practice has a guardian. Except one.

PracticeWho enforces itHolds as volume grows?
Consistent style & formattingLinters, formatters✓ Automatic
CorrectnessTests, CI, type systems✓ Automatic
Readable code, good namesReview norms + coding assistants✓ Mostly
Small, focused diffsTeam norms✓ If you insist
What your docs say about the codeNobody✗ Goes stale silently
Dependency direction & boundariesNobody, in 93% of the repos above✗ Erodes silently
Modules keeping their internals privateNobody✗ Erodes silently
No dependency cyclesNobody✗ Erodes silently

The unguarded rows share a property: they're global. You cannot check them by looking at one file, one diff, or one PR. They exist only in the relationships between components, which is exactly what per-file tooling can't see.

The practice with no guardian is architecture: what your docs say the system is, which component depends on which, whether boundaries hold, whether the shape is drifting. In a docs-first workflow that is also the practice the agents lean on hardest, because the architecture doc is what they read before they touch anything.

When the docs are the program, drift is a bug

A stale doc used to cost a confused new hire an afternoon. Now it costs every agent run that reads it, and agents do what the docs say. ETH Zurich’s study of AGENTS.md files found that “instructions in the context files are well followed by coding agents.” An engineer who documented a 108,000-line codebase for agents reported what happens when those instructions go stale:

"Outdated context documents caused agents to generate code that conflicted with recent refactors." And: "Agents trust documentation, and out-of-date specs can mislead sessions and lead to silent failures."
Codified Context, arXiv 2602.20478

Böckeler saw it in her own tests: given notes describing classes that already existed, the agent “took them as a new specification and generated them all over again, creating duplicates.” And keeping the instructions current measurably matters: in a study of AI IDE rule files, compliance rose “from 49.14% to 72.13%” after the rules were updated (arXiv 2606.12231). Meanwhile, in a sample of 100 context files, 24% had been generated once and never reviewed again (arXiv 2606.15828).

And the docs are drifting faster than anyone is watching them:

Output scaled. Oversight didn't.

OutputOversight
+23% pull requests merged on GitHub per month, year over year+91% PR review time on high-AI-adoption teams
+98% pull requests merged on high-AI-adoption teams4.6x longer wait before an AI-generated PR gets reviewed
+154% average PR size on the same teams8% of repositories that give agents docs check any architecture rule

Sources: Octoverse 2025; Faros AI; LinearB 2026 benchmarks (8.1M pull requests; a vendor study); our survey above. Writing stopped being the bottleneck. Noticing what the writing did to the system still is.

The mechanism is simple. A diff shows you lines. It does not show you that those lines made a sentence in your own README false, or created the first-ever edge from your core into a plugin. That information lives in the relationship between this change and everything around it: other files, other documents, every change before it. It is structurally absent from the thing your reviewers are reading. Here is a real one:

The same pull request, two representations

What review sees multithreads/NodeWorker.java · +9 −150 − import …repair.scheduler.RepairScheduler; − private final RepairScheduler myRepairScheduler; + private final SchemaRefresher mySchemaRefresher; + mySchemaRefresher.onTableCreated(myNode, tableEvent); ✓ Review approved ✓ Merged A clean refactor. Nothing to flag. What the README still says core.impl/README.md · not in the diff 133 NodeWorker — A continuously-running background thread… 134 - Discovers all replicated tables for its node 135 - Fetches repair configurations per table 136 - Calls RepairScheduler.putConfigurations() to keep jobs up to date 137 - Loops on a configurable refresh interval ✗ Violated: NodeWorker depends on RepairScheduler True at the base revision, false after this change

Ericsson/ecchronos #1786, a real pull request. The call moved to a new class, SchemaRefresher; the sentence on the right did not move with it. It sits in a file the diff does not contain, so no amount of careful diff-reading surfaces it, and as of September 2026 it is still on master. The next agent asked to work on NodeWorker reads that sentence first. The whole story.

This is not a story about careless review. That pull request was reviewed and approved by people who are good at their jobs. The information simply was not in front of them.

The bill comes due quietly

The failure mode isn’t dramatic. Nothing crashes. The docs drift one clean-looking PR at a time, the agents keep building from them, and the codebase accumulates coupling, cycles and duplicated responsibilities until the symptoms surface as things nobody connects back to architecture: builds got slow, onboarding takes months, every estimate has a fudge factor.

Teams that ship faster while their structural oversight stays flat aren't being efficient. They're borrowing. The loan comes due as a system that technically passes every check while becoming harder to change every week, and by the time it's obvious, the cheap moment to fix it is hundreds of merges in the past.

So what do you actually do?

Keep every practice you already have, and keep writing the design docs, specs and plans: that is the right way to work with agents. But be honest about the gap. If the docs are what your agents build from, the docs have to stay true, and in 92% of the repositories that hand their agents docs, nothing checks that they do.

Closing that gap doesn’t mean hiring architects to trace dependencies by hand, and it certainly doesn’t mean slowing your team down to pre-AI speed. It means giving the one unguarded practice the same thing every other practice already has: an automatic, per-PR guardian.

That’s what Striff is. It reads the architecture your docs already describe, turns every checkable sentence into a rule, and evaluates each one at both revisions of every pull request. A rule can be as plain as where a class lives or as sharp as “the domain module must not depend on infrastructure”: if your team wrote it down, the pull request that breaks it is told which sentence it broke, quoted from the file it lives in. Every pull request also gets a diagram of what changed.

The bar is deliberately high, and the consequence is that it is quiet. That is the same bargain your linter makes: you trust it because it does not shout. Your linter guards style, your CI guards correctness, and the documents your agents build from finally get a guardian of their own, at whatever speed your team ships.

Install the GitHub App and open your next pull request.