Spec-Driven Development: is it the solution to Vibe Coding?
Spec-driven development, tested
Someone posted spec-driven development on LinkedIn this week as the answer to vibe coding — to prompting an agent, half-understanding what you're building, and ending up with code you can't vouch for. The linked toolkit has 127,000 stars and comes from GitHub itself. The pitch lands.
So I installed it and pointed it at a deliberately trivial task. One of the three principles it wrote for me was a dependency policy I never asked for — hold that thought.
Twenty minutes isn't a verdict, though. Two engineers have tested this properly, on real problems, long enough for the seams to show. They used different tools, on different continents, seven months apart — and both reached for the same comparison, unprompted: the last time our industry tried to generate working code from documents. On the one question that decides whether any of this survives contact with AI features, they flatly contradict each other. Neither has a measurement.
What spec-driven development is
If the term is new to you, the idea is simple. Instead of prompting an agent and iterating until the code looks right, you write a structured specification first — what you're building, why, and what "done" means — and the agent works from that document rather than from your prompt. The spec, not the code, becomes the thing everyone points at.
Spec Kit is GitHub's implementation. It installs into a repo once, not per task, and gives
your agent a set of commands: constitution to set project principles, then specify, plan,
tasks, implement. Nothing runs in the background — it drops templates and command
definitions into your project, and from then on you're talking to your agent. Closer to a linter
config than a product.
Before any of that, though, comes constitution — written first, before anyone has seen the
problem, and everything downstream gets judged against it. Hold on to that too.
The setup step tells you what it's really aiming at:
Thirty-plus agents. This isn't a GitHub-only tool — it wants to be the layer every coding agent plugs into, which goes a long way to explaining the star count.
What actually happens when you use it
Birgitta Böckeler, Distinguished Engineer at Thoughtworks, trialled three of these tools by hand — Kiro, Spec Kit and Tessl — and published what she found in October 2025.
She asked Kiro to fix a small bug. It produced four user stories and sixteen acceptance criteria, including — verbatim — "As a developer, I want the transformation function to handle edge cases gracefully, so that the system remains robust when new category formats are introduced." Her summary: "like using a sledgehammer to crack a nut."
On Spec Kit with a real feature — a few days' work, by her own estimate — she never finished the implementation, and reckons she could have built the thing by hand in the time she spent reviewing artifacts. The line that will land with anyone who has done a code review:
To be honest, I'd rather review code than all these markdown files.
She also found the agent ignoring the documents meant to steer it. Spec Kit's research step correctly catalogued existing classes; the agent then read those descriptions as a specification and generated the classes again, as duplicates. And the reverse failure — the agent going "way overboard because it was too eagerly following instructions (e.g. one of the constitution articles)."
Alex Punnen went narrower and deeper, and published the whole transcript with line citations. Spec Kit v0.8.9, on a problem at real scale: querying US elevation data across 1,756 map tiles — 23 billion measurements, 180GB.
His finding is subtler than "the tool invents things." He asked for principles covering code quality, testing, consistency and performance, and says plainly: "The principles themselves are reasonable." What went wrong, he argues, is that one of them quietly tilted every later decision:
New dependencies MUST be justified in writing: problem solved, alternatives considered, license verified.
Sensible in isolation. But as Punnen puts it, "the stdlib option always wins ties because it costs zero justification entries." Four phases later the plan chose SQLite because — first reason listed — "Stdlib, zero new dependency… that's the cheapest possible answer." Three of the four rejected alternatives fell to that same rule. His verdict: "The constitution did the rejecting; the agent was just the microphone."
Then comes the part I found hardest to shake. Earlier in the process the agent had written a five-minute performance budget into the spec — a number it guessed, with no measurement behind it, filed under the label SC-008. Later, that guess came back as the reason a better design couldn't be used: "The transcode cost blows past SC-008." Only under direct pushback did it concede: "You're right that I overweighted reason #2." The better design, Punnen writes, needed nothing that wasn't already in the spec, "except the willingness to revise an arbitrary number the spec itself produced."
His summary of that phase applies to the whole category:
The artefact looks done because every template slot is filled — not because the engineering question is answered.
Two things make this more than a one-off. His constitution prompt was essentially GitHub's own documented example, lightly adapted — he followed the quickstart. And the resistance he hit is partly by design: GitHub's methodology document calls the constitution "a set of immutable principles," with a section headed "The Power of Immutable Principles."
To be precise, the five-minute budget was not a constitutional principle — it was a success criterion the agent generated downstream of one. The document never claims those are immutable. But the number carried that authority anyway, and it took a human to dislodge it. The methodology argues for fixing principles and says nothing about what happens when guesses derived from them inherit the same standing.
The three levels nobody agrees on
Böckeler's most useful contribution is a distinction the rest of the debate skips. "Spec-driven development" covers three different practices:
Her verdict on where the tools actually sit: "All SDD approaches and definitions I've found are spec-first, but not all strive to be spec-anchored or spec-as-source." Including Spec Kit. GitHub's methodology aspires far higher — "Specifications don't serve code—code serves specifications" — but Spec Kit creates a branch per spec, so a spec lives for the lifetime of a change request, not a feature. Her conclusion: "spec-kit is still what I would call spec-first only, not spec-anchored over time."
Worth knowing that IBM published the identical three-level taxonomy seven months later, footnoted. If you've seen it credited to IBM, it's hers.
What I got on a trivial task
I ran Spec Kit at commit 83883a2 on a deliberately minimal prompt — "Principles for a small
Python utility. Keep it minimal — I have no strong constraints."
One of the three principles it wrote:
### II. Minimal Dependencies
Prefer the Python standard library. A third-party dependency MAY be added only when it
removes clearly more complexity than it introduces, and MUST be recorded in the project's
dependency file (e.g. `requirements.txt` or `pyproject.toml`).
A dependency policy, written in the MUST/MAY language of a formal standard, from a prompt where I said I had no constraints. It isn't in the local template or the skill file — but Article I of GitHub's nine constitutional articles does ask for implementations "with clear boundaries and minimal dependencies." So it's consistent with the published philosophy rather than invented on the spot. I can't tell you the mechanism, only what went in and what came out.
One run, one trivial task, and it cost me nothing because nothing was at stake. Punnen's case shows what this kind of bias costs when the problem is hard enough for it to be wrong. Mine only shows it turns up unasked — which matters because, as Böckeler notes, Spec Kit's constitution is its memory bank, "a very powerful rules file" applied to every change. An unrequested preference doesn't sit in a document you'll discard. It becomes a standing rule.
Both of them reached for the 1990s
Here's what convinced me this is worth taking seriously rather than dismissing or evangelising.
Böckeler, who worked on model-driven development early in her career, sees MDD:
I wonder if spec-as-source, and even spec-anchoring, might end up with the downsides of both MDD and LLMs: Inflexibility and non-determinism.
Punnen, twenty years in telecom, reaches independently for Rational Rose and UML — "treated as the silver bullet of its decade: draw boxes, arrows, and diagrams, and the tool would magically turn them into working code."
Neither cites the other. Different tools, different problems, different countries. Both land on the same era: the last time our industry believed a document could be the source and code the output.
Böckeler is careful about the comparison — "I'm not nostalgic about my MDD experience." Her point is that today's tools drop the parts that made MDD painful: you no longer need a special spec language or a purpose-built generator. What she wonders is whether the exchange is a good one, since the old approach at least produced the same output every time.
Punnen names the trap underneath:
The specification has to be very rigorous in the first place, but to become rigorous it needs to be iteratively refined alongside the generated code.
A Catch-22, in other words: on his account you can't write a rigorous spec for a problem you don't yet understand, and understanding arrives while building. He traces the thought back to Fred Brooks, forty years ago: "descriptions of a software entity that abstract away its complexity often abstract away its essence."
Where they disagree — and why it matters to you
On one question these two are in direct opposition.
Böckeler, generating code repeatedly from one Tessl spec: "I have seen the non-determinism in action… an interesting exercise to iterate on the spec and make it more and more specific to increase the repeatability of the code generation."
Punnen: "This is not a major problem in practice. SDD frameworks act as structured prompts, and modern models produce highly consistent outputs when guided by them."
GitHub takes Punnen's side and goes further, claiming "Consistency Across LLMs: Different AI models produce architecturally compatible code." Not the same model twice — different models. No evidence offered.
Two experienced engineers, opposite conclusions, a vendor claim stronger than either, and not a number between them. It matters more than it looks. Keeping a spec and its code in step — the spec-anchored idea — relies on automated tests to catch the drift. That works for deterministic code, where the same input gives the same output.
Point it at an LLM feature and the check stops working. assert response == expected means
nothing when the response differs every run. Unless you swap tests for evals, where each
acceptance criterion becomes a scored assertion: did it classify correctly, did it return valid
JSON against the schema, did it refuse when it should have.
That bridge survives nondeterminism, and it's the harness I've spent several tutorials building on the WEC Inference API. It also makes the disagreement measurable: write the spec, turn its criteria into eval assertions, and run them across a long session to see whether adherence holds or decays.
That's the next post.
If you're going to try it
Punnen's practical takeaways are better than anything I'd invent, and he earned them:
- Treat "no new deps" rules as biases, not neutrals. If the right answer needs a dependency, you'll have to defend it — the framework won't.
- Treat generated success criteria as guesses until an engineer ratifies them. The agent will quote them back at you later as if they were measured.
- Read every clarify menu as a design proposal in disguise. If the option you want isn't listed, that's the failure — not a prompt to pick the best of three.
- Push back during clarify, not during plan. Plans are long, internally consistent, and exhausting to revise.
One of my own: write your principles with a date and a rationale, so a later you can supersede them. IBM suggests treating specs as "stackable versioned artifacts, like architecture decision records" — and an ADR is something you can mark superseded. The methodology calls these principles immutable. Your problem isn't.
Add IBM's cost test, the sharpest practical line any of them wrote: the cost of refining the spec should always be lower than the cost of fixing misunderstandings in implementation. When that balance flips, stop polishing and start building.
And check the org before you install. The repo doing the rounds on LinkedIn is a fork with
eleven stars; the real project is github/spec-kit. That
matters more here than for most tools: Spec Kit's job is writing instruction files your agent
then obeys, and on first run you approve that folder in one keystroke without having read them.
So does it replace vibe coding? Not the way the pitch suggests. It doesn't help when you don't understand your problem — it helps when you understand it and communicate it badly. Those are different failures, and only one of them has a template.
Both found real value in the early phases — Punnen rates the clarification step his most useful of all — and both found the artifacts looking most authoritative exactly where the decisions underneath them were most arbitrary. Punnen: "they are at their most dangerous when they look the most rigorous."
Böckeler reaches for a German compound word for it — Verschlimmbesserung. Making something worse in the attempt of making it better.
The hard part was never writing the code.