Why a Clear Spec Beats a Bigger Model: Testing Claude Haiku 5.5 on a PRD
Can a cheap AI model like Claude Haiku 5.5 build real product features?
Can a small, cheap model build production-grade features? Our working answer is yes, but only when the spec does the heavy lifting. Claude Haiku 5.5 cuts Anthropic's small-model pricing by roughly 75%. That's a real change for founders. But token price was never what decided whether software shipped.
Anthropic built Haiku 5.5 for high-volume, cost-sensitive agent work, and founders noticed. It was one of the top AI stories on Hacker News this week, at about 1,000 points. The interest is reasonable. If a model costs a quarter of what it used to, a founder on a tight budget can afford to run four times as many experiments.
So we're testing it. At Codalio, we're running Haiku 5.5 against one of our own real PRDs to answer a single question: does a cheap model plus a clear spec beat an expensive model plus vibes?
This post lays out the test, what "held up" has to mean, and why we expect the spec to decide the outcome more than the model.
Why the model's price was never the real cost of building software
If you're a founder who has tried an AI app builder, you've probably seen this happen. You describe the product in a paragraph. A few minutes later something appears on screen, and it looks close. Then you spend three weeks explaining what you actually meant.
The tokens were the cheap part. The expensive part was everything after: re-prompting, patching, finding out the login flow ignores the admin role, realizing "invoice" means three different things on three different screens. That rework is where the time and money go, and a price cut on the model doesn't touch it.
Large models are good at hiding this. They fill gaps with plausible guesses, so a vague prompt still produces something that demos well. A bigger model doesn't remove the ambiguity in your requirements. It just guesses about it more convincingly.
A smaller model's guesses are easier to spot, which makes it a more honest test. If your requirements are thin, a model like Haiku 5.5 will show you where. If they're solid, the guessing mostly stops mattering because there's little left to guess.
That's the bet behind this experiment, and it's the bet Codalio is built on. Vibe coding ships prototypes. Spec-driven ships products. If that holds, a cheap model working from a strong spec should do far better than its price suggests.
What a spec has to contain before a small model can build from it
Codalio starts from a simple position: business logic belongs in the spec, not in the model's head. Once the requirements are explicit, the model's job shrinks from "work out what this founder wants" to "build what's written down." Small models are much better at the second job than the first.
The PRD in this test is a real Codalio document, not a toy example. These are the five things it spells out, and we think any spec needs them before a cheap model can build from it reliably:
-
Users and roles. Who can do what, and what each role is explicitly not allowed to do.
-
The data model. The core entities, how they relate, and which fields are required.
-
Business rules and edge cases. What happens when a payment fails, a record is deleted, or two people edit the same thing.
-
Acceptance criteria. For each feature, the observable behavior that counts as "done."
-
What's out of scope. The features the model should not invent. This matters just as much as the rest.
"Held up" can't just mean "the code runs." Any model can manage that. It has to mean the output matches the spec: roles enforced correctly, data relationships intact, edge cases handled the way the document says, and nothing built that wasn't asked for. That's the same standard we'd hold a human engineer to.
So is the model the hard part? Building Codalio has taught us it usually isn't. We'll publish the results feature by feature, including where Haiku 5.5 falls short, because the failures will tell us as much about the spec as about the model.
How non-technical founders can test a cheap model against their own spec
You can run your own version of this before our results are out. If you're a non-technical founder choosing which AI tools to trust with your product, the method works the same way at your scale.
Write the spec before you open any tool. Use the five areas above as your outline. If you get stuck on business rules, that's useful: you've found the work that actually needs doing, and no model will do it for you.
Test one feature, not the whole app. Pick something with real logic, like checkout, permissions, or scheduling. Give the same written spec to a cheap model and an expensive one. Judge both against your acceptance criteria, not on how polished they look.
Measure rework, not the first draft. Count how many follow-up prompts each model needed before the feature matched the spec. That count is your real cost, and most founders never track it.
If the cheap model gets close on a strong spec, you've just made your experimentation budget go a lot further. If it falls apart, check your spec before you blame the model, because the gap is usually in the document. That's the thinking behind Codalio: put the rigor into the requirements so every model and every engineer downstream builds the same product.
Codalio is the only AI platform that takes non-technical founders from idea to PRD to working prototype, without the rebuild cycles that come from skipping the spec.
Start with the spec, not the model
Cheaper models make the case for spec-driven building stronger. Codalio exists to help founders write the spec that gets any model, cheap or expensive, to build the right thing.
Our full Haiku 5.5 results, including what held up and what didn't, are coming in a follow-up post.
References
-
Anthropic: Claude Haiku 5.5 announcement (anthropic.com/claude-haiku-5-5)
-
Hacker News discussion of the Haiku 5.5 release