The biggest models win headlines, but they rarely win on a single narrow task. If your work is extracting fields from invoices, or rewriting support replies in a fixed tone, a small model that has been tuned for that job will often beat a frontier model that has to be talked into it.
Why size stops helping
Parameters buy general knowledge. Once a task is narrow, most of that knowledge is dead weight: it costs memory, slows inference, and adds no accuracy. A model with seven billion parameters that has seen ten thousand examples of your exact format can be sharper than a model with hundreds of billions that has seen everything a little.
The three things that actually decide quality
- Task fit: does the model see your format during training or fine-tuning?
- Instruction shape: a short, explicit prompt with one example often beats a long instruction list.
- Context hygiene: irrelevant context measurably degrades small models faster than large ones.
What you give up
Small models are worse at handling surprises. Ask one to reason across three unfamiliar documents and it will confidently miss the thread. They also forget longer conversations sooner. The trade is real, and it is worth naming before you commit.
Where small models win outright
Anything that must run offline, anything with a per-request budget, and anything where your data cannot leave the building. A local model costs nothing per call once it is running, which changes which features are economically possible.
A practical test
Take twenty real examples of the task. Run the small model and the frontier model on the same prompt, blind-score the outputs, and count how many you would ship without editing. If the small model reaches the same useful rate, the larger one is costing you money for reassurance.
Bottom line: choose the smallest model that clears your accuracy bar on your own examples. Parameter count is a marketing number until it is measured against your task.
