Cost Engineering for AI Coding: The Cheapest Model Isn't Always the Cheapest Solution

In the previous article, I made a case that AI coding is becoming an engineering discipline. One principle kept surfacing while writing it: better models don't replace better thinking.

The natural next question is: what does that actually look like day to day?

Part of the answer lies in a cost most engineers don't think about until it starts hurting—and it isn't the API bill.

My First Mistake Was Looking at the Wrong Number

When I first moved from a subscription-based AI tool to pay-as-you-go models, I started optimizing for API cost because it was the easiest number to measure.

Over time I realized I was spending more time trying to make one model work than simply choosing the right tool for the task.

The API bill stayed low, but the afternoon disappeared.

I was optimizing the visible number while ignoring the expensive one.

The Cheapest Model Isn't Always the Cheapest Solution

One assumption I carried into pay-as-you-go models: use the cheapest model possible, reduce costs naturally.

Reality is more nuanced.

Where cheap models work well:

• CRUD endpoints and boilerplate
• Simple automation scripts
• Small, isolated refactors
• Basic test generation

Where they become expensive:

At some point a model hits the limit of what it can reason through — and instead of stopping, it keeps trying the same approach in slightly different ways. The output gets worse. The time spent grows.

The API bill stays low. The engineering cost doesn't.

The cheapest model becomes the most expensive solution.

The goal isn't to use the cheapest model. The goal is to complete the task with the lowest overall engineering cost.

The Most Expensive Model Isn't Always the Best Investment

The opposite is equally true.

Premium models cost more for a reason — larger context windows, better reasoning, more reliable output on hard problems. Those capabilities are genuinely valuable for:

• Architecture discussions
• Complex debugging sessions
• Unfamiliar domains
• Recovery from compounding failures

But using a frontier model to rename variables or generate boilerplate isn't engineering. It's overspending.

Like every engineering decision,¹ model selection should be proportional to the problem.

Context Is One of the Biggest Hidden Costs

Most of my token spend wasn't coming from long prompts. It was coming from context that quietly accumulated over the course of a session.

Entire files. Long conversations. Stack traces. Logs. Documentation. All of it gets pulled into the context window as a session grows.

Every additional piece of context has a cost — and it's not just financial:

• Latency increases — more tokens to process
• Quality drifts — the model has more noise to reason through
• Cost compounds — every message in the session carries the full context forward

I've become much more deliberate about compartmentalising work. A conversation should contain only the information necessary to solve the current problem.

Switching Models Is Often Cheaper Than Continuing

This has probably been the most significant shift in my workflow.

I used to keep iterating with the same model — rewriting the prompt, adding more context, explaining it differently, trying once more.

Changing prompts doesn't necessarily change the reasoning. Changing the model often does.

I've found it more productive to switch models earlier rather than keep spending — in both time and tokens — trying to salvage a session that's going nowhere.

The most expensive part of my workflow wasn't the model. It was me, waiting for the wrong model to figure something out.

Switching isn't admitting failure. It's choosing a different engineering tool for the job.

Cost Depends on Workflow More Than Pricing

While researching this article, I ran a small structured experiment: five models, three task types (build, bug fix, refactor), fifteen total evaluations, each scored across nine engineering dimensions by an independent judge model.

The most interesting observation wasn't which model produced the best code — it was how differently they approached the work.

Three observations stood out.

Task complexity mattered more than model pricing. One of the lowest-cost models outperformed a significantly more expensive frontier model on a well-defined implementation task.

Consistency was more valuable than peak performance. Some models behaved predictably across all three task types, while others excelled in one scenario but struggled in another.

Refactoring was the great equaliser. Across every model I evaluated, refactoring proved more challenging than greenfield implementation or bug fixing. Workflow discipline mattered more than raw capability.

Cost isn't determined by the model alone. It's determined by the workflow surrounding it.

The model is only one variable in the system. Engineering decisions determine how effectively that capability is used.

What Cost Engineering Means to Me

I've stopped thinking about AI costs as a pricing problem.

Instead, I think about balancing five engineering variables:

Optimising one without thinking about the others usually backfires. Good cost engineering is about balancing all five.

Practical Habits That Reduced Both Cost and Friction

These are the habits I've found consistently useful:

• Match the model to the task. Routine work doesn't need a frontier model. Complex reasoning shouldn't be handed to the cheapest option.
• Keep sessions narrow. One problem per conversation. Only the context that's directly relevant. Everything else is cost and noise.
• Switch earlier than feels comfortable. When progress stalls, switching models is usually faster than another round of prompt rewrites.
• Measure engineering output, not token spend. A cheaper session that took twice as long isn't cheaper.
• Spend more where it saves more. A frontier model that solves a problem in one pass can cost less overall than a cheaper model that takes six.

None of these require a different AI model. They require different decisions.

Closing Thoughts

The easiest metric to measure is API cost. The hardest is engineering time — and it's usually the one that matters more.

I've stopped asking which model is the cheapest. Instead, I ask:

Which decision gives me the best return for the time and money I'm about to spend?

Cost engineering isn't about spending less.

It's about making better engineering decisions with every dollar and every minute you invest.

Engineering Notes

The data and experiments behind this article are published separately as engineering notes — the raw numbers, per-model breakdowns, and evaluation transcripts are there if you want to dig in.

• AI Coding Cost: What 8 Days of Data Actually Showed — the usage statistics and where the spend was actually hiding
• Model Evaluation Experiment: Does the Expensive Model Win? — five models, three task types, fifteen evaluations, $0.39 total cost