Never bake a model into your logic

Model choice does not appear anywhere in my business logic. Every call goes through a proxy service, the proxy decides which model handles it, and the proxy is built provider-agnostic so OpenAI, Anthropic and Google are all pluggable behind the same interface.

Every system then exposes model configuration in an admin panel. Changing which model handles what is a config change, not a code change, which means it is not a deploy either.

The part most people miss is what the dial is attached to. You do not route by model, you route by task class. Analysis, text extraction, context extraction, deep research: each one gets its own assignment. Frontier reasoning for the hard calls, something cheap and fast for extraction. Once the taxonomy is the unit, swapping models stops being a judgement about which model is best overall, which is an unanswerable question, and becomes a judgement about which model is best at one narrow job, which is a question you can actually run.

The receipt is unglamorous and that is the point. When Opus 5 launched I changed the model name in a dashboard. That was the entire migration. And the reverse case matters at least as much: when reports come in that a new model is underperforming on something, I am back on the old one in the same thirty seconds.

The objection I get is latency. You have added a hop. True. The proxy is also where fallback logic lives, so a provider outage, a rate limit or exhausted credits routes around itself instead of taking the product down. I have never measured the proxy overhead because I have never been able to perceive it in usage testing, and I have never found a reason to go looking. An unmeasured, imperceptible overhead in exchange for surviving the single most common production failure mode is not a close call.

Reasoning effort is the same lever one level down. You do not need the biggest model at maximum effort for every task. High or medium is right for most of them. Reaching for max by default is the same mistake as reaching for the frontier model by default, just harder to see on the invoice.

Because the proxy is a chokepoint, instrumentation falls out of it almost free. I get cost per completed task, broken down by task class, which is the number that actually belongs on a P&L, rather than cost per token, which tells you nothing about whether the work got done. I keep rolling one, seven and thirty day averages, so when I swap a model I can watch the extraction cost curve move and either accept it or revert. Token usage per call feeds straight into the customer billing table for usage-based pricing.

People ask why I do not just use LangSmith or Helicone. No strong objection to either. The proxy already sits at the chokepoint, so the instrumentation is nearly free once it exists, and I get exactly the task taxonomy I care about instead of a generic one.

This is running in Pericls, Hiveclaw, Strata, Tessera and Panio. All of them, now.

The one thing the pattern cannot absorb is an entirely new model family with a different interaction pattern. Realtime voice models are the obvious example: that is not a swap, that is a new integration. But if the model layer is already its own service, you are changing one place instead of forty.

“Route by task class, not by model. That is the bit most people miss.”