Six Patterns to Control LLM Cost — From $4,200/mo to $890

First-month LLM bill — $420. Small. Six months later — $4,200. Same user count, same quality. Cost only — 10×.

Six weeks of cost-control work and the bill was $890. Same users, same quality. This post is the breakdown of those six patterns.

Pattern 1 — model ladder

We initially used the strongest model (GPT-5 / Claude Opus tier) for every call. Reality — 60% of calls were simple classification — Haiku / Mini was enough.

Fix: map call type to model.

Call type                    | Model
─────────────────────────────┼──────────────
Simple classification / labels | Haiku / Mini
Mid-complexity reasoning      | Sonnet / GPT-4 mid
Complex decisions / codegen   | Opus / GPT-5

That single change cut cost -45%.

Pattern 2 — prompt caching

Our system prompt was repeating across every call. Enable prompt caching (Anthropic / OpenAI both support it).

The pattern of long system prompt (thousands of tokens) + short user input is 80% of our usage. Caching it gives a 90% discount on the repeated portion.

Cost -25%. Work 2 days.

Pattern 3 — context window trimming

We were sending the entire conversation history to every next turn. The last 5 turns + episodic-memory retrieval was sufficient. Long history also hurt accuracy (lost-in-the-middle).

→ Agent memory — what to remember and forget

Cost -15%. Accuracy +5% (unexpected bonus).

Pattern 4 — decision-step caching

Discovered a pattern of repeated calls with identical inputs. When users retry the same action in succession, the same LLM call ran dozens of times.

Fix: cache decision results keyed by input hash with a short TTL. Same input within 5 minutes → cache hit.

Cost -12%. Work 3 days.

Pattern 5 — streaming saves tokens

With streaming responses, when the user cancels mid-stream, the unsent tokens are not billed. In our usage, 15% of users see only part of the answer and cancel.

Fix: switch all long-form responses to streaming. On user cancel, abort the token generation server-side.

Cost -7%. Work 1 week.

Pattern 6 — deterministic fallback

High-confidence simple decisions don't need LLM at all — deterministic rules suffice. LLM only for uncertain decisions.

Example — if the request is clearly a known intent (e.g., "list my deploys"), no LLM call. Direct API.

Cost -5%. Work 4 days. Bonus — latency improved 70%.

Six-pattern accumulation

Start:                       $4,200 / mo
Pattern 1 (model ladder):    $2,310 (-45%)
Pattern 2 (prompt cache):    $1,733 (-25%)
Pattern 3 (context trim):    $1,473 (-15%)
Pattern 4 (decision cache):  $1,296 (-12%)
Pattern 5 (streaming):       $1,205 (-7%)
Pattern 6 (deterministic):   $1,145 (-5%)
─────────────────────────────
Actual cumulative:           $890 (total -79%)

The percentages aren't multiplicative — they're sequential, with interactions. After Pattern 1, Pattern 2's saving rate differs. The actual cumulative landed at $890.

The one-line principle

What the six patterns share:

Not every LLM call has equal value. Spend less on low-value calls and exactly the right amount on high-value ones.

We were spending the same cost on every call — that's the source of $4,200. Allocating by value produced $890.

If your LLM bill is growing fast and you've applied fewer than three of the six patterns above, six weeks of work can cut 70–80%. Order matters — model ladder always first, the largest lever.