How to cut LLM API costs

What drives an LLM API bill?

Token volume and model choice drive most of the bill. OpenAI’s cost optimization guide names three strategies for lower cost and latency, which are fewer requests, fewer input and output tokens, and a smaller model. Anthropic prices every model per million input and output tokens, with separate rates for cache writes and cache reads (Anthropic prompt caching).

This guide covers the levers both providers document. For per-model rates, see OpenAI API pricing and Anthropic API pricing.

Which cost levers do OpenAI and Anthropic offer?

Both providers offer smaller models, prompt caching, batch processing, output limits, structured outputs and spend controls. OpenAI also documents flex processing. The table maps each lever to the provider feature and the discount as each provider states it.

Lever	OpenAI feature	Anthropic feature	Discount as officially stated	When it fits
Right-size the model	”Select a smaller model” (cost optimization)	Start efficiency-first with Claude Haiku 5.5, or tune the effort parameter (choosing a model)	Depends on the models compared. See the pricing pages.	High-volume, simple tasks
Prompt caching	On by default for supported models (prompt caching)	cache_control, automatic or on chosen blocks (prompt caching)	OpenAI cached input “discounted up to 95%”. Anthropic cache reads at 0.1 times base input on most models.	Long prefixes sent again and again
Batch processing	Batch API (batch)	Message Batches API (batch processing)	OpenAI “50% cost discount”. Anthropic “50 percent of the standard API prices”.	Jobs that can wait for results
Flex processing	service_tier: "flex" (flex processing)	Not covered in the Anthropic docs reviewed	Priced at Batch API rates, per OpenAI	Low-priority work that can tolerate delays and retries
Shorter prompts and outputs	Minimize tokens (cost optimization guide), lower max_tokens	max_tokens on every request	No rate discount. Fewer tokens are billed.	Every workload
Structured outputs	JSON Schema responses (structured outputs)	Constrained decoding (structured outputs)	No rate discount. Fewer retried calls.	Apps that parse model output
Spend limits and alerts	Organization and project spend limits (spend limits)	Organization spend limit and workspace spend limits (rate limits, workspaces)	No rate discount. Caps monthly spend.	Any team with a budget

How much can a smaller model save?

A smaller model saves the difference between two models’ token rates, so the saving depends on which models you compare. OpenAI lists “select a smaller model” as a core strategy and says to use models that keep accuracy while cutting cost and latency (OpenAI cost optimization).

Anthropic describes two ways to start (choosing a model):

• Efficiency-first means building with Claude Haiku 5.5, testing, and upgrading only for specific capability gaps. Anthropic recommends it for cost-sensitive and high-volume, straightforward tasks.
• Capability-first means starting with Claude Opus 5.5 and moving to more efficient models later.

Anthropic also says tuning the effort parameter on supported models “is often a better lever than switching models.” Run your own evaluation set before and after any switch, so a cheaper model doesn’t quietly cost you quality.

How does prompt caching lower costs?

Prompt caching bills a reused prompt prefix at a lower input rate. Each provider sets it up and prices it differently.

OpenAI. Caching is on by default for supported models, and cached input is “discounted up to 95%” (OpenAI prompt caching). The rules depend on the model:

• On GPT-5.6 and later, a prefix must reach 1,024 tokens to be cached. Cache writes cost 1.25 times the standard input rate for the default 30-minute lifetime, or 2.5 times for a 12-hour lifetime in the Responses API. Reads cost 0.1 times standard input on most of these models and 0.05 times on GPT-6.1 Sol and GPT-6.1 Terra.
• Earlier models use implicit caching only, with no additional cache-write charge.
• The cache matches the whole rendered prefix. Changing the model, tools, the structured output schema or other listed settings ahead of the cached point prevents a match.

Anthropic. Add cache_control once at the top level of a request for automatic caching, or place it on individual content blocks (Anthropic prompt caching). Pricing follows these multipliers of the base input price:

• Cache writes with the default 5-minute lifetime cost 1.25 times base input.
• Cache writes with a 1-hour lifetime cost 2 times base input.
• Cache reads cost 0.1 times base input. Anthropic lists lower read rates for some models in its pricing table.
• The multipliers stack with other pricing modifiers, including the Batch API discount.

For both providers, put stable content such as instructions, tool definitions and reference documents first, and put content that changes on every request last. A timestamp or user message near the top breaks the match and turns every request into a cache write.

When should you use batch processing?

Use batch processing for work that doesn’t need an immediate answer, such as evaluations, classification and bulk generation. Both providers price batch work at half their standard rates.

• OpenAI Batch API. Requests go in a .jsonl file with a “50% cost discount compared to synchronous APIs”, a separate pool of higher rate limits, and completion within 24 hours (OpenAI batch). Each input file holds requests for one model only.
• Anthropic Message Batches. All usage is charged at 50 percent of standard API prices. Most batches finish in under an hour, and a batch expires if processing doesn’t finish within 24 hours. A batch holds up to 100,000 requests or 256 MB (Anthropic batch processing).

Anthropic suggests the 1-hour cache lifetime for batches that share context, because batches can take longer than 5 minutes. It also warns that batches can go slightly over a workspace spend limit because of their high throughput.

What is OpenAI flex processing?

Flex processing is an OpenAI service tier that trades speed and availability for a lower price. Set service_tier to flex on a Responses or Chat Completions request, and tokens are priced at Batch API rates, with additional discounts from prompt caching (OpenAI flex processing).

The conditions matter:

• Flex is in beta with limited model availability. Supported models are listed on OpenAI’s pricing page.
• Responses are slower, so OpenAI suggests raising the SDK timeout above its 10-minute default.
• When capacity runs short, requests return a 429 Resource Unavailable error, and OpenAI says you are not charged for those.

Flex fits evaluations, data enrichment and other asynchronous work where you want a regular API call instead of a batch file.

How do shorter prompts and outputs reduce the bill?

Both providers bill each token you send and each token the model generates, so fewer tokens cost less on any model. OpenAI’s guidance is to cut input tokens and aim for shorter outputs (OpenAI cost optimization).

Practical steps:

• Remove repeated instructions, unused tool definitions and old conversation turns you no longer need.
• Ask for the length you want in the prompt, and set max_tokens as a ceiling. OpenAI notes that requests with a lower max_tokens also see less latency (production best practices).
• Watch n and best_of on OpenAI requests. Values above 1 generate several outputs per request.

On Anthropic, output rate limits count only the tokens actually generated, and max_tokens doesn’t factor in (Anthropic rate limits). The ceiling exists to stop runaway answers, not to reserve capacity.

Do structured outputs save money?

Structured outputs save money by cutting retries. If a response fails to parse and you send the request again, you pay for both calls.

• OpenAI says Structured Outputs make the model always follow your JSON Schema, so there’s “no need to validate or retry incorrectly formatted responses” (OpenAI structured outputs).
• Anthropic says its structured outputs guarantee schema-compliant responses through constrained decoding, with “no retries needed for schema violations” (Anthropic structured outputs).

On OpenAI, the output schema is part of the cached prefix, so keep the schema stable if you also rely on prompt caching (OpenAI prompt caching).

How do spend limits and alerts protect a budget?

Spend limits don’t lower any price, but they stop a bug or a traffic spike from running up an unplanned bill. Alerts notify you while traffic continues, and hard limits stop traffic.

OpenAI (spend limits):

• Set a monthly spend limit for the organization or for one project.
• Turn on “Enforce a hard limit” to make affected requests fail with a 429 error (organization_spend_limit_exceeded or project_spend_limit_exceeded).
• Enforcement isn’t instantaneous, so recorded spend can slightly exceed the limit.
• Spend alerts send a notification and leave traffic running.

Anthropic (rate limits, workspaces):

• Each usage tier carries a monthly spend cap. When an organization reaches it, API usage pauses until 00:00 UTC on the first day of the next month unless it gets a higher limit sooner.
• Set your own lower limit in the Spend limits section of the Billing page in the Claude Console.
• Set spend limits and alerts per workspace on the Workspaces page. Anthropic notes you can’t set spend limits on the Default Workspace.

How should you monitor usage while you cut costs?

Check usage after each change, so you can see which lever moved the bill. OpenAI has a usage dashboard, and its prompt caching guide points to a Prompt Caching Dashboard that shows cache read hit rates. Anthropic’s Usage and Cost Admin API returns data similar to the Console’s Usage and Cost pages, and Anthropic lists rate limit optimization, including tuning prompt caching, among its uses (Anthropic Usage and Cost API).

For dashboards, cost APIs, per-project keys and receipts, see how to track AI API spend.

For OpenAI alone, read how to cut OpenAI API costs on the ernd blog. To compare models at list prices, try the AI API cost calculator.