How to Estimate AI Request Cost Before Sending

An AI client can estimate request cost before sending, but it usually cannot know the exact final cost in advance.

Why? Because the request may contain a known amount of input while the output length, reasoning usage, cache behavior, retries, tool rounds, and provider-specific accounting are still unknown.

A useful cost estimate therefore looks like a range or scenario, not a fake exact number.

Start with separately priced usage categories

A simple token-only model is:

$$Cost = InputTokens \times InputRate + OutputTokens \times OutputRate$$

Modern APIs can require more categories:

$$Cost = \sum_i Tokens_i \times Rate_i$$

where $i$ may include:

uncached input
cached input
output
reasoning/thinking
image/audio/video usage
provider-specific categories

Not every model has all of these.

Do not hard-code one universal formula

A provider-neutral app should not assume:

all models price input and output the same way

Rates can differ by:

• model;
• provider;
• route;
• modality;
• cache state;
• batch/background mode;
• usage tier or account contract;
• date.

Keep pricing data separate from the request engine.

See Understanding AI API Costs.

Estimate input after assembling the actual request

The user’s newest message is only part of the input.

A real request may contain:

system/developer instructions
project instructions
conversation history
retrieved evidence
tool definitions
tool results
attachments
current user message
provider continuation state

Cost estimation should happen after the context manager has assembled the candidate request.

Otherwise the estimate can be far too low.

Output is a budget, not a prediction

Before generation, you might know the configured maximum output but not how much the model will actually use.

Suppose:

max output = 8,000 tokens

The model might finish after:

600 tokens

or consume most of the budget.

For preflight cost, distinguish:

expected output
maximum possible configured output

Use scenarios instead of one number

For example:

Input estimate: 12,000 tokens
Expected output: 1,500
Maximum output: 6,000

Then show or calculate:

expected request cost
upper-bound request cost under current settings

This is much more honest than displaying $0.014237 before execution when several inputs are uncertain.

Reasoning models add uncertainty

A reasoning model can consume internal reasoning/thinking tokens that are not visible in the final answer.

A high reasoning setting may increase usage even when visible output remains short.

That means a cost estimator may need to consider:

visible output budget
reasoning effort/level
provider-specific reasoning accounting

Do not estimate reasoning cost by tokenizing a rendered reasoning summary. The summary may represent only a fraction of internal usage.

See Reasoning Effort Explained.

Effort levels are not exact token reservations

If an API exposes:

low
medium
high

that usually does not mean:

low = 1,000 reasoning tokens
medium = 4,000
high = 8,000

unless the provider explicitly documents such a mapping for the selected model/API.

For preflight cost, treat effort as a behavioral risk factor and learn from historical usage rather than inventing a deterministic conversion.

Historical usage can improve estimates

If the app has local, privacy-preserving analytics, it can measure:

model
request category
reasoning setting
input tokens
output tokens
reasoning tokens
final cost

without storing the conversation text.

Then an estimator can derive empirical ranges such as:

similar requests at medium effort usually generate 1k–2k output tokens

Avoid presenting those ranges as provider guarantees.

Prompt caching changes input cost, not logical input size

Suppose a request contains 40,000 input tokens, of which 30,000 are a stable cached prefix.

The context is still approximately 40,000 input tokens, but the price may be split:

10,000 uncached input
30,000 cached input

A cost estimator therefore needs both:

logical input size
expected cache classification

See AI Prompt Caching Explained.

Preflight cache prediction is uncertain

You may know that the prompt prefix is stable, but the provider ultimately decides whether it qualifies as a cache hit under its current caching semantics.

Reasons a presumed cache hit may miss include:

• model changed;
• prefix changed;
• tool schemas changed;
• cache expired;
• provider routing changed;
• minimum cacheable length not met;
• unsupported endpoint/model.

A conservative estimator should not assume every eligible token will definitely be cached.

Tool calls can create multiple model requests

An agentic turn may be:

model request #1
→ tool call
→ tool execution
→ model request #2
→ another tool call
→ model request #3
→ final answer

The cost of the user’s “one message” is therefore the sum of several provider requests.

A preflight estimate cannot know the exact number of future tool rounds unless the workflow is deterministic.

Estimate tool-enabled workflows as a budget

A useful policy might define:

maximum tool rounds = 4
per-turn cost budget = X
wall-clock deadline = Y

Then execution stops or asks for confirmation before exceeding the product/user policy.

This is safer than assuming one prompt equals one API charge.

Tool payloads also grow later requests

When a tool returns a large result, that result may be included in the next model call.

For example:

request #1 input = 10k
search result = 20k
request #2 input = 30k+

The second call can cost significantly more than the first.

Use result-size limits and retrieval selection rather than blindly forwarding full tool payloads.

Retries can multiply cost

A failed network request does not always mean the provider did no work.

If the client retries an unknown-outcome generation, it may incur a second charge.

Your cost model should distinguish:

logical user operation
provider attempts

See Designing Reliable AI Retries.

Provider fallback can change the rate table

Suppose a request starts on Model A and falls back to Model B.

The final logical operation can contain usage from both:

attempt on A
+
attempt on B

Do not overwrite A’s usage record when B succeeds.

See Reliable AI Provider Fallback and Model Routing.

Image and audio requests need modality-aware estimates

A multimodal request may include:

text + images
text + audio
text + video

The provider can account for these inputs using modality-specific tokenization or units.

A text-only tokenizer cannot produce an accurate estimate.

If the model/provider exposes an official count/estimate method, use it. Otherwise display uncertainty rather than pretending image bytes map directly to text tokens.

File uploads can have two different cost paths

A file may be:

uploaded to provider storage
then searched/retrieved

or:

read/extracted locally
then injected into context

Those paths can have different billing dimensions.

Separate:

provider storage/indexing cost
model request cost
application infrastructure cost

when applicable.

BYOK changes who pays, not the math

In a BYOK application, the user’s API account is typically billed directly by the provider.

The app can estimate the request, but it does not control the provider’s final invoice.

Therefore the UI should say something like:

Estimated from current published/model-configured rates

rather than presenting the estimate as an authoritative charge.

Pricing data needs provenance

If your app displays cost estimates, each rate should be associated with:

provider
model ID
rate category
currency/unit
source/update timestamp

This makes stale pricing detectable.

Do not silently apply the rate for a similarly named model.

Custom providers may have unknown pricing

A custom OpenAI-compatible endpoint may be:

• free/local;
• self-hosted;
• billed privately;
• proxied through an enterprise contract;
• priced differently from the upstream model.

In that case, cost can be:

unknown

Do not force $0.00 unless the user explicitly configured zero cost.

Separate usage from price

A durable analytics record should store provider usage even when price is unknown:

struct UsageRecord {
    let inputTokens: Int?
    let outputTokens: Int?
    let reasoningTokens: Int?
    let cachedInputTokens: Int?
}

Pricing can be applied later:

struct CostEstimate {
    let amount: Decimal?
    let currency: String?
    let rateVersion: String?
}

This lets you update price tables without changing historical token facts.

Historical cost should preserve the rate used

If a model’s price changes next month, recalculating an old request using the new rate can rewrite history incorrectly.

For a durable “estimated spend” feature, persist either:

• the calculated cost at execution time; or
• the rate/version used for the calculation.

The app can optionally show a separate “cost at current rates” view, clearly labeled.

Use decimal arithmetic for money

Do not calculate money with binary floating point if exact display/accounting matters.

Prefer a decimal type:

Decimal

or an integer representation of a sufficiently small currency unit.

Token rates may be quoted per thousand or million tokens, so normalize carefully.

Example calculation

Suppose, purely as an example:

input estimate         = 20,000 tokens
expected output        = 2,000 tokens
input rate             = Ri per 1M tokens
output rate            = Ro per 1M tokens

Then:

$$EstimatedCost = \frac{20{,}000}{1{,}000{,}000}R_i + \frac{2{,}000}{1{,}000{,}000}R_o$$

If cached input or reasoning uses different rates, add separate terms.

Use actual current provider rates in the app, not hard-coded example values from an article.

Upper-bound estimation needs care

You might calculate:

maximum configured output × output rate

as an upper bound.

But it may still exclude:

• retries;
• multiple tool rounds;
• provider fallback;
• background-job duplication after ambiguous failures;
• separate storage/search/tool fees.

Label it accurately:

maximum estimated cost for one model generation under current token limits

not “maximum this user action can ever cost” unless you bound the entire workflow.

A per-operation cost budget can protect agentic workflows

For long tool loops, define a logical-operation budget:

struct OperationBudget {
    let maxProviderAttempts: Int
    let maxToolRounds: Int
    let maxEstimatedCost: Decimal?
    let deadline: Date
}

Before each new model call, calculate projected remaining cost.

If the budget would be exceeded:

stop
or
ask user for explicit continuation

Cost estimation should not block basic chat unnecessarily

Fetching live pricing or calling a token-count endpoint before every message can add latency and failure modes.

Consider a tiered design:

fast local estimate for normal UI
provider-reported usage after completion
optional authoritative refresh for pricing metadata

Cost visibility should not make the chat client fragile.

Cache pricing tables, but expire them intentionally

A pricing catalog can be cached locally with:

last updated timestamp
source URL/version
model aliases

Refresh when stale or when the provider/model is unknown.

If refresh fails, show the last-known estimate as stale rather than hiding usage completely.

Model aliases can make rates ambiguous

An alias may point to a newer snapshot over time.

If exact historical pricing matters, preserve the concrete model identifier returned by the provider when available.

Do not assume an alias has permanent behavior or rates.

Cost UI should communicate uncertainty

Good labels include:

Estimated
Approx.
Based on current configured rates
Actual provider billing may differ

Bad UI:

Cost: $0.0139427

before the request has even generated output.

False precision undermines trust.

Post-request reconciliation

After completion:

preflight estimate
      ↓
provider request
      ↓
reported usage
      ↓
current/historical rate table
      ↓
reconciled estimated spend

Store both estimate and actual usage if you want to measure estimator quality.

Measure estimate error

For completed requests:

$$CostError = ActualEstimatedCost - PreflightEstimatedCost$$

Track this by:

model
reasoning setting
request type
tool-enabled vs plain
cache expected vs cache hit

You can improve preflight estimates without storing prompt content.

Common mistakes

Estimating from only the current user message

Ignores history, tools, retrieval, and instructions.

Assuming maximum output will always be used

Produces overly pessimistic “expected” costs.

Ignoring reasoning tokens

Can underestimate reasoning-model usage.

Assuming every stable prefix is a cache hit

Cache eligibility is not a guaranteed hit.

Treating one user turn as one provider call

Tool loops, retries, and fallback can multiply requests.

Recording price but not usage

Makes future auditing difficult.

Reporting unknown custom-provider pricing as free

Unknown is not zero.

A practical preflight algorithm

Cost-estimation checklist

• Estimate the full assembled request, not just the newest message.
• Keep input, output, cached input, reasoning, and modality usage separate.
• Treat output and reasoning as uncertain before execution.
• Account for multiple model calls in tool loops.
• Track retries/fallback attempts separately.
• Keep pricing metadata versioned and provider/model-specific.
• Preserve usage even when pricing is unknown.
• Use decimal arithmetic for money.
• Label preflight values as estimates.
• Reconcile against provider-reported usage afterward.

Where BYOKchat fits

A BYOK client can calculate local preflight estimates from the selected model, assembled context, configured generation budget, and known rate metadata. After the provider returns usage, the app can replace projections with a more accurate local spend estimate without sending conversation content to an analytics server.

That makes cost visibility useful while keeping the provider—not the client—the authoritative billing system.

Further reading

• Understanding AI API Costs
• AI Tokens Explained for Developers
• AI Prompt Caching Explained
• Reasoning Effort Explained
• Designing Reliable AI Retries