How to Estimate Per-Conversation AI Cost

A user asks a seemingly simple question:

How much did this conversation cost me?

For one request with one provider and one published token price, the arithmetic can be straightforward.

A real conversation can contain:

multiple turns
multiple models
cached input
reasoning tokens
regenerations
retries
tool loops
provider fallback
background jobs
file processing

The right design is not one global formula.

Store provider-reported usage per generation attempt, store the pricing metadata used for the estimate, and aggregate those records at conversation level.

Start with usage, not money

The durable measurement is usage:

input tokens
cached input tokens
output tokens
reasoning tokens if separately reported
requests
provider attempts
tool calls

Money is derived from usage plus a pricing model.

If you store only:

cost = $0.0137

then later you cannot explain how that number was produced.

A usage record should belong to a generation attempt

Conceptually:

interface UsageRecord {
  generationId: string
  providerConnectionId: string
  modelId: string
  inputTokens?: number
  cachedInputTokens?: number
  outputTokens?: number
  reasoningTokens?: number
  providerReported: boolean
}

Provider schemas differ, so the normalized record can include optional native fields too.

Provider-reported usage is usually better than client token estimates

Before a request, the client may estimate input tokens for context budgeting.

After completion, the provider may report authoritative billable usage according to its own tokenizer/accounting rules.

Use:

preflight estimate -> planning
provider usage -> post-request accounting

Do not silently replace provider usage with a local tokenizer approximation when billing semantics differ.

See AI Tokens Explained for Developers.

Token categories are provider-specific

A simple model may expose:

input
output

Another may expose distinctions such as:

cached input
cache creation/write
reasoning
image/audio units

Do not force every native category into the wrong generic bucket.

Keep a normalized core plus provider-native usage metadata when needed.

A basic cost equation

For a model with ordinary input/output rates:

$$C = \frac{T_{in}}{1{,}000{,}000}P_{in} + \frac{T_{out}}{1{,}000{,}000}P_{out}$$

where:

T_in  = billable input tokens
T_out = billable output tokens
P_in  = price per million input tokens
P_out = price per million output tokens

If the provider prices cached input differently, split it:

$$C = \frac{T_{uncached}}{10^6}P_{in} + \frac{T_{cached}}{10^6}P_{cached} + \frac{T_{out}}{10^6}P_{out}$$

The exact categories depend on current provider pricing.

Do not hard-code prices deep in request logic

Pricing changes.

Keep a pricing catalog/data layer such as:

interface PriceSpec {
  provider: string
  modelId: string
  effectiveFrom: string
  currency: string
  unit: "per_million_tokens"
  input?: number
  cachedInput?: number
  output?: number
  sourceRevision?: string
}

Then cost estimation is a pure function over usage plus a price spec.

Historical cost needs historical pricing provenance

Suppose a provider changes the model price next month.

If the app recomputes every old conversation with the new price, historical estimates change.

That can be confusing.

Better options:

Store estimate-at-generation

estimatedCost = 0.0137
pricingRevision = 2026-09-04

Or store the exact price spec used

Then the estimate is reproducible.

You can still offer a separate:

estimated at current pricing

view if useful.

Label estimates as estimates

The provider’s invoice can include billing rules that a client does not fully model.

Examples can include:

minimum charges
batch discounts
regional differences
enterprise contracts
free credits
provider routing differences
non-token tool pricing

A local client generally knows public/list pricing, not the user’s final invoice.

Use language such as:

Estimated API cost

not:

Exact amount charged

unless the provider explicitly supplies that exact charge.

Per-conversation cost is a sum of generation costs

If generations are:

G1 $0.01
G2 $0.02
G3 $0.03

then:

conversation estimate = sum(G1..Gn)

But define which generations belong.

A conversation can contain hidden branches and failed attempts.

Regenerations still cost money

If the user regenerates one response three times, all successful provider generations can consume billable usage.

The visible selected branch may show only one answer, but actual spend includes the alternatives.

Your cost view can distinguish:

selected-path usage
all conversation activity

The total amount spent should normally include all provider requests that actually occurred.

See How AI Chat Branching and Regeneration Should Work.

Failed requests may still have usage

Do not assume:

HTTP failure = zero cost

A stream can fail after substantial generation.

A background job may finish remotely even if the local client misses the final event.

If usage is returned or later retrievable, attach it to the attempt.

If usage is unknown, mark it unknown rather than guessing zero.

Partial streams need incomplete accounting states

A generation record can have:

usage.status = provider_reported
usage.status = locally_estimated
usage.status = unknown

Then the conversation total can say:

Estimated $0.42 + one request with unknown usage

That is more truthful than silently treating unknown as free.

Retries can multiply cost

One logical user action may create:

attempt 1 -> timeout after generation began
attempt 2 -> success

Both may have consumed provider resources.

Track attempts separately under one logical generation/request.

See Designing Reliable AI Retries.

SDK retries can be invisible unless you account for them

If a provider SDK performs automatic retry internally, your app-level request count may not equal provider attempts.

Where possible, instrument the SDK/transport layer or disable hidden retries so your reliability and cost accounting remain understandable.

At minimum, document that the estimate may not include unobservable SDK attempts.

Provider fallback creates multiple billable targets

A routing sequence might be:

Provider A attempt -> failure
Provider B attempt -> success

The conversation cost needs both records.

Do not attribute the entire logical request only to the final successful provider.

See How to Build Reliable AI Provider Fallback and Model Routing.

Tool loops can generate several model rounds

A user sees one message:

"Book the meeting and summarize the result"

The model runtime may do:

round 1 -> tool call
round 2 -> tool result -> another tool call
round 3 -> final answer

Each model round can have its own usage.

A conversation-level view should aggregate them while preserving round-level diagnostics.

See How Multi-Round AI Tool Loops Work.

Tool execution can have external cost too

A tool may call a paid API.

That cost is not necessarily part of the AI provider bill.

Keep categories separate:

model API estimated cost
tool/external service cost if known
local compute cost not monetarily estimated

Do not quietly combine unlike billing systems into one number without labels.

Hosted tools can have separate provider pricing

Some providers may price hosted search, code execution, storage, or other tools separately from tokens.

The pricing model should support additive line items:

interface CostLineItem {
  category: string
  quantity: number
  unit: string
  unitPrice: number
  estimatedCost: number
}

Avoid assuming tokens are the only billable unit forever.

File handling can introduce additional operations

A file-enabled conversation may include:

file upload
file processing
embeddings/retrieval
model request

Whether those have billable cost depends on the provider.

Keep the operation types visible in accounting when the provider exposes them.

See How File Attachments Flow Through AI APIs.

Prompt caching changes marginal cost

If repeated project instructions or prior context are cached, two requests with the same nominal input token count can have different cost.

Store the provider’s cached-input usage rather than assuming a cache hit from repeated text alone.

See AI Prompt Caching Explained.

Reasoning usage can be reported differently

Reasoning models may expose reasoning-related usage in provider-specific ways.

Do not assume:

reasoning tokens always added to output tokens

or:

reasoning tokens always separately billed

Use the provider’s documented accounting and keep native fields when the normalization cannot be lossless.

See How AI Reasoning Models Use Context.

Input token growth matters across turns

In stateless replay, later turns can resend much of the conversation:

Turn 1 input = 1k
Turn 2 input = 3k
Turn 3 input = 6k
Turn 4 input = 10k

The user may think a 10k-token conversation has cost only 10k input tokens total.

Actual cumulative input processed can be much larger.

This is why per-generation accounting is important.

Stateful APIs can change replay economics

A provider that supports server-side continuation may not require the client to resend every prior message in the same way.

But billing/accounting still follows that provider’s rules.

Do not estimate stateful API cost by blindly summing local transcript tokens.

See Stateful vs Stateless AI Conversations.

Pre-send cost estimation has uncertainty

Before the model runs, you can estimate:

input tokens
known cached prefix possibility
max output allowance

but you do not know exact output length.

A useful preflight display can show a range:

Estimated input cost: ...
Maximum additional output cost at configured cap: ...

rather than one fake exact prediction.

Output reserve is not expected output

If max_output_tokens = 8000, that does not mean the model will produce 8000 tokens.

Treat it as an upper bound where applicable, not a forecast.

Pricing metadata freshness matters

If the app caches model prices locally, store:

last_updated_at
source

A stale catalog can still be useful, but label it:

Estimate based on pricing last updated ...

Do not block all usage analytics just because current pricing is unavailable.

User-specific contract pricing may differ

A BYOK app usually cannot know:

enterprise discount
promotional credits
prepaid balance
special contract rate

Therefore public/list pricing is an estimate.

The provider dashboard/invoice remains the billing source of truth.

Multiple accounts need separate attribution

A user may configure two connections to the same provider.

Store:

connection ID
provider type
model

on each usage record.

This allows per-account totals without exposing API key material.

Custom endpoints may have no known price

A local or private OpenAI-compatible endpoint might be:

free internal server
paid gateway
self-hosted GPU

If the app has no verified pricing metadata, show:

Usage: 18,230 tokens
Estimated API cost: unavailable

Do not invent $0 merely because pricing is unknown.

Local inference has resource cost but not necessarily API price

For a local model, a useful label is:

No provider API charge

rather than:

Free

Local inference still uses hardware, electricity, battery, and time.

Unless the product has a credible way to price those, keep them outside API cost accounting.

Currency needs a defined policy

Provider prices are commonly denominated in a particular currency.

If you convert to the user’s local currency, exchange rates introduce another time-varying estimate.

Store:

base cost currency
conversion rate source/date
converted display value

or simply show the provider pricing currency.

Do not lose the original amount.

Round at display time, not during accumulation

If each tiny generation is rounded to cents before summing, totals drift.

Store precise decimal values and round for UI presentation.

Avoid binary floating-point for financial calculations where exact decimal arithmetic is available.

Pricing calculations should be deterministic

Given:

usage record
price spec

the cost function should produce the same result every time.

This makes tests easy and prevents UI-specific math from diverging.

A cost breakdown is more useful than one total

For example:

Conversation total: $0.84 estimated

Input:           $0.31
Cached input:    $0.04
Output:          $0.45
Hosted tools:    $0.04
Unknown usage:   1 interrupted request

Users can understand why a long conversation became expensive.

Show cost by model/provider

A multi-provider conversation may have:

Model A: $0.18
Model B: $0.52
Local model: no API charge
Fallback attempts: $0.03

This is useful for model selection decisions.

Cost per turn can explain context growth

A chart/table can show:

Turn 1 $0.002
Turn 2 $0.004
Turn 3 $0.009
Turn 4 $0.018

This helps users see the cost of long context.

Do not needlessly collect prompt content to provide this view.

Analytics should separate token usage from pricing

A privacy-safe local analytics store can keep:

token counts
request counts
model ID
timing
estimated cost
pricing revision

without prompts or responses.

See Privacy-Preserving Analytics for AI Apps.

Cost should survive export carefully

A conversation export can include historical usage and estimate-at-time values.

Label them as historical estimates with pricing provenance.

Do not export current API credentials or billing account identifiers.

See How to Export AI Conversations Portably.

A practical data flow

Useful invariants

usage is stored independently from cost
provider-reported usage outranks rough local estimates after completion
unknown usage is not silently treated as zero
regenerations/retries/fallback attempts remain billable history
historical estimates retain pricing provenance
custom/local targets can have usage without known monetary cost
credentials and prompt content are not required for accounting

Test the accounting layer

Important cases:

simple input/output model
cached input category
reasoning-specific usage
multiple models in one chat
regeneration hidden branch
retry after partial stream
fallback across providers
unknown usage after timeout
multi-round tool loop
hosted tool line item
custom endpoint without pricing
price changes between generations
currency conversion optional
backup/export round trip

Use fixed fixtures and decimal expected values.

Where BYOKchat fits

A multi-provider client can calculate useful local estimates because it sees provider/model identity, normalized usage, retries, tool rounds, timing, and conversation membership for each request. It does not need to collect prompt or response content to show tokens and estimated spend.

The important boundary is to present the number as an estimate based on known pricing metadata, while leaving the provider’s billing dashboard as the final authority.

Further reading

• Understanding AI API Costs
• AI Tokens Explained for Developers
• AI Prompt Caching Explained
• Designing Reliable AI Retries
• Privacy-Preserving Analytics for AI Apps