How to Classify AI API Errors

A production AI client should not reduce every failure to:

Request failed

That throws away the information needed to decide what happens next.

A timeout, invalid API key, context-window overflow, safety refusal, malformed stream event, failed tool, user cancellation, and local database error can all terminate one chat turn. They do not have the same cause, retry policy, fallback behavior, or user action.

A useful error system therefore answers four questions:

Which layer failed?
Is the failure retryable?
Did the operation have an unknown or partial outcome?
What can the user or application do next?

Classify the layer before the status code

A practical taxonomy for AI clients is:

transport
HTTP
provider/account
model/capability
policy/safety
tool execution
protocol/parsing
stream lifecycle
application/storage
cancellation
unknown

The category is more useful than a single numeric code because some failures have no HTTP response at all, while the same HTTP status can represent different provider-specific causes.

Keep logical requests separate from provider attempts

One user action can create multiple network attempts:

logical request
  ├─ attempt 1 -> connection reset
  ├─ attempt 2 -> 429
  └─ attempt 3 -> success

Observability should preserve both identities.

The user sees one generation. The reliability layer sees three attempts.

Without this distinction, retry counts, error rates, cost estimates, and latency metrics become misleading.

Transport errors happen before HTTP semantics

Examples include:

DNS lookup failed
connection refused
TLS handshake failed
certificate validation failed
connection reset
network became unavailable
socket closed

There may be no HTTP status because the request never reached a point where an HTTP response could be received.

A transport error should record information such as:

network stage
endpoint class
elapsed time
whether any response bytes arrived
whether the request body may already have been sent

That final detail matters because a lost connection does not always mean the server did nothing.

Separate connection failure from unknown outcome

Consider:

client sends request body
provider receives it
provider starts generation
connection disappears
client sees network error

The client cannot safely conclude:

request never happened

For an ordinary generation, retrying may create a second answer and additional usage. For a request that can trigger side effects, the risk is greater.

Represent outcome certainty explicitly:

interface AttemptOutcome {
  execution: "not_started" | "started" | "completed" | "unknown";
  response: "none" | "partial" | "complete";
}

The exact type can differ, but the uncertainty should not be hidden inside a generic network error.

HTTP status is a layer, not the whole diagnosis

Once a response exists, the status code gives a useful first classification.

Common groups are:

400-499 -> request, authentication, authorization, quota, policy
500-599 -> provider/server/gateway failure

But do not hard-code one meaning for every provider.

For example, a 400-class response can represent:

• malformed JSON;
• unsupported parameter;
• invalid tool schema;
• model not available to the account;
• context too long;
• incompatible image/file input.

Read the provider’s structured error object when one exists.

401, 403, and 429 deserve distinct categories

A useful baseline is:

401 -> authentication problem
403 -> permission/policy/account authorization problem
429 -> rate, quota, or temporary capacity pressure

A 401 should normally lead to credential repair, not retry backoff.

A 403 may require a different account, permission, region, model, or organization setting.

A 429 may be retryable, but only according to provider guidance and bounded retry policy.

See AI API Error 401 vs 403 vs 429.

Provider errors should preserve native metadata

A provider may return fields such as:

{
  "error": {
    "type": "rate_limit_error",
    "code": "...",
    "message": "..."
  },
  "request_id": "..."
}

Normalize the category for application behavior, but keep safe provider-native metadata for diagnostics.

A good normalized record can contain:

interface AIErrorRecord {
  category: ErrorCategory;
  provider: string;
  model?: string;
  httpStatus?: number;
  providerCode?: string;
  requestId?: string;
  retryable: boolean;
  outcome: "known" | "partial" | "unknown";
}

Do not store API keys, authorization headers, prompts, reasoning, tool arguments, or private endpoint details just because the provider returned an error.

Model and capability errors are not provider outages

Examples:

context length exceeded
model does not support images
model does not support tools
reasoning control unsupported
structured output unsupported
model identifier invalid

These failures often mean the request and provider are healthy but the selected model cannot satisfy the operation.

That distinction matters for routing.

If a model rejects an image input, marking the whole provider unhealthy can unnecessarily trip circuit breakers and fallback logic.

See How to Detect AI API Capability Mismatches Before Sending.

Context overflow should be its own actionable subtype

A context-window failure is usually not solved by sending the same request again.

Possible recovery actions include:

trim old turns
summarize history
remove unused attachments
reduce tool schemas
reserve less output
switch to a model with sufficient context

Classification should therefore expose a semantic reason such as:

model.context_limit_exceeded

rather than only 400.

Policy and safety outcomes are not transport failures

Providers can reject or stop requests because of policy, safety, content restrictions, account rules, or regional constraints.

These should not enter generic transient-retry loops.

The application needs to distinguish:

provider refused the request intentionally

from:

provider could not process the request temporarily

Repeatedly retrying an intentional refusal wastes requests and can create confusing UX.

A model refusal is different from an API error

A model may successfully return a response whose semantic content is a refusal.

That is often a successful API operation, not a failed HTTP request.

The transport, provider, and parser all worked.

Do not inflate API reliability error metrics by counting every model refusal as a server error.

Product analytics can track model outcome separately if useful.

Tool errors have at least three layers

A tool-enabled turn can fail because:

1. model produced invalid tool arguments
2. application denied or rejected the call
3. tool execution itself failed

These are different.

Invalid arguments

The model produced data that failed schema or business validation.

Permission denial

The application or user intentionally refused the operation.

Execution failure

The approved tool ran but its underlying service, file operation, API, or command failed.

The model should receive a structured tool result appropriate to the workflow, while observability records the actual execution category.

See How to Validate AI Tool Arguments Safely and How to Debug AI Tool-Calling Loops.

Tool cancellation is not the same as tool failure

If the user presses Cancel while a tool is running, classify that separately.

Otherwise metrics can suggest the tool is unreliable when the actual behavior was user intent.

This distinction also matters when deciding whether the model should continue the multi-round loop.

Parsing and protocol failures deserve their own bucket

A provider can return HTTP 200 and still produce unusable data.

Examples:

invalid JSON response
malformed SSE field
unknown required event shape
invalid UTF-8 handling
truncated structured object
unexpected content type
schema violation in a supposedly structured response

Do not label these as provider 5xx failures if the HTTP status was successful.

The failure occurred in the protocol/adapter boundary.

This is especially important for OpenAI-compatible endpoints, where implementations may accept the same request path but differ in response details.

Streaming introduces partial-success states

A stream can emit 500 tokens and then fail.

The request is neither simply:

success

nor:

no result

Store a finish state such as:

completed
cancelled
failed_before_output
failed_after_partial_output
interrupted_unknown

This preserves the user’s visible partial answer and prevents retry logic from pretending nothing happened.

See Why AI Streams Break in the Middle.

End-of-stream semantics matter

A TCP/HTTP connection closing is not always the same as a provider-declared completion event.

Your adapter should know whether it observed the semantic completion boundary expected for that API.

If the connection disappears before the completion signal, classify the stream as interrupted even if the text looks grammatically complete.

Otherwise the application can miss:

• final usage data;
• finish reason;
• final tool-call boundary;
• provider state identifier;
• structured-output completion.

Cancellation should be first-class

User cancellation is usually not an error.

Represent it explicitly:

finish = cancelled_by_user

Likewise, app lifecycle cancellation, superseded requests, and shutdown may deserve distinct operational reasons.

This keeps reliability metrics honest.

Application errors are outside the provider

Examples include:

failed to persist conversation
attachment file missing locally
Keychain lookup failed
invalid local provider configuration
renderer crashed on unexpected state
queue record corrupted

A provider request may have succeeded even if the app failed afterward.

If persistence fails after a generation completes, retrying the provider request is usually the wrong recovery action.

The application should recover or reconcile local state instead.

Authentication setup errors can happen before the request

A BYOK client can detect:

credential missing
base URL empty
protected header unresolved
connection disabled

before opening the network connection.

Classify these as local configuration errors, not HTTP authentication failures.

That makes diagnostics more precise and avoids fake provider attempts in metrics.

Do not classify by message text alone

This is fragile:

if (message.includes("rate limit")) {
  retryable = true;
}

Error messages are human-facing, provider-specific, localized, and changeable.

Prefer, in order:

structured provider code/type
HTTP status + documented semantics
adapter-specific mapping
known transport error type
fallback unknown classification

Message text can remain useful for display and debugging, but it should not be the primary control plane.

Unknown must remain a valid category

Forcing every new provider error into a known bucket can be worse than admitting uncertainty.

Use:

category = unknown
retryable = false or conservative policy

and preserve sanitized diagnostics.

Then update the adapter when the behavior is understood.

Retryability is derived, not identical to category

A transport error may be retryable in one state and unsafe in another.

For example:

DNS failure before request -> often safe to retry
connection lost after request body -> outcome may be unknown

Likewise, a 429 may be retryable but only after waiting, while a context-limit error is not retryable unchanged.

Represent retryability independently from the main category.

Fallback is also a separate decision

A request that cannot run on one model may be able to run on another.

But fallback must re-check:

capabilities
privacy policy
tool compatibility
reasoning state
attachments
context size
user routing policy

An error classifier should provide signals to the router; it should not silently choose another provider by itself.

See AI Provider Fallback and Model Routing.

Map categories to user actions

A useful UI does more than show an error string.

Category	Typical action
Missing/invalid credential	Edit connection
Permission denied	Check account/model access
Rate limited	Wait/retry later
Context too long	Reduce context or change model
Capability mismatch	Change model/settings
Tool approval denied	Continue without tool or stop
Tool execution failed	Retry tool if safe or inspect failure
Network unavailable	Retry when connectivity returns
Partial stream interrupted	Keep partial result, regenerate/continue
Local persistence failure	Repair/retry local save

The message can still include provider detail, but the action should come from the normalized category.

Error data should be privacy-safe by construction

Useful fields include:

provider kind
model ID if non-sensitive
HTTP status
provider error code
request ID
duration
TTFT if any
bytes/events received
retry attempt
streaming yes/no
tool phase
finish state

Avoid collecting:

prompt text
response text
reasoning text
API keys
Authorization headers
tool arguments/results
attachment contents
private custom endpoint URLs

See Privacy-Preserving Analytics for AI Apps.

Classify once near the boundary

A good architecture is:

The provider adapter understands native error details.

The shared classifier exposes stable application semantics.

The UI, retry layer, and analytics do not need to parse provider-specific strings independently.

Test the taxonomy with deterministic failures

A provider simulator should cover at least:

DNS failure
TLS failure
connection reset before response
401
403
429 with retry guidance
500
503
malformed JSON
malformed SSE
context overflow
unsupported tool capability
policy rejection
invalid tool arguments
tool timeout
stream failure after partial output
user cancellation
local persistence failure
unknown provider code

For each case assert:

normalized category
retryable flag
outcome certainty
user action
fallback eligibility
metric classification

A classification checklist

Before shipping a failure path, ask:

• Did we identify the layer that failed?
• Did we preserve HTTP/provider metadata without leaking secrets?
• Did we distinguish request failure from model refusal?
• Did we distinguish model incompatibility from provider health?
• Did we preserve partial-stream state?
• Did we distinguish cancellation from failure?
• Did we represent unknown execution outcomes?
• Did we avoid automatic retries for configuration/policy errors?
• Did we avoid marking local app failures as provider failures?
• Can the UI offer a meaningful next action?

Where BYOKchat fits

A multi-provider BYOK client benefits from a shared error model because provider-native APIs, OpenAI-compatible servers, local endpoints, MCP tools, and app-local persistence all fail differently.

Provider adapters can preserve native request IDs and error codes while mapping them into stable categories. The reliability layer can then decide retry, fallback, or stop without teaching every screen about every provider’s error schema.

That architecture also makes analytics more trustworthy: authentication mistakes do not look like outages, context overflow does not poison provider-health scores, user cancellation does not look like instability, and partial streams remain visible as partial outcomes.

Further reading

• AI API Error 401 vs 403 vs 429
• Designing Reliable AI Retries
• Why AI Streams Break in the Middle
• How to Debug AI Tool-Calling Loops
• AI Provider Fallback and Model Routing