Designing Offline and Degraded Modes for AI Apps

An AI app is often treated as either:

online and working

or:

offline and broken

Real products have many intermediate states.

Examples:

internet unavailable but local chats readable
cloud provider down but local model available
one provider rate-limited while another is healthy
MCP server unavailable but ordinary chat works
web search unavailable but base model works
file upload failed while cached files remain local
remote model unavailable but drafts/history remain editable

A good degraded mode preserves what still works and tells the user clearly what changed.

Offline is a capability state, not one boolean

Instead of one global flag:

offline = true

consider a capability matrix:

read local chats      yes
edit drafts           yes
search local history  yes
cloud generation      no
local generation      maybe
remote MCP tools      no
local MCP tools       maybe
web search            no
provider file upload  no

This leads to much better UX and fewer unnecessary feature shutdowns.

Start with local-first data access

If conversations and projects are stored locally, the app should remain useful without network access.

Offline users can often still:

open chats
search history
read attachments already stored locally
edit project instructions
compose drafts
export local content
inspect analytics already stored locally

Do not gate local data behind a provider connectivity check.

Drafts should work offline

Composing text is local work.

If the user writes a message while offline, preserve it as a draft rather than throwing it away when Send fails.

Possible UX:

No connection. Draft saved locally.

Whether the app supports a true “send later” queue is a separate product decision.

Be careful with automatic send-later

Automatically sending queued prompts when connectivity returns can surprise users.

The context may have changed, and the prompt may be sensitive.

A safer default for interactive chat is often:

preserve draft / failed send
ask user to retry

If you implement send-later, show the queued state clearly and let the user cancel before dispatch.

See How to Build an AI Request Queue.

Local models can provide real offline generation

If the app has access to an on-device model or a local server that remains reachable without the public internet, offline generation may continue.

But distinguish:

on-device model
same-LAN server
private overlay network server
cloud endpoint

A LAN server may remain available during an ISP outage, while a Tailscale/private remote host may still require internet connectivity.

Do not silently switch to local because cloud failed

Local fallback can be useful, but model capability and answer quality may differ substantially.

The user should know the target changed.

Possible behavior:

Cloud provider unavailable.
Use Local Model instead?

or an explicit policy configured in advance:

Fallback to local when cloud is unavailable

Do not make invisible model changes inside a conversation unless the product clearly communicates them.

Privacy policy outranks availability

If the user selected:

local only

then restoring availability by sending the prompt to a cloud provider violates the policy.

Likewise, a chat pinned to one provider should not silently move to another vendor merely because the first is degraded.

Fallback requires prior policy or user consent.

See Cloud Model vs Local Model Routing.

Capability matching still applies during fallback

Suppose the cloud model was using:

image input
tool calling
large context
strict JSON schema
provider-hosted search

A fallback local model may not support all of these.

Before offering fallback, compute the capability difference.

A good UI can say:

Local fallback is available, but web search and image input will be disabled.

That is better than sending a request that later fails mysteriously.

Degrade features independently

An AI workflow can have several dependencies:

If web search fails, the base model may still answer without live grounding.

If MCP is unavailable, ordinary chat may still work.

If a file service is unavailable, text-only requests may still work.

Do not collapse every dependency failure into “AI unavailable.”

Grounding loss must be explicit

If a user asked for current information and web search becomes unavailable, silently answering from model knowledge can be misleading.

The correct degraded behavior may be:

Web search is unavailable, so I cannot verify current information.

or ask whether the user wants an ungrounded answer.

See AI Grounding Explained.

Tool loss can change what the model is allowed to promise

If a calendar/email/tool server is unavailable, the model must not pretend it completed the action.

The host should remove/disable unavailable tools from the current capability set or return structured execution failures.

A truthful response is:

I couldn't access the calendar tool.

not:

Done

Provider health can drive degraded UI

Circuit breakers and recent health signals can expose states such as:

healthy
degraded
temporarily unavailable
unknown

The UI can show these at connection/model selection without blocking unrelated providers.

See Circuit Breakers for AI Providers and Health Scoring AI Providers Without Fake Precision.

Network reachability is only a hint

A device being “online” does not prove a provider is reachable.

Likewise, a reachability framework reporting no internet does not necessarily mean a LAN-hosted model is unreachable.

The authoritative signal is the actual operation/endpoint result.

Use network-path information to improve UX and scheduling, not as a universal replacement for request errors.

Avoid constant health polling

When offline/degraded, repeatedly pinging every provider wastes battery and can create noise.

Prefer:

react to network path changes
respect circuit cooldowns
probe on user action or bounded schedule
use lightweight documented endpoints when needed

Do not consume user BYOK quota with synthetic model generations just to make a status light green.

Cached data needs freshness labels

Offline mode may use cached:

model lists
provider capabilities
pricing metadata
document indexes
search results
remote file metadata

A cache can keep the app usable, but stale data should not masquerade as current truth.

Useful labels include:

Last updated 2 hours ago
Cached model list
Pricing estimate may be outdated

Model discovery can degrade gracefully

If provider /models discovery fails but the user already selected a known model, the app can often keep that cached model configuration.

Do not necessarily erase the model list because one refresh failed.

But if the provider later rejects the model, update the local capability/model state accordingly.

Offline search can still work locally

If conversation search is implemented with a local lexical index, it can remain available without network access.

Semantic search may also work offline if embeddings/indexing are local.

If semantic search depends on a cloud embedding provider, degrade to lexical search rather than disabling all search if the product can support that transition clearly.

Attachments have local and remote states

A file can be:

stored locally
uploaded to provider
indexed locally
indexed remotely

Offline mode should distinguish them.

A user can still open the local file even if its provider upload ID cannot be used for a new cloud request.

Do not delete or hide local content because remote metadata is unavailable.

Background jobs need reconciliation after connectivity returns

A remote provider job may have completed while the device was offline.

On reconnection:

load durable job IDs
query authoritative remote status
finalize completed operations
resume appropriate polling

Do not recreate the job just because local status is stale.

See How Long-Running AI Tasks Work.

Retry queues should respect user intent

When connectivity returns, the queue should not unleash a large backlog simultaneously.

Use:

priority
rate-limit scheduling
cancellation
staleness checks
concurrency limits

Interactive user actions should usually outrank old maintenance work.

Queued prompts should be revalidated against current conversation state.

Degraded mode should be reversible

Avoid UI state that sticks after recovery.

For example:

provider circuit closes -> remove degraded warning
network returns -> refresh eligible stale data
local server reconnects -> re-enable local model actions
MCP server recovers -> rediscover tools if needed

Recovery is part of the state machine, not an app restart requirement.

Keep partial generation when connectivity disappears

If a stream fails after producing output:

preserve partial assistant text
mark it interrupted
show failure cause
allow regenerate/continue where appropriate

Do not discard the text merely because the final event was not received.

See Why AI Streams Break in the Middle.

Be careful with “continue” across providers

A disconnected provider stream may have non-portable reasoning state, tool-call state, or server-side response identity.

Switching to another provider may require rebuilding context from portable conversation messages.

Call this a new continuation/regeneration rather than pretending it is byte-for-byte resumption of the original stream.

Offline mode should not weaken security

Do not respond to connectivity problems by:

disabling TLS validation
accepting arbitrary certificates
sending credentials in URL params
opening local server to all interfaces without auth

Availability pressure is not a reason to cross trust boundaries.

See Certificate Validation for Custom AI Endpoints.

Local HTTP exceptions should remain explicit

A user may deliberately configure private-LAN HTTP for a local model.

That is a connection policy decision, not a general fallback when HTTPS fails.

Never transform:

https://server.example failed

into:

try http://server.example

silently.

UX should distinguish dependency states

Instead of one red banner:

Offline

use context-specific messaging:

No internet — local chats remain available
Anthropic connection unavailable
Local model available
Web grounding unavailable
2 background jobs will be checked when online

This is more actionable and less alarming.

Keep destructive actions available when possible

Users should often still be able to:

delete local conversations
remove credentials
export local data
change settings
cancel queued work

while offline.

Security/privacy controls should not depend unnecessarily on cloud reachability.

Cost UI should degrade honestly

If pricing metadata is cached and may be stale, an offline cost estimate should say so.

Do not hide the estimate if it remains useful, but avoid presenting cached provider pricing as guaranteed current pricing.

Token usage already stored locally can still be displayed.

Notifications after recovery need deduplication

If several background operations complete while connectivity is absent, reconciliation may discover them together.

Use stable operation IDs so the app does not show duplicate completion notifications after repeated status checks.

An offline/degraded state model

Real capability state can be more granular than these UI-level labels.

Observe degraded-mode behavior

Useful privacy-safe metrics include:

network unavailable events
provider circuit open duration
fallback offered/accepted
offline local generations
queued requests cancelled as stale
background jobs reconciled after offline period
feature capability disabled reason

Do not collect the prompt that triggered fallback.

See Privacy-Preserving Analytics for AI Apps.

Test dependency combinations

A robust test matrix includes:

no internet + local on-device model
no internet + LAN model
internet + selected provider down
selected account rate-limited + second account healthy
MCP down + model healthy
web grounding down + model healthy
attachment stored local + provider upload unavailable
app resumes after offline background job completion
partial stream + network loss
network returns with 20 queued tasks
local-only privacy policy + cloud healthy

Assert which actions remain enabled and which fallback choices are offered.

A degraded-mode checklist

Offline state is expressed as capabilities, not one global boolean.
Local data remains usable without provider connectivity.
Drafts survive failed sends.
Automatic send-later is explicit and cancellable if implemented.
Local/cloud fallback respects user privacy and provider choice.
Capability mismatches are shown before fallback.
Search, tools, files, and grounding can degrade independently.
Cached metadata is labeled when freshness matters.
Partial streams are preserved.
Durable remote jobs reconcile instead of recreating.
Security policies do not weaken during outages.
Recovery removes stale degraded state automatically.
Queues resume gradually and revalidate stale work.
User-facing messages describe what still works.

Where BYOKchat fits

A local-first multi-provider client has unusually strong degraded-mode options because conversations, projects, provider configuration, and analytics can remain on-device while provider requests go directly to their destinations.

That means losing one cloud provider does not have to make the app useless. Users can keep reading and organizing local data, use another explicitly chosen provider or local model when policy permits, and recover interrupted work without routing prompts through a mandatory central service.