How to Resume or Recover an Interrupted AI Generation

When an AI stream disconnects halfway through, the right recovery strategy depends on what survived.

There is no universal “resume” button at the transport layer.

The client might have:

• only partial visible text;
• a provider response ID;
• a durable background operation;
• a completed tool result that the model never acknowledged;
• provider-native reasoning state;
• nothing except the last safe conversation turn.

So recovery starts with a state inventory, not an automatic retry.

The key rule is:

Recover from the last state whose meaning you understand. Never assume re-sending the request is equivalent to resuming the original generation.

Resume, continue, regenerate, and reconcile are different

These words are often mixed together.

Resume

Continue receiving the same provider operation after interruption.

This requires provider/server support for durable operation state or replayable events.

Continue

Start a new model request using state from the interrupted response.

For example:

partial assistant response
+ provider continuation object
+ new instruction: continue

This is a new inference operation even if it feels continuous to the user.

Regenerate

Discard the interrupted model output as authoritative and run the turn again from the last safe conversation state.

Reconcile

Ask a durable backend/provider what actually happened before deciding what to do next.

This is especially important when tools or other side effects may have executed.

Model the generation as durable state

Do not represent generation only as:

isStreaming = true

A richer record might include:

type GenerationRecord = {
  id: string;
  chatId: string;
  turnId: string;
  status:
    | "starting"
    | "streaming"
    | "tooling"
    | "interrupted"
    | "recovering"
    | "completed"
    | "cancelled"
    | "failed";
  provider: string;
  model: string;
  providerOperationId?: string;
  partialText: string;
  toolExecutions: ToolExecutionRecord[];
  startedAt: string;
  lastEventAt?: string;
};

The exact schema is product-specific, but the record should answer:

what was running?
how far did it get?
what side effects happened?
can the provider still identify it?

Persist the turn before starting inference

A crash-resistant flow is:

persist user message
→ create assistant generation record
→ commit local state
→ send provider request
→ stream updates into generation record

If the app dies immediately after sending the request, relaunch can still see that an operation was in flight.

If you create local state only after the first token arrives, some failure windows become invisible.

Store partial output explicitly

Do not throw away partial text on interruption.

Store it with a non-complete status:

assistant text: "The main difference is..."
status: interrupted

Then the UI can distinguish:

completed answer
vs
partial output from interrupted generation

This is better than rendering the same text with no warning.

Partial output is useful even if you regenerate

Keeping it can help:

• users recover information they already saw;
• debugging;
• comparing regenerated output;
• support diagnostics;
• deciding whether manual continuation is acceptable.

The product can still choose not to include the partial text in the next model context.

Persistence and context construction are separate decisions.

Provider response IDs can make recovery much stronger

If the provider returns a durable response/operation identifier early enough, persist it immediately.

Conceptually:

local generation ID: gen_42
provider response ID: resp_abc

On relaunch or reconnect, the client may be able to:

• retrieve final state;
• inspect whether it completed;
• continue from provider state;
• cancel the operation;
• avoid creating a duplicate request.

Whether these operations exist is provider-specific.

Do not invent semantics for an ID merely because the provider calls it a response ID.

Background operations are easier to recover than ephemeral streams

A durable background API can separate execution from delivery:

create operation
→ receive operation ID
→ stream/poll status
→ client disconnects
→ operation continues
→ client later fetches result

This is stronger than assuming the original socket must remain connected until inference ends.

For very long generations, this architecture is often preferable.

See How Long-Running AI Tasks Work.

Unexpected disconnect creates an ambiguity window

Imagine:

client sends request
server accepts request
server begins generation
network disconnects before client sees response ID

The client cannot know from the network error alone whether the operation exists.

This is the same distributed-systems problem seen in payment APIs and job queues:

request outcome = unknown

If the API supports idempotency keys or client-supplied operation IDs, use them according to provider semantics.

If it does not, automatic retry may create a second operation.

Pure text regeneration is usually simpler

Suppose:

user asks a question
model streams 200 tokens
network drops
no tools or external side effects occurred

A practical recovery option is:

keep partial output marked interrupted
→ offer Regenerate
→ start a new assistant turn from last safe context

The regenerated answer may differ, but the semantic risk is low.

You may also offer “Continue” if the application can construct a sensible continuation prompt, but that should be presented as a new generation, not a byte-perfect stream resume.

Do not automatically feed arbitrary partial output back to the model

Suppose the partial text ends mid-sentence:

The API key should be stored in a sec

Sending that back as if it were a completed assistant message can distort context.

Options include:

• exclude interrupted assistant text from regenerated context;
• include it with explicit metadata such as “partial interrupted output”;
• use provider-native continuation state instead;
• let the user choose whether to continue from it.

The right choice depends on the provider and product.

Tools make recovery much more dangerous

Consider:

model → create_calendar_event(...)
client executes tool successfully
network fails before model sends final answer

The conversation UI may show no confirmation, but the calendar event already exists.

If you simply regenerate the user turn, the model may call the tool again.

Recovery must know:

which tool call executed?
what arguments were used?
what result returned?
was the result committed locally?
is the tool operation idempotent?

Persist tool execution before model continuation

A safe tool loop can be:

receive complete tool call
→ validate + authorize
→ create local execution record
→ execute tool
→ persist result/status
→ send tool result back to model

Then a crash between execution and continuation still leaves evidence that the side effect happened.

The application can recover by reusing the persisted result rather than calling the tool again.

Tool call IDs are useful but not sufficient

Provider tool-call IDs help correlate:

model request
→ tool result

But they do not automatically make the external operation idempotent.

A create_issue tool may need an application-level idempotency key or duplicate check.

Distinguish:

provider call correlation ID
vs
side-effect idempotency identity

They may be related, but should not be assumed identical.

Recovery after a read-only tool is easier

For tools like:

search_docs
get_weather
read_file

re-execution is usually lower risk, though results may change.

For tools like:

send_email
transfer_money
create_issue
delete_file

re-execution can be harmful.

Recovery policy should classify tool effects.

Keep a last-safe context checkpoint

A useful concept is the last conversation state that is definitely consistent.

Example:

Turn 12 user message persisted
Turn 12 assistant generation begins
Tool A completed and persisted
Tool result not yet accepted by provider continuation

The last safe checkpoint may include the tool result even though visible final text is missing.

Persist enough semantic state to rebuild from that point.

Conversation storage should support incomplete turns

A strict schema like:

user message
assistant message
user message
assistant message

is too simplistic for reliable streaming.

A turn may contain:

user input
assistant reasoning summary
assistant tool call
local approval
local tool execution
assistant continuation
partial visible output
interrupt

Your persistence model should represent that sequence without forcing everything into one finished string.

See Stateful vs Stateless AI Conversations.

Recovery on app relaunch

At startup, scan for generation records left in non-terminal states:

starting
streaming
tooling
recovering

For each one:

1. determine whether the process exited normally;
2. inspect provider operation ID if available;
3. inspect persisted tool executions;
4. query durable provider state if supported;
5. otherwise mark the generation interrupted;
6. expose an appropriate user action.

Do not blindly restart every generation.

Recovery actions should reflect certainty

Good UI actions might include:

Resume

Only if the same operation can genuinely be resumed/replayed.

Check status

When a durable provider/background task may still be running.

Continue

Start a new model turn using known continuation state.

Regenerate

Start over from the last safe context.

Keep partial

Preserve the output as-is without more inference.

Discard partial

Remove the interrupted assistant turn.

Naming matters. “Resume” should not secretly mean “rerun.”

Reconcile before retrying unknown side effects

If a request may have caused an external mutation, prefer:

query external system
→ determine whether operation already happened
→ only retry if absent

For example:

create GitHub issue
network failure
→ search by idempotency marker/client operation ID
→ reuse existing issue if found

This is distributed-systems recovery, not model-specific magic.

Reasoning state may be provider-specific

Some reasoning models expose continuation state that should be carried forward in provider-native form rather than converted into visible assistant text.

That state may be:

• opaque;
• encrypted;
• tied to one provider/model family;
• invalid after editing prior context.

Store it as provider metadata alongside portable conversation history.

Do not render opaque reasoning continuation data to users or send it to another provider.

See How to Preserve Reasoning Across AI Turns.

Editing history invalidates some recovery paths

If the user edits the prompt while an interrupted generation exists, the old continuation state may no longer correspond to the visible conversation branch.

A clean rule is:

edit before interrupted turn
→ invalidate provider continuation after edit point
→ regenerate from edited branch

Do not silently attach old provider state to new semantic history.

Cancellation has its own recovery rules

If the user intentionally stopped generation, do not automatically resume it on relaunch.

Persist:

status = cancelled
cancel_requested_at
provider_cancel_result = optional

If provider cancellation was uncertain, the app may still reconcile remote state for accounting or side effects, but it should not restart output without user intent.

Detect duplicate generations

Retries can create multiple assistant responses for one user turn.

Track a stable local operation identity:

user_turn_id = turn_20
generation_attempt = 1, 2, 3...

Then the product can show history accurately or keep only the chosen branch.

Do not overwrite an interrupted attempt with a regenerated answer if auditability matters.

Recovery should have bounded automation

Automatic recovery is useful when semantics are clear:

transient fetch of durable status failed
→ retry status query

It is risky when semantics are ambiguous:

unknown side-effecting tool outcome
→ rerun whole agent automatically

A good policy distinguishes:

• safe transport retry;
• safe status retry;
• model regeneration;
• side-effect replay.

Only the first two are usually strong candidates for fully automatic behavior.

Observability fields for interrupted generations

Useful telemetry can record sanitized metadata such as:

provider/model
local generation ID
provider operation ID hash/reference if safe
stage when interrupted
elapsed duration
text bytes/chars received
tool calls completed
recovery action selected
recovery succeeded/failed

Avoid logging:

• prompt text;
• assistant text;
• tool arguments/results;
• credentials;
• sensitive endpoint URLs.

Test the ugly transitions

Important recovery tests include:

• disconnect before headers;
• disconnect before provider ID;
• disconnect after first token;
• disconnect after 90% of text;
• explicit provider failure after partial output;
• app termination during stream;
• app termination after tool side effect;
• app termination after tool result persisted but before continuation;
• provider operation completes while client is offline;
• user cancels during recovery;
• user edits conversation before recovery;
• regenerate creates a second attempt;
• provider continuation ID becomes invalid;
• external tool reports duplicate/idempotent success.

A recovery decision tree

The exact tree varies by provider, but the important point is that side effects and durable operation state come before blind retry.

Recovery checklist

Before shipping interrupted-generation recovery, verify:

• a local generation record exists before inference starts;
• partial text can be persisted as incomplete;
• provider operation IDs are stored when available;
• tool calls and results have durable execution records;
• side-effecting tools have idempotency/reconciliation strategy;
• app relaunch scans incomplete generations;
• user cancellation is distinct from failure;
• editing history invalidates stale continuation state;
• regeneration creates a new attempt intentionally;
• the UI distinguishes Resume, Continue, Regenerate, and Check Status;
• retries are bounded and semantically safe.

Where BYOKchat fits

A local-first BYOK client can own the durable conversation/generation record while provider adapters contribute optional recovery capabilities. One provider may support durable response retrieval, another may only support regeneration, and a local model may have no remote operation to query at all.

The shared product behavior should still preserve partial output, tool history, and a truthful recovery state.

Further reading

• Why AI Streams Break in the Middle
• Designing Reliable AI Retries
• How Long-Running AI Tasks Work
• Stateful vs Stateless AI Conversations
• How to Preserve Reasoning Across AI Turns