Designing Cancel and Stop for Streaming AI

A Stop button looks simple:

user taps Stop
→ generation stops

In a real AI client, several independent operations may be active:

the HTTP response stream;
provider-side inference;
reasoning generation;
a tool call being assembled;
a local or remote tool already executing;
a follow-up model request after a tool result;
local persistence and rendering.

Stopping one does not necessarily stop the others.

The central rule is:

Treat cancellation as an explicit state transition with best-effort propagation, not as “close the socket and assume nothing else happened.”

There are multiple cancellation targets

A generation can involve:

A Stop action can mean different things at each layer.

Local UI cancellation

Stop rendering new output and mark the generation cancelled.

Transport cancellation

Abort the active HTTP/WebSocket operation.

Provider cancellation

Ask the provider to cancel the server-side operation if an API exists.

Tool cancellation

Cancel the currently executing tool if the tool supports cancellation.

Workflow cancellation

Prevent the host from starting any new tool/model rounds after the current cancellation point.

A robust client coordinates all of them.

Cancellation should have its own terminal state

Do not report user cancellation as an error.

Use a model such as:

type GenerationStatus =
  | "starting"
  | "streaming"
  | "tooling"
  | "completed"
  | "cancelled"
  | "interrupted"
  | "failed";

Then analytics and UI can distinguish:

user chose Stop
vs
network failed
vs
provider returned error

The local state transition should happen immediately

When the user taps Stop, the UI should respond without waiting for a network round-trip.

A typical flow:

set cancellationRequested = true
→ disable further model/tool rounds
→ abort active transport
→ attempt provider/tool cancellation
→ persist final local cancellation state

The Stop button should not remain spinning while the app waits for a remote acknowledgement unless that distinction is essential to the product.

Aborting the HTTP request may not cancel inference

If the client closes the connection after the provider already accepted the request, the server may continue generating.

From the client’s perspective:

stream closed

From the provider’s perspective:

operation may still be running

Some providers can notice disconnect and stop work; others may expose an explicit cancellation API; others may not guarantee either.

Do not state stronger cancellation guarantees than the provider documents.

Persist provider operation IDs early

If the provider exposes a durable response/operation ID, store it as soon as possible.

Then cancellation can potentially do:

local generation gen_123
→ provider operation resp_abc
→ cancel resp_abc

This is stronger than only aborting the socket.

It also allows later reconciliation if the cancellation acknowledgement is lost.

Cancellation can race with completion

The classic race:

model finishes
client sends cancel
completion event and cancel cross in flight

Possible local observations:

cancel requested first
completion received first
cancel API says already completed

Define a deterministic policy.

For example:

if user requested cancellation before local completion was observed:
    preserve status as cancelled
    keep any already-received partial text

Or your product may choose to accept a completion that arrived just before the cancel could take effect.

The exact UX can vary, but it should be consistent and tested.

“Stop” should prevent new tool rounds immediately

Suppose the stream currently contains a complete tool call but execution has not started.

If the user taps Stop:

cancel requested
→ do not execute pending tool

That is straightforward.

The host should check cancellation state at the tool-execution boundary even if the provider emitted the tool call milliseconds earlier.

Partial tool calls must never execute after Stop

If the client has only:

{"recipient":"ali

then cancellation simply discards the incomplete executable intent.

Do not attempt to complete or repair it.

See How Streaming Tool Calls Work.

Cancelling an already-running tool is different

A local tool may support cooperative cancellation:

await tool.run(args, signal);

If so, propagate the cancellation token/signal.

But a tool may have already committed a side effect:

email sent
issue created
file deleted
payment submitted

Cancellation cannot reverse reality.

The UI should not imply rollback unless the tool provides an explicit compensating action.

Tool side effects need durable state

Before starting a side-effecting tool, persist an execution record.

Then if Stop happens during execution:

execution status = cancelling / unknown / completed

On recovery, reconcile with the external system rather than assuming the action did not happen.

This is especially important if the app is terminated immediately after cancellation.

Stop can happen between model rounds

A tool-using workflow may look like:

model request 1 completes with tool call
→ tool executes
→ app is about to send result in request 2

If the user taps Stop in that gap, do not start request 2.

The generation state machine must own the entire multi-round workflow, not just one HTTP task.

Stop can happen during reasoning

If visible text has not started yet, the user still expects Stop to work.

Do not tie the button’s enabled state to:

answerText.isEmpty == false

Enable cancellation whenever a generation operation is active.

Reasoning models can spend significant time before answer text appears.

Preserve partial answer text

If the user stops after 300 tokens, a good default is usually to keep them.

Store:

assistant text = partial output
status = cancelled

Then the UI can show that the answer was intentionally stopped.

Discarding immediately can feel destructive.

Reasoning summary can also be partial

If your app displays reasoning summaries, keep their channel separate:

reasoning summary = partial
answer = partial/empty
status = cancelled

Do not merge reasoning into the answer to fill the gap.

See How Streaming Reasoning Differs From Streaming Answers.

Decide what happens to incomplete Markdown

A stopped answer can end inside:

```swift
func foo() {

The renderer should still display the partial Markdown gracefully.

Do not mutate stored source by automatically adding closing syntax.

The canonical assistant text should remain what the model actually emitted.

See How to Render Markdown While AI Is Still Streaming.

Regenerate after cancellation should create a new attempt

If the user later taps Regenerate:

cancelled generation attempt 1
→ new generation attempt 2

Do not silently change the status/content of attempt 1 if the product supports history/branching.

This makes recovery and tool auditing easier.

Stop is not Undo

This distinction should be explicit:

Stop = prevent more work where possible
Undo = reverse completed side effects

Many AI tool actions cannot be undone automatically.

If an app supports Undo for a specific tool, implement it as a separate tool/domain feature.

Cancellation propagation should be cooperative

A shared cancellation token can be checked at boundaries:

before provider request
while reading stream
before tool validation
before approval
before tool execution
inside cancellable tool
before continuation request
before persistence finalization

This reduces races where the workflow keeps progressing after Stop.

But do not let cancellation corrupt persistence

Once Stop is requested, you still need enough execution time to persist a consistent terminal state.

Conceptually:

cancel work
≠ cancel cleanup

Cleanup may include:

flush partial text;
store cancelled status;
persist tool execution outcome;
release resources;
record sanitized timing.

Use a cleanup path that is not itself discarded by the same cancellation signal.

Transport cancellation errors should be expected

Aborting a request often causes the networking API to report a cancellation/abort error.

Do not log that as an application failure when it matches user intent.

Example classification:

network error = cancelled
AND cancellationRequested = true
→ expected terminal path

Unexpected connection resets remain interruptions.

Provider cancellation failure is not necessarily local failure

Suppose:

user taps Stop
local stream aborts
provider cancel endpoint times out

The user intent has still been honored locally: the client will not continue the workflow.

Persist remote status as uncertain if necessary:

local status = cancelled
provider cancellation = unknown

You may reconcile later if provider state matters.

Long-running background jobs need different UX

A background job can continue independently of a live stream.

For such workflows, Stop may mean:

send cancel operation
→ poll/retrieve resulting status

The job might transition through:

cancelling
cancelled
completed before cancellation
failed

Do not force background semantics into a foreground stream abstraction.

See How Long-Running AI Tasks Work.

Disable automatic retries after user cancellation

A dangerous bug:

user taps Stop
→ request aborts
→ generic retry middleware sees transient network error
→ starts request again

Cancellation must bypass retry policy.

The retry layer should receive an explicit cancellation reason/state.

See Designing Reliable AI Retries.

Queue schedulers must remove cancelled work

If generations are queued, Stop may occur before the request even starts.

Then:

remove or mark queued operation cancelled
→ never dispatch provider request

Do not treat it as “cancel failed because no network request exists.”

The generation operation exists at the product level before transport begins.

Multi-device or multi-window clients need operation ownership

If the same chat is visible in multiple windows/devices, define who can cancel an operation.

A local-only app may keep generation ownership in one process.

A synchronized/server-coordinated app may need:

operation ID
owner/session
authorization
cancel command
status broadcast

Otherwise two clients can disagree about whether a generation is active.

Accessibility and interaction details matter

A Stop control should:

be keyboard accessible;
have a clear accessibility label;
not jump position while output grows;
remain available during reasoning/tool phases where cancellation is possible;
change state immediately after activation;
avoid repeated double-cancel requests from rapid tapping.

Disable/debounce duplicate activation after the first cancellation request.

Observability for cancellation

Useful sanitized fields:

provider/model
stage when Stop requested
elapsed duration
partial output size
active tool state
provider cancel supported/attempted
provider cancel result
cleanup duration

Avoid logging model content or tool arguments.

This helps answer:

Do users often stop during long reasoning?
Are tools ignoring cancellation?
Does one provider keep operations alive after disconnect?

A cancellation state machine

For tool workflows, add sub-state for tool cancellation/reconciliation.

Tests that catch cancellation bugs

Test Stop:

before network dispatch;
after request dispatch but before headers;
during visible text;
during reasoning before visible text;
while assembling an incomplete tool call;
after tool call finalization but before approval;
while approval card is open;
during read-only tool execution;
during side-effecting tool execution;
after tool completes but before continuation request;
during continuation stream;
exactly when completion event arrives;
with generic retry middleware enabled;
during app backgrounding;
immediately before app termination;
when provider cancellation API fails;
when provider operation already completed.

Verify persisted state after every case.

A practical implementation sequence

When Stop is requested:

1. atomically mark cancellation requested
2. prevent new workflow steps
3. abort current transport
4. signal current cancellable tool
5. attempt provider-side cancel if supported/relevant
6. persist partial output + tool state
7. mark local generation cancelled
8. update UI
9. optionally reconcile uncertain remote operation later

Order can vary by architecture, but local intent should be recorded before asynchronous remote cancellation work.

Where BYOKchat fits

A multi-provider client can expose one Stop interaction while each adapter optionally implements provider-native cancellation. The shared generation layer owns partial output, multi-round tool state, local cancellation tokens, and terminal status, so providers without explicit cancel APIs still behave predictably.

The UI promise should be precise: Stop prevents the client from continuing the generation workflow and makes best-effort cancellation of work already in progress.