How Long-Running AI Tasks Work: Background Jobs, Webhooks, and Recovery

Some AI requests finish in a second. Others can take minutes.

Long reasoning runs, large document analysis, multi-step tool workflows, batch-style jobs, or heavyweight model calls can exceed the lifetime of a normal foreground HTTP request.

A reliable application therefore needs to distinguish:

request accepted
work still running
work completed

from the simpler synchronous model:

request → wait → response

That distinction changes persistence, retry logic, cancellation, UI state, and recovery.

Why ordinary synchronous requests break down

In a synchronous flow, one network request stays open until generation finishes:

This works well when the request completes quickly and the app remains connected.

It becomes fragile when:

• the model takes several minutes;
• a mobile app backgrounds;
• a reverse proxy has a shorter timeout;
• the network changes;
• the process restarts;
• the user closes the screen;
• a webhook-oriented backend should own completion.

The model may still be working even after your original connection disappears.

Background execution separates creation from completion

A background-capable API accepts work and exposes a durable response/job identity.

Conceptually:

POST create work
→ accepted + job/response ID

later:
GET job/response ID
→ queued | running | completed | failed | ...

OpenAI’s current Responses API exposes a background option for running a model response asynchronously. This is especially useful for models or requests that may take long enough to risk an ordinary request timeout.

The exact states and retrieval API are provider-specific, but the application architecture is general.

A job ID becomes application state

Once work outlives the original request, its provider ID should not live only in a local stack variable.

Persist enough to recover:

application operation ID
provider
provider response/job ID
conversation/turn ID
created time
last known status
last checked time
cancellation state

If the app restarts, this record lets it ask the provider what happened.

Keep your own operation ID too

Provider IDs are protocol identities. Your application may need a stable identity before the provider successfully returns one.

For example:

app operation: op_42
provider response: resp_abc123

The application ID can correlate:

• user action;
• local optimistic UI;
• provider request creation;
• retries;
• webhook events;
• analytics;
• crash recovery.

This is especially useful if the create request has an unknown outcome.

Polling is the simplest completion mechanism

The client periodically retrieves the job status:

Polling is easy to reason about and works without a public callback endpoint.

It is a natural fit for native apps that can check while foregrounded.

Polling should back off

Do not poll a ten-minute job every 100 milliseconds.

A reasonable policy can use increasing intervals:

1s → 2s → 4s → 8s → capped interval

or follow provider-supplied retry guidance where available.

The correct cadence depends on:

• expected task duration;
• provider rate limits;
• UX latency requirements;
• whether the app is foregrounded;
• server guidance.

Polling is not guaranteed to continue on mobile

An iOS app can be suspended after entering the background. You cannot assume a timer will keep firing indefinitely.

That means a durable native workflow should recover when the app becomes active again:

app foregrounds
→ find unfinished operations
→ refresh authoritative provider status
→ update conversation/UI

Background execution is valuable precisely because the provider can keep working when your process cannot.

Webhooks invert the completion flow

A server-side application can expose a webhook endpoint and let the provider deliver lifecycle events.

Conceptually:

OpenAI’s current webhook tooling includes response lifecycle events such as completed and failed events, and its SDK documentation demonstrates webhook signature verification before processing event data.

Webhook event names and availability can evolve, so application code should rely on current provider documentation rather than hard-coding assumptions from another provider.

Verify webhook authenticity before trusting payloads

A public webhook endpoint receives internet traffic.

Do not accept:

{
  "type": "response.completed",
  "data": {"id": "..."}
}

merely because the JSON looks right.

Use the provider’s documented signature-verification mechanism against the raw request body and relevant headers.

Only verified events should mutate durable job state.

Webhooks are usually at-least-once style application inputs

Even if a provider attempts reliable delivery, your handler should tolerate duplicate events.

A safe pattern is:

verify event
→ identify event/job
→ check whether state transition already applied
→ apply idempotently
→ acknowledge

Never assume one event will arrive exactly once.

Event ordering can be surprising

Distributed systems can retry and reorder deliveries.

Your state machine should avoid regressions such as:

completed
↓ receives delayed "running" event
running   ← wrong

Define monotonic transitions where possible:

created → queued → running → terminal

with terminal states such as:

completed
failed
cancelled
incomplete
expired

Provider-specific status names may differ.

Polling and webhooks can coexist

A backend can use webhooks as the main completion signal while retaining polling as a recovery path.

For example:

webhook normally updates job

if no update after expected interval:
    poll provider
    reconcile authoritative status

This protects against missed callbacks, deployment outages, or handler bugs.

The provider is authoritative for provider job state

Your local state might say running while the provider has already completed.

After a crash, reconnect, or suspicious event sequence, refresh from the provider rather than guessing based on elapsed time.

Do not let the model infer:

"It has been five minutes, so the task must be done."

External task state should come from the system executing the task.

Unknown outcomes are the hardest failure

Imagine:

1. app sends create request
2. provider accepts and starts job
3. network drops before app receives response ID

The application does not know whether the job exists.

Blindly retrying can create duplicate work.

This is why create-operation idempotency, client request IDs, or provider-supported deduplication mechanisms matter where available.

When no safe deduplication mechanism exists, model the state explicitly:

creation_outcome_unknown

rather than pretending it cleanly failed.

Cancellation is a state transition, not a button animation

When the user taps Cancel, several things can happen:

local request cancelled before provider accepts
provider job cancellation requested
provider already completed
provider cannot cancel this job type
network failed during cancellation

Represent these distinctly.

A robust internal model might include:

type JobState =
  | "creating"
  | "running"
  | "cancel_requested"
  | "completed"
  | "failed"
  | "cancelled"
  | "unknown";

Do not immediately mark a provider job cancelled just because the user requested cancellation.

A cancelled client connection is not necessarily a cancelled job

Closing an HTTP stream or dismissing a screen may only stop local observation.

If the provider supports explicit job cancellation, use it when the user means “stop the work.”

Otherwise the provider may continue consuming compute after the UI disappears.

Conversation state should point to the durable operation

For chat applications, the assistant turn can reference a long-running generation record:

assistant turn
  status: generating
  operation_id: op_42

On completion:

operation op_42 → completed
assistant turn → attach final output

This separates the user’s conversation model from provider polling implementation.

Do not hold database transactions open while AI runs

A model call that takes minutes should not keep an application transaction or request-scoped lock open.

Prefer:

persist requested operation
commit
start/submit provider work
later update persisted operation

Long-running AI belongs in a workflow/state machine, not one giant synchronous transaction.

Long-running tool workflows are even more complex

A model may generate, call a tool, wait for external work, then continue.

State can include:

model response state
tool execution state
external task state
conversation turn state

Avoid collapsing all of this into one isLoading boolean.

See How AI Tool Calling Works and How Interactive MCP Workflows Work.

Retries need phase awareness

A retry before work is accepted is different from a retry after side effects occur.

Classify the phase:

create request not sent
create sent, outcome unknown
job known and running
status check failed
completion handling failed

Then decide what can safely repeat.

Recreating the entire model job because one status poll timed out is usually wrong.

Completion processing should be idempotent

Suppose two webhook deliveries arrive concurrently.

Both may attempt to:

• write assistant output;
• send a notification;
• update analytics;
• trigger downstream work.

Use a durable completion guard so these effects occur once.

For example:

transaction:
  if operation.completed_at is null:
      persist final output
      mark completed
      enqueue one downstream event

The exact implementation depends on your datastore, but the invariant matters.

Store enough for debugging without storing secrets

Useful metadata includes:

provider
model
operation ID
provider job ID
timestamps
status transitions
retry count
webhook event IDs
duration
error category

Avoid logging:

• API keys;
• private prompt content unless explicitly required;
• sensitive tool outputs;
• webhook signing secrets.

Reliability telemetry does not require copying all user content into logs.

Timeouts should mean something precise

There are multiple timeouts:

HTTP connection timeout
provider job runtime limit
application deadline
user patience / UX timeout
poll timeout
webhook delivery timeout

A local HTTP timeout does not necessarily mean provider execution failed.

Use names that reflect the boundary instead of a generic timeout error everywhere.

Recovery after app or server restart

On startup, scan unfinished operations:

for operation in nonTerminalOperations:
    if providerJobID exists:
        refresh status
    else if creation outcome unknown:
        run safe reconciliation policy

Then update stale UI state.

This is much more reliable than assuming every in-progress job failed when the process stopped.

Notifications can complete the user experience

If a task takes several minutes, the user may leave the app.

A backend-driven product can notify them when work finishes, but the notification should point to durable application state rather than contain the only copy of the result.

If notification delivery fails, opening the app should still show the completed operation.

Background work changes cost controls

A foreground user can stop an obviously runaway generation. A detached job can continue unnoticed.

Set limits:

• allowed models;
• maximum output;
• tool/round budgets;
• wall-clock deadlines;
• per-user concurrency;
• account spending guardrails where appropriate.

Background execution should not mean unbounded execution.

Test the state machine, not only the happy path

A long-running workflow should test:

• immediate completion;
• several-minute completion;
• provider failure;
• create request timeout with unknown outcome;
• duplicate webhook;
• invalid webhook signature;
• webhook arrives after poll already completed job;
• delayed/out-of-order event;
• cancellation race with completion;
• app restart while running;
• backend deployment during execution;
• provider status fetch temporarily fails;
• terminal result processing crashes halfway through.

These are normal distributed-system cases, not exotic edge conditions.

A practical architecture

The operation record is the durable center. Polling and webhooks are merely ways to learn provider state.

A long-running task checklist

Before shipping background AI work, verify that:

• the app has its own durable operation identity;
• provider job IDs are persisted;
• unfinished jobs recover after restart;
• polling uses reasonable backoff;
• webhook signatures are verified;
• duplicate/out-of-order events are tolerated;
• completion processing is idempotent;
• cancellation is reconciled with provider state;
• unknown create outcomes are modeled explicitly;
• terminal state comes from the authoritative provider;
• long work does not hold request/database transactions open;
• cost and runtime limits exist.

Where BYOKchat fits

A native BYOK client can model long-running provider operations separately from chat rendering. Provider adapters can expose background-capable APIs where supported, while the shared conversation layer persists turn status and recovers unfinished work after foreground/background transitions or relaunch.

That makes long-running reasoning a reliability feature instead of a fragile long-lived network request.

Further reading

• Designing Reliable AI Retries
• How AI Streaming Works
• How AI Tool Calling Works
• Stateful vs Stateless AI Conversations