How AI Chat Branching and Regeneration Should Work

Chat interfaces make regeneration look simple:

user message
assistant answer
Regenerate

Underneath, regeneration changes the conversation graph.

The same is true when a user edits an old message, resends from the middle of a chat, switches model and retries, retries after a partial stream, or reruns a tool-enabled turn.

The safest mental model is a conversation tree with one currently selected path.

A linear transcript is only a projection

What the user sees may be:

U1 -> A1 -> U2 -> A2 -> U3 -> A3

After regenerating A2, the real structure becomes:

U1 -> A1 -> U2
              |-> A2a -> U3a -> A3a
              |-> A2b

If the UI selects A2b, the visible transcript can still look linear.

The application should preserve the hidden sibling path instead of overwriting it silently.

Why overwrite-in-place is dangerous

Overwriting an old answer can destroy:

the original provider/model;
usage and cost;
tool calls and results;
timing diagnostics;
approval history;
the exact context later messages saw.

That makes debugging, export, and model comparison ambiguous.

Use immutable message identity

A useful message record has a stable identifier:

interface MessageNode {
  id: string
  parentId: string | null
  role: "system" | "user" | "assistant" | "tool"
  content: ContentPart[]
  createdAt: string
  generationId?: string
}

Editing a historical user message should normally create a new node rather than mutate the old node’s semantic identity.

Separate message identity from generation identity

A generation record can store runtime metadata:

interface GenerationRecord {
  id: string
  messageId: string
  providerConnectionId: string
  modelId: string
  startedAt: string
  completedAt?: string
  status: "streaming" | "completed" | "interrupted" | "failed" | "cancelled"
  usage?: UsageRecord
}

This keeps conversation semantics separate from transport/runtime state.

Regeneration creates a sibling assistant node

Given:

U2 -> A2a

regeneration should produce:

U2 -> A2a
   -> A2b

The selected sibling feeds future turns.

Editing a user message creates a new downstream branch

Changing:

Explain TLS

to:

Explain TLS to an iOS developer

creates a new user node and a new downstream answer:

U2a -> A2a
U2b -> A2b

The old descendants remain historical siblings.

Resend and edit are different

Resend unchanged

same user content
new assistant generation

Edit and resend

new user node
new assistant generation

The distinction matters for history and analytics.

Provider and model belong to generation history

A conversation can have a current default, but each generation should preserve the exact provider connection and model that produced it.

This lets one conversation contain:

Anthropic turn
Gemini regeneration
local-model branch

without rewriting history.

Provider-native state complicates branches

Some APIs can continue server-side state rather than replaying the whole transcript.

If the user edits an earlier message, that state may no longer describe the selected branch.

The application should rebuild from portable semantic history or begin a new provider-native chain.

See Stateful vs Stateless AI Conversations.

Do not attach one provider continuation ID to the whole chat

A field such as:

conversation.providerResponseId

is not enough once branching exists.

Provider continuation identifiers belong to the generation/path that created them.

Tool calls make regeneration a side-effect problem

Imagine an old branch called:

send_email(...)

Regenerating that turn must not automatically send the email again.

Historical execution is history; new execution needs fresh validation, authorization, and idempotency handling.

See Idempotency for AI Tool Execution and How to Build Human Approval Into AI Tool Calls.

Tool results have freshness semantics

A read-only historical tool result can be:

reused as historical context
rerun for fresh data
omitted from the new branch

Those choices mean different things.

Do not silently replay old external truth as current truth.

Keep tool execution identity tied to the call

interface ToolExecution {
  id: string
  generationId: string
  toolCallId: string
  toolName: string
  normalizedArguments: unknown
  status: string
  result?: unknown
}

A new branch gets new execution identities even when arguments happen to match.

Preserve interrupted generations

If a stream breaks after useful text arrives, keep the partial response and mark it:

interrupted

Then retry/regenerate as a sibling.

See How to Resume or Recover an Interrupted AI Generation.

Continue and regenerate are different

Regenerate

Creates another answer from the same parent context.

Continue

Extends an interrupted or completed answer using a new generation step.

The internal model should distinguish them even if both use the same provider endpoint.

Completed assistant content should be immutable history

Streaming can update a draft buffer while active.

After completion, modifying the response in place makes historical provenance harder to understand.

If the product allows editing assistant text, represent it as an explicit revision.

Branch selection is user state

A conversation can store:

currentLeafId

The visible transcript is the ancestor chain from that leaf to the root.

Sibling counts can power controls such as:

< 2 / 3 >

without exposing a full graph UI.

A leaf-based model simplifies context building

Given a selected leaf:

A5b

walk parents:

A5b <- U5 <- A4 <- U4 <- ... <- root

Reverse that list and build model context from only that path.

Keep branch creation transactional

An edit can involve:

create replacement user node
change selected branch
create generation record
create assistant draft

If the app crashes halfway through, the conversation should still have a valid selected path.

Use transactions or an operation journal.

Do not delete hidden descendants automatically

Old paths remain useful for:

undo;
branch switching;
export;
model comparison;
analytics;
debugging.

Offer explicit branch deletion if needed.

Branch deletion needs graph cleanup

Deleting a subtree should also clean up unreachable:

generation metadata
tool records
attachment references
derived search records

according to ownership rules.

Attachments belong to the message that introduced them

If an edited user turn removes or replaces a file, the new branch must use the new attachment set.

See How File Attachments Flow Through AI APIs.

Only the selected branch enters active context

Accidentally including sibling branches can:

waste tokens;
introduce contradictory answers;
leak rejected drafts;
confuse tool state.

Siblings are history, not current model context.

Summaries are branch-specific

A long-conversation summary corresponds to a particular ancestor range.

Editing before that range can invalidate the summary.

Store enough provenance to know when a summary must be rebuilt.

See How to Design Context Management for Long AI Conversations.

Snapshot effective generation settings

Useful historical metadata includes:

provider connection
model
reasoning effort
sampling controls when used
output limit
instruction revision
tool set
attachment set
context strategy

Current settings should never be used to explain an old answer retroactively.

Analytics should count all generated branches

A regeneration is a new request and may have real usage/cost even if it is not the currently selected answer.

Track:

conversation
user turn
generation branch
provider attempt
usage
latency
finish status

See How to Estimate Per-Conversation AI Cost.

Search needs an explicit hidden-branch policy

Two valid choices:

search selected paths only
search all history and label alternate branches

See How to Design AI Conversation Search.

Export should preserve branch structure when format allows

Markdown can export the selected path.

Structured JSON can preserve:

node IDs
parent IDs
current leaf
generation metadata
tool events
attachments

See How to Export AI Conversations Portably.

Side effects remain real when a branch changes

Editing a message does not undo:

email sent
issue created
file modified
calendar event created

A useful warning can say:

Editing creates a new branch. Actions already performed on the previous branch are not undone.

Regeneration should keep settings stable unless the user changes them

The least surprising default is:

same parent context
same target
same generation settings
new generation

If the user changes model or reasoning first, the new branch should record the difference visibly.

Branches are a natural model-comparison primitive

same parent
-> Model A answer
-> Model B answer
-> local model answer

The client can compare latency, usage, tool behavior, and user preference without inventing a universal score.

A practical persistence shape

Useful invariants

every non-root message has a valid parent
current leaf belongs to the conversation
only selected-path ancestors enter context
completed generations are immutable historical records
historical approvals never authorize new execution
provider-native state stays attached to the path that created it

Test the graph, not just the renderer

Important cases:

regenerate latest answer
regenerate middle answer
edit middle user turn
switch branch then send
switch model then regenerate
partial stream then regenerate
side-effecting tool on old branch
attachment changed during edit
provider continuation invalidated by edit
summary invalidated by upstream change
export selected path
export full graph
search hidden branch
branch deletion
app relaunch with alternate sibling selected

Where BYOKchat fits

A multi-provider local-first chat client benefits from explicit branching because editing, resend, regeneration, model switching, interrupted-generation recovery, and tool use can all preserve exact history instead of mutating it away.

The UI can remain simple while the underlying graph keeps provider metadata, usage, tool rounds, and alternate answers correct.