Idempotency for AI Tool Execution

An AI tool loop can fail at the worst possible moment:

model requests send_email
→ app sends email successfully
→ network/app crashes
→ tool result never reaches model
→ workflow retries

If the retry executes the tool again, the user gets two emails.

The same problem is more serious for:

payments;
ticket creation;
file deletion;
calendar events;
database writes;
purchases;
messages;
deployments.

The solution is idempotent execution: make repeated attempts for the same logical tool action produce one committed side effect.

Idempotency is about logical operations, not HTTP requests

A retry can create a new HTTP request while representing the same intended action.

These two requests may be different packets:

POST attempt 1
POST attempt 2

but the application should know they both represent:

operation = send email for tool call call_123

Idempotency requires a stable identity above transport retries.

Give every side-effecting call a durable execution ID

A useful key can derive from:

conversation/operation ID
+ model round
+ provider tool call ID
+ tool identity/version

For example:

op_77:round_3:call_123:send_email:v1

Do not generate a fresh idempotency key for each retry. That defeats the purpose.

Persist before executing

The safest state machine records intent first:

pending
→ executing
→ committed
→ result_delivered

A durable record might be:

struct ToolExecutionRecord {
  let executionID: String
  let toolName: String
  let normalizedArgumentsHash: String
  var status: Status
  var externalOperationID: String?
  var result: ToolResult?
  var startedAt: Date?
  var committedAt: Date?
}

The exact schema varies, but the important property is that the app can answer:

Did this logical call already commit?

Bind the key to normalized arguments

Reusing an idempotency key with different arguments is dangerous.

Store a digest of the canonical normalized request:

key = op_77:call_123
argsHash = SHA256(canonical(toolName + normalizedArgs))

On retry:

same key + same hash → replay/recover
same key + different hash → reject as conflict

This prevents a stale approval or model mutation from changing the operation under an existing key.

Downstream API idempotency is ideal

Some external APIs accept an idempotency key directly.

Conceptually:

POST /payments
Idempotency-Key: op_77-call_123

If the first request succeeded but the response was lost, the second request can return the original result rather than creating another payment.

When available, use the downstream service’s idempotency mechanism and keep an app-owned execution record.

The downstream guarantee protects the remote side effect; the app record protects orchestration and recovery state.

Do not assume every POST is non-idempotent or every PUT is safe

HTTP method semantics can help, but application behavior is what matters.

Examples:

POST /search → read-only
POST /send-email → side effect
PUT /counter/increment → may still duplicate effect if semantics are odd
DELETE /resource/123 → often idempotent, but audit/notifications may not be

Classify tools by actual business behavior.

Read-only tools need less machinery

A weather lookup can normally be retried because repeating it does not create a durable side effect.

Still consider:

rate limits;
cost;
non-deterministic results;
external load.

Idempotency is most critical for writes, but retry discipline applies everywhere.

The unknown-outcome state is the hard case

Consider:

request sent
→ connection drops before response

There are two possibilities:

side effect never happened
side effect happened, response was lost

The app cannot infer which one from the network error.

Represent this as:

outcome_unknown

not simply failed.

Then recovery can reconcile with the external service before retrying.

Reconcile before replay when possible

If the external service returns an operation ID or supports lookup by idempotency key, query it:

GET status by idempotency key
→ committed? reuse result
→ not found? safe to retry under same key

For systems without idempotency lookup, use domain-specific reconciliation:

search for calendar event with app operation tag
look up payment reference
check sent-message record

Do not blindly retry an unknown-outcome write.

Store external IDs immediately

If a side effect returns:

{
  "event_id": "evt_456"
}

persist that identifier before starting another model round.

Then recovery can reconstruct the tool result:

{
  "ok": true,
  "event_id": "evt_456"
}

without creating a second event.

The model should not create idempotency keys

The application controls execution identity.

Do not add:

{
  "idempotency_key": "model-generated-value"
}

to the tool schema unless the domain specifically requires a model-visible business reference.

The model can repeat, hallucinate, or mutate keys.

Generate them from trusted orchestration state.

Approval must bind to the same execution

Suppose the user approves:

Send $50 to account A

The approval record should bind to:

execution ID
normalized arguments hash
amount/target preview

If the model later changes the amount to $500, the execution ID/arguments no longer match and approval must be requested again.

Idempotency and approval integrity reinforce each other.

Parallel calls need separate keys

If one model turn requests:

send_email(A)
send_email(B)

these are two logical operations.

Use separate keys:

call_A → key_A
call_B → key_B

Do not reuse one batch key unless the downstream system provides true atomic batch semantics and your application models that intentionally.

Batch side effects need explicit atomicity semantics

If the model asks to create five records, decide whether the tool means:

all-or-nothing batch

or:

five independent records

This affects recovery.

For independent items, persist per-item completion so a crash after item 3 resumes at item 4 rather than duplicating 1–3.

For atomic transactions, let the database/service transaction define the commit boundary.

A transaction does not replace idempotency across retries

A database transaction ensures one attempt is internally atomic.

It does not stop two successful retries from running the same transaction twice.

Use both when needed:

transaction → one attempt is atomic
idempotency → repeated attempts represent one logical operation

Tool results should be replayable

Once a side effect commits, save enough result data to return the same semantic result to the model after recovery.

For example:

{
  "ok": true,
  "message_id": "msg_912",
  "sent_at": "2026-09-04T08:20:00+07:00"
}

A duplicate model round should receive the persisted result rather than re-executing the tool.

Result replay must not leak stale mutable data unintentionally

Some results are snapshots.

If the original operation created a task and returned its current title, replaying the stored result later is appropriate for reconstructing the original tool round.

If the model needs the current task state, it should call a read tool separately.

Do not mutate historical tool results during replay.

Tool retries and model retries are different

A transient network failure while executing a read-only tool may be retried inside the executor without another model round.

A model retry regenerates a new model output and may choose different calls.

Keep these layers separate:

executor retry → same logical tool call
model retry → new model generation attempt

Only the first should reuse the same tool execution identity automatically.

Regeneration can create a new logical operation

Suppose the user regenerates an assistant turn that previously sent an email.

You should not automatically replay the old side effect just because the prompt is similar.

A regenerated model turn is normally a new operation context.

The UI may need to warn that regenerating a side-effecting branch cannot undo previous actions.

Conversation branching and real-world side effects are different histories.

Cancellation after commit cannot undo the side effect

If the user presses Stop after the payment/email already committed:

mark tool = committed
mark parent operation = cancelled_after_side_effect

Do not lie by changing the tool record to cancelled.

If the domain supports compensation (refund, delete event), that is a separate operation with its own authorization/idempotency key.

Compensation is not rollback

For distributed systems, you often cannot roll back a remote action atomically.

Instead:

send payment
→ later issue refund

or:

create calendar event
→ later delete event

These are compensating actions and should be modeled as new audited tool executions.

Durable state beats in-memory flags

This is insufficient:

var alreadySent = true

if the process can crash.

For meaningful side effects, persist the execution record in durable storage before and after commit boundaries.

Mobile apps are especially likely to be suspended or terminated mid-workflow.

Background jobs need the same identity

If a tool delegates work to a background job:

tool call → create job → job completes later

store:

tool execution ID
provider/job ID
status
result

Polling and webhooks should reconcile into the same record rather than creating new execution identities.

See How Long-Running AI Tasks Work.

Webhooks are often duplicated

Webhook delivery is commonly at-least-once.

If two identical completion events arrive, process them idempotently:

webhook event ID already processed?
→ ignore/reconcile

Then update the same tool execution record.

Define a state machine explicitly

A useful state model:

prepared
→ approved (if needed)
→ executing
→ committed
→ result_delivered

with side states:

rejected
failed_before_commit
outcome_unknown
cancelled_before_commit

Transitions should be validated; do not let random code paths overwrite committed with failed after a response-delivery error.

A reference execution flow

Test crash boundaries deliberately

Simulate crashes at:

after prepared record
before downstream request
while request is in flight
after remote commit before local persist
after local committed persist before model result
after model result sent before conversation save

The system should never create an unexplained duplicate side effect.

Test downstream behavior too

Include cases where the external API:

supports idempotency keys
rejects duplicate keys with changed body
returns original response on replay
has no idempotency support
timeouts after committing
returns 500 before commit
returns 500 after commit with lookup available

Your executor policy should be domain-specific.

Observability should expose replay/reconciliation

Useful metrics:

tool execution attempts
idempotent replays
outcome_unknown count
reconciliation success
prevented duplicate count
side-effect latency

Do not log sensitive arguments just to obtain these metrics.

Where BYOKchat fits

A provider-neutral tool layer can assign durable execution IDs independently of OpenAI/Anthropic/Gemini/MCP call formats. Provider call IDs become correlation metadata, while the application owns idempotency, approvals, durable commit state, and recovery.

That is what lets a multi-round AI workflow survive retries without turning uncertainty into duplicate real-world actions.