How to Debug AI Tool-Calling Loops

A tool-calling loop can fail even when every individual component appears to work.

The model produces a valid tool call. The tool succeeds. The result is sent back. Then the model calls the same tool again. Or it asks for impossible arguments. Or it never recognizes the result you returned. Or the application silently loses the relationship between a tool call and its result.

These failures are difficult to debug if your only log is:

Tool failed.

The solution is to treat a tool loop as a state machine and trace every boundary.

Start with the actual loop

A basic multi-round tool workflow looks like this:

When the loop breaks, identify which edge is wrong.

Do not jump immediately to “the model is confused.”

The bug may be in:

• tool schema design;
• streaming assembly;
• argument parsing;
• authorization;
• execution;
• result serialization;
• call/result identifiers;
• conversation reconstruction;
• provider adapter behavior;
• loop termination;
• retries;
• persistence.

Give every round an identity

A useful trace needs stable IDs.

type ToolTrace = {
  requestId: string
  conversationId: string
  modelTurn: number
  toolCallId: string
  toolName: string
  startedAt: string
}

The exact fields are less important than being able to answer:

Which model turn created this tool call, and which later message contains its result?

Without that relationship, repeated or missing calls become guesswork.

Log the state transition, not just the payload

A compact event log is more useful than dumping entire prompts.

{"event":"model_turn_started","turn":3}
{"event":"tool_call_completed","callId":"call_17","tool":"weather_lookup"}
{"event":"tool_validation_passed","callId":"call_17"}
{"event":"tool_execution_started","callId":"call_17"}
{"event":"tool_execution_completed","callId":"call_17","durationMs":184}
{"event":"tool_result_attached","callId":"call_17"}
{"event":"model_turn_started","turn":4}

This lets you inspect control flow without logging private tool results or user conversations.

Failure class 1: invalid arguments

A model may select the correct tool but produce invalid arguments.

Example schema:

{
  "type": "object",
  "properties": {
    "city": { "type": "string" },
    "days": { "type": "integer", "minimum": 1, "maximum": 10 }
  },
  "required": ["city", "days"]
}

Model output:

{
  "city": "Tokyo",
  "days": "next week"
}

Do not silently coerce everything.

Return a structured error that the model can understand:

{
  "status": "error",
  "code": "invalid_arguments",
  "details": {
    "field": "days",
    "expected": "integer 1-10"
  }
}

Then trace whether the next turn repairs the argument or repeats the same invalid call.

Read How to Validate AI Tool Arguments Safely for the validation boundary.

Do not hide validation errors from the model

A common bug looks like this:

model -> invalid call
client rejects call
client retries model with no tool result
model -> same invalid call

From the model’s perspective, nothing changed.

If the tool call was rejected, include an explicit tool-result error in the next context when the provider’s tool protocol expects one.

The model needs evidence that its previous attempt failed.

Failure class 2: the same tool is called repeatedly

Repeated tool calls have several possible causes.

Cause: result was never attached

The application executes:

lookup_order(123)

but the next model request accidentally omits the result.

The model reasonably concludes it still needs the lookup.

Cause: call/result IDs do not match

The model created:

call_abc

but your adapter returns a result associated with:

call_xyz

Some APIs require an explicit relationship between tool call and tool result. If that link is wrong, the provider may reject the request or the model may not treat the result as completion of the requested action.

Cause: result does not answer the requested question

The model asks:

find_issue(number=42)

The tool returns:

{"status":"success"}

but no issue content.

The model may call again because the result is semantically incomplete.

Cause: schema encourages retries

A vague tool description can make the model uncertain about whether the tool performed a read or initiated an action.

Compare:

process_order

with:

get_order_status

and:

submit_order_cancellation

Clear tool boundaries reduce loop ambiguity.

Detect identical repeated calls

Compute a canonical fingerprint:

function fingerprint(call: ToolCall) {
  return hash(canonicalJson({
    name: call.name,
    arguments: call.arguments
  }))
}

Track repetitions inside one agent run:

if (sameFingerprintCount >= 3) {
  stop("repeated_identical_tool_call")
}

Do not automatically block the second identical call in every workflow. A repeated read may be legitimate if state can change.

But several identical calls with identical results are a strong stuck-loop signal.

Failure class 3: parallel calls are treated as dependent

Suppose a model emits two calls in one turn:

get_customer(42)
get_invoice(9001)

These calls were proposed together. One should not be assumed to depend on the result of the other unless the protocol or application explicitly represents that dependency.

If the second call truly needs output from the first, the correct shape is normally another model round:

model -> get_customer
client -> customer result
model -> get_invoices(customerId)

A client that invents dependencies among sibling calls can create deadlocks or incorrect scheduling.

See How to Execute Parallel AI Tool Calls Safely for concurrency rules.

Failure class 4: one parallel call fails

Imagine three calls:

A succeeds
B fails validation
C succeeds

Do not accidentally discard A and C just because B failed.

The next model turn should receive a complete representation of the round according to the provider’s expected format.

Conceptually:

[
  {"callId":"A","status":"success","result":{}},
  {"callId":"B","status":"error","code":"invalid_arguments"},
  {"callId":"C","status":"success","result":{}}
]

Then the model can decide whether B needs repair while retaining the successful information.

Your provider adapter may require a different wire representation, but the internal state should preserve all outcomes.

Failure class 5: a tool succeeds twice

This is much worse for side effects than repeated reads.

Example:

send_invoice_email(...)

The app executes it successfully, loses the response during a network transition, then retries the tool execution.

Now the customer receives two emails.

This is not primarily a model problem. It is an execution-idempotency problem.

Persist a durable operation key before the side effect:

type Operation = {
  key: string
  toolCallId: string
  state: "pending" | "completed" | "failed"
  resultRef?: string
}

If the same operation appears again, reconcile with the stored outcome instead of blindly executing again.

See Idempotency for AI Tool Execution.

Failure class 6: approval loops

A user denies a tool call.

Then the next model turn proposes exactly the same call.

The app asks again.

The user denies again.

This can continue forever unless denial becomes part of loop state.

Return a clear result:

{
  "status": "error",
  "code": "user_denied",
  "message": "The user declined this action. Do not repeat it unless the user changes the request."
}

Also apply a deterministic local limit.

if (deniedFingerprintCount >= 2) {
  disableCallForCurrentRun(fingerprint)
}

The model should be informed, but the application should enforce the stop condition.

Failure class 7: the tool result is too large

A tool may return thousands of rows, an entire repository file, or a massive web page.

That can cause:

• context-window pressure;
• provider request rejection;
• high input cost;
• slow follow-up turns;
• model confusion;
• lost important instructions.

Trace result size before sending:

{
  "event": "tool_result_prepared",
  "callId": "call_17",
  "bytes": 842194,
  "estimatedTokens": 211000
}

Then apply tool-specific limits.

Better tool:

search_logs(query, limit=50)

instead of:

read_all_logs()

Tool design is part of context management.

Failure class 8: the model never finishes

An agent loop should have explicit budgets.

At minimum consider:

const budget = {
  maxModelTurns: 10,
  maxToolCalls: 20,
  maxWallTimeMs: 120_000
}

Side-effecting workflows may need additional limits:

maxExternalWrites: 3
maxDestructiveCalls: 1

When a limit is hit, stop with an explicit terminal reason.

Do not pretend the run completed normally.

{
  "finishReason": "tool_loop_limit",
  "modelTurns": 10,
  "toolCalls": 17
}

A loop needs a terminal-state model

Avoid a single boolean such as:

isRunning: boolean

Use meaningful states:

type RunState =
  | "awaiting_model"
  | "awaiting_approval"
  | "executing_tools"
  | "awaiting_tool_results_submission"
  | "completed"
  | "cancelled"
  | "failed"
  | "limit_reached"

Now logs and UI can tell you where the run actually stopped.

Failure class 9: streaming assembly corrupts tool arguments

Tool arguments may arrive incrementally during streaming.

A stream can conceptually emit:

{"city"
:"San
 Francisco"}

If you attempt JSON parsing on every fragment, you will generate false errors.

Accumulate fragments according to the provider’s event semantics, and validate only when the tool call is complete.

Trace both phases:

tool_call_delta
tool_call_delta
tool_call_completed
arguments_parse_started

This makes it obvious whether corruption occurred during transport assembly or during schema validation.

See How Streaming Tool Calls Work for the stream boundary.

Failure class 10: adapter normalization loses information

A provider-neutral client usually converts provider-specific events into internal types.

For example:

type NormalizedToolCall = {
  id: string
  name: string
  argumentsJson: string
  providerMetadata?: unknown
}

If the adapter drops a provider-required identifier or state item, the next round may fail even though the internal tool execution succeeded.

Debug both representations:

provider event
    ↓
normalized call
    ↓
executed result
    ↓
provider-native result encoding

When possible, log safe structural metadata at each transformation.

Do not log credentials or private payloads just to make debugging convenient.

Failure class 11: conversation reconstruction is wrong

A persistent chat application may rebuild model context from a database before every request.

If tool rounds are stored incompletely, a restored conversation can become invalid.

Persist enough information to reconstruct:

• model turn order;
• tool call IDs;
• tool names;
• tool arguments or a safe durable representation;
• result status;
• result content or reference;
• provider-native continuation metadata when required;
• cancellation or partial states.

A conversation that visually looks correct in the UI may still be structurally invalid for the provider API.

This is one reason a provider-neutral message model needs explicit tool-round types rather than flattening everything into assistant text.

Failure class 12: retries happen at multiple layers

You may have retries in:

• HTTP client;
• provider adapter;
• tool executor;
• background job;
• agent loop.

These layers can multiply each other.

For example:

3 provider retries
× 3 tool retries
× 2 outer run retries
= up to 18 attempts

The exact multiplication depends on where failures occur, but the principle is important: retries must have one overall budget.

Read Designing Reliable AI Retries for bounded retry design.

Distinguish retry from another model decision

These are not the same:

retry exact HTTP request

and:

ask model what to do after an error

The first repeats an operation at the transport layer.

The second creates a new semantic turn that may choose different arguments or a different tool.

Your trace should distinguish them.

{"event":"transport_retry","attempt":2}

versus:

{"event":"model_recovery_turn","turn":5}

Otherwise a loop can look like mysterious duplicate activity.

Trace tool results by digest when content is sensitive

You often need to know whether two results were identical without storing them in logs.

Use metadata such as:

{
  "callId": "call_17",
  "status": "success",
  "resultBytes": 4821,
  "resultDigest": "sha256:..."
}

Now you can detect:

same call -> same result -> same next call

without retaining the entire private result.

Build a replayable simulator

Tool-loop bugs become much easier to reproduce when your provider adapter can replay deterministic events.

A fixture might specify:

{
  "turns": [
    {
      "model": {
        "toolCalls": [
          {"id":"c1","name":"lookup","arguments":{"id":42}}
        ]
      }
    },
    {
      "afterToolResult":"fixture:lookup-42",
      "model": {
        "text":"The order is ready."
      }
    }
  ]
}

Then test variations:

• malformed arguments;
• duplicate call ID;
• missing result;
• tool timeout;
• user denial;
• partial stream;
• cancellation;
• provider disconnect;
• repeated identical call.

Deterministic failure injection is much more useful than waiting for a live provider to reproduce a rare loop.

Record why the loop stopped

A completed tool run should have a terminal reason.

Examples:

final_answer
user_cancelled
provider_error
tool_error_unrecoverable
approval_denied
tool_loop_limit
wall_clock_limit
context_limit

This improves analytics and debugging.

If every abnormal stop is recorded as error, you lose useful distinctions.

A useful debug timeline

For one run, aim to reconstruct something like:

00.000  model turn 1 started
00.620  tool call c1 completed: search_docs
00.623  validation passed
00.630  execution started
00.812  execution completed
00.816  result attached to c1
00.820  model turn 2 started
01.401  tool call c2 completed: open_doc
01.404  approval not required
01.409  execution started
01.451  execution completed
01.455  result attached to c2
01.459  model turn 3 started
02.108  final answer completed

For a broken run:

00.000  model turn 1
00.520  c1 lookup_order
00.700  c1 success
00.710  result attached
00.720  model turn 2
01.201  c2 lookup_order [same fingerprint]
01.390  c2 success [same result digest]
01.400  model turn 3
01.882  c3 lookup_order [same fingerprint]
01.883  stopped: repeated_identical_tool_call

The bug is now obvious.

Do not let debugging bypass security

Developer tooling should not become a secret exfiltration path.

Avoid convenience logs that dump:

• API keys;
• OAuth tokens;
• authorization headers;
• full private files;
• tool results containing credentials;
• hidden provider reasoning.

Prefer structural traces, redacted previews, hashes, lengths, and explicit opt-in debug captures.

A debugging checklist

When a tool loop behaves incorrectly, inspect these in order:

1. Did the provider emit the tool call you think it emitted?
2. Were streaming fragments assembled correctly?
3. Did the normalized call preserve its ID and name?
4. Did schema validation produce the expected arguments?
5. Did authorization alter or deny the call?
6. Did the tool execute once or multiple times?
7. Was the outcome persisted before any risky retry?
8. Was the result associated with the correct call ID?
9. Was every sibling call in the same round represented?
10. Did the next provider request actually include the result?
11. Did provider-native continuation state survive normalization?
12. Did the model receive a clear error for failures and denials?
13. Are identical calls repeating?
14. Are retry layers multiplying attempts?
15. Did the run hit a deterministic turn/tool/time budget?
16. Is the terminal reason recorded accurately?

That sequence usually narrows the problem to one boundary quickly.

Where BYOKchat fits

A provider-neutral client needs to preserve tool calls and results as structured conversation state, expose approvals visibly, and keep enough execution metadata to recover interrupted multi-round flows.

Those same structures make debugging possible.

The central lesson is simple:

Do not debug an AI tool loop as a blob of model behavior. Debug it as a deterministic state machine surrounding a probabilistic model.

The model can choose surprising actions. Your application should still be able to explain exactly what happened next.