On this page
- Wait for the complete tool call
- Parse JSON strictly
- Validate against a trusted schema
- Reject unknown fields by default
- Normalize only deterministic representations
- Schema validity is not semantic validity
- Authorization comes after parsing, before execution
- Prefer stable IDs for mutations
- Validate paths against an allowed root
- Validate URLs before fetching
- Validate email/message recipients from trusted context
- Limit arrays and text sizes
- Treat null, omitted, and empty values separately
- Handle model hallucination as a normal validation failure
- Never expose internal stack traces to the model
- Human approval belongs after validation
- Bind approval to the exact normalized arguments
- Side-effect tools need idempotency
- Apply timeouts and cancellation
- Result validation matters too
- A reference executor
- Test adversarial arguments
- Where BYOKchat fits
- Further reading
A model-generated tool call is untrusted input.
Even when the provider constrains arguments with JSON Schema, the application should not jump directly from:
model output
to:
execute side effect
A safe pipeline is:
accumulate complete call
→ parse
→ schema validate
→ normalize
→ semantic validate
→ authorize
→ preview/approve if needed
→ execute with limits
→ persist result
Each step addresses a different failure class.
Wait for the complete tool call
Streaming APIs may deliver tool arguments incrementally.
Partial input can look like:
{"path":"/Users/al
Do not parse or execute on every delta.
Accumulate the call until the provider’s content/tool boundary says it is complete.
Then validate once against the final structured input.
Parse JSON strictly
If the provider returns arguments as a JSON string, parsing can fail.
Possible problems:
truncated JSON
trailing text
invalid escape sequences
wrong top-level type
duplicate/ambiguous fields
Treat parse failure as a tool-call failure, not as an excuse to run a best-effort string parser for destructive actions.
For low-risk workflows you can ask the model to repair the call in another round.
Validate against a trusted schema
The schema used for validation should come from your application/tool registry—not from the model response.
Check:
- required fields;
- field types;
- enums;
- array bounds;
- string/number constraints;
- unknown fields;
- nested shape.
A provider’s schema-constrained generation improves probability of valid output, but local validation remains defense in depth.
Reject unknown fields by default
Suppose the expected arguments are:
{
"project_id": "proj_123",
"name": "New Name"
}
and the model returns:
{
"project_id": "proj_123",
"name": "New Name",
"force_delete": true
}
Ignoring unknown fields may be safe in this example, but it can hide model confusion or protocol drift.
For security-sensitive tools, rejecting unexpected fields makes failures visible.
Normalize only deterministic representations
Useful normalization includes:
trim surrounding whitespace
canonicalize enum case
parse ISO date into Date
normalize phone number using known locale rules
resolve relative path under an allowed root
Avoid “smart” fuzzy normalization that changes meaning.
For example, do not silently convert:
"next Friday"
into a date without a defined timezone/current-date interpretation if the tool has consequences.
Schema validity is not semantic validity
This can pass schema validation:
{
"amount": 1000000,
"currency": "USD"
}
but violate application rules.
Semantic validation can enforce:
amount > 0
amount <= transfer limit
currency supported by account
recipient exists
operation allowed in current state
Business invariants belong in application code.
Authorization comes after parsing, before execution
Never let model arguments choose an authorization scope the user does not have.
For example:
{
"tenant_id": "other-company",
"document_id": "doc_123"
}
should not be accepted merely because both are valid strings.
Bind authorization from trusted application/session state:
authenticated tenant
+ requested document
→ authorization check
The model is not an identity provider.
Prefer stable IDs for mutations
If a tool mutates an object, validate an exact stable ID rather than a fuzzy name where possible.
Bad:
{ "project": "Production" }
Better:
{ "project_id": "proj_812" }
Use a read-only search/list tool first if the model needs to discover that ID.
Validate paths against an allowed root
For filesystem tools, never concatenate raw model strings:
root + modelPath
without canonicalization and containment checks.
Defend against:
../
absolute paths
symlink escapes
Unicode/path normalization tricks
A file-reading tool should be scoped to explicitly allowed directories/resources.
Validate URLs before fetching
A model-generated URL can target:
public web
localhost
private LAN
cloud metadata endpoints
file://
custom schemes
Apply a URL/network policy before any fetch.
For tools that should access public web only, block private/link-local/loopback targets after DNS resolution as appropriate to prevent SSRF-style behavior.
See How to Handle Untrusted AI-Generated URLs.
Validate email/message recipients from trusted context
For communication tools:
- parse addresses;
- resolve saved contacts if required;
- display recipients in approval UI;
- enforce organization/domain policy;
- cap recipient count.
Do not let hidden BCC-like fields appear unless the schema explicitly supports them and the preview shows them.
Limit arrays and text sizes
A model can accidentally generate huge payloads.
Enforce maximums such as:
20 search terms
10 recipients
100 IDs
20 KB message body
The exact numbers are product-specific.
Limits protect downstream services and keep approval UI understandable.
Treat null, omitted, and empty values separately
These may have different meaning:
field omitted → use default
field null → explicitly clear value
field "" → set empty string
Your schema/domain model should define the difference.
Do not normalize all three into one value automatically.
Handle model hallucination as a normal validation failure
Models can invent:
- enum values;
- IDs;
- fields;
- unsupported operations.
The executor should return a structured failure:
{
"ok": false,
"error": {
"code": "invalid_argument",
"field": "project_id",
"message": "Project does not exist."
}
}
Then the model can recover by searching or asking the user.
Never expose internal stack traces to the model
A tool exception can contain:
- database schema;
- internal paths;
- secrets;
- service hostnames;
- source-code details.
Map exceptions to safe error results.
Keep detailed diagnostics in protected local logs when needed.
Human approval belongs after validation
Do not ask the user to approve malformed raw arguments.
First normalize and validate, then render a semantic preview:
Delete file?
~/Project/build/output.zip
Size: 43 MB
The user approves the exact operation the executor will perform.
If arguments change after approval, approval is invalid.
Bind approval to the exact normalized arguments
A strong approval record can contain a digest of:
tool name
normalized arguments
target identity
conversation/operation ID
Then the executor can verify that the approved operation is the one being run.
Do not approve send_email generically and let the model modify recipients afterward.
Side-effect tools need idempotency
Once a validated write executes, store a durable execution record before continuing the model loop.
If the network/app crashes after the side effect but before the result is returned, the retry should not repeat the action blindly.
Use an idempotency key when the downstream API supports one.
See Idempotency for AI Tool Execution.
Apply timeouts and cancellation
Validation cannot make an external tool reliable.
Execution should still have:
per-tool timeout
cancellation token
response size cap
retry policy
Do not let one stuck tool freeze an unlimited model loop.
Result validation matters too
A tool can return malformed or unexpectedly large data.
Before adding it to model context:
- validate expected result shape;
- redact secrets;
- truncate/limit huge payloads;
- preserve provenance;
- mark errors explicitly.
Tool results are untrusted input to the model, especially when they contain external content.
A reference executor
async function execute(call: ToolCall, context: AuthContext) {
const tool = registry.require(call.name);
const raw = parseJSON(call.arguments);
const typed = tool.schema.validate(raw);
const normalized = tool.normalize(typed);
tool.authorize(normalized, context);
if (tool.requiresApproval(normalized)) {
await approvals.require(tool.preview(normalized));
}
return await tool.run(normalized, {
timeout: tool.timeout,
cancellation: context.cancellation,
});
}
Real code needs more error handling, but the order is the key idea.
Test adversarial arguments
Include:
invalid JSON
unknown field
missing required field
wrong enum
negative number
overflow/huge number
oversized array
path traversal
private-network URL
wrong tenant ID
stale object ID
Unicode edge cases
arguments changed after approval
duplicate execution attempt
Tool safety lives in these edges.
Where BYOKchat fits
A provider-neutral client can take tool calls from Anthropic, Gemini, DeepSeek, OpenAI-compatible models, or MCP-backed workflows and run them through one trusted validation/permission executor. Provider-specific parsing ends before authorization begins.
That keeps the application’s security model independent from model behavior.