Hosted Tools vs Client-Executed Tools

Tool calling is often described as one feature, but there are two very different execution architectures hiding behind that phrase:

1. hosted tools, where the model provider operates the tool or execution environment;
2. client-executed tools, where your application receives a tool request and performs the action itself.

They can look similar in a chat UI. Architecturally, they are not the same.

The distinction changes who holds credentials, where data flows, how authorization works, how portable your conversation is, and what happens when a request fails halfway through.

The shortest useful distinction

A hosted tool runs on infrastructure controlled by the AI provider or a service directly integrated into the provider-side execution path.

A client-executed tool runs through your application.

versus:

That one placement decision changes almost everything downstream.

Hosted tools can reduce integration work

Provider-hosted tools can be attractive because the provider handles much of the orchestration.

Depending on the provider and tool, the platform may handle parts of:

• tool selection;
• search or retrieval execution;
• browser-like fetching;
• code execution;
• file indexing;
• result packaging;
• retries;
• state associated with the tool run;
• streaming tool events back to the client.

The application may only need to enable the capability in the request and render the resulting events.

That can be significantly simpler than building your own executor.

But simplicity comes from moving responsibility somewhere else, not from making the responsibility disappear.

Client-executed tools give the application more control

With client-executed tools, the provider usually decides that a tool should be called and returns a structured request.

Your application then owns the rest:

model proposes tool call
        ↓
client validates arguments
        ↓
client checks authorization
        ↓
client executes tool
        ↓
client records result
        ↓
client sends result back to model

This architecture requires more code, but it gives the client direct control over:

• credentials;
• authorization;
• network boundaries;
• local resources;
• retries;
• idempotency;
• logging;
• persistence;
• human approval;
• tool portability across providers.

Compare the two architectures

Concern	Hosted tool	Client-executed tool
Execution location	Provider/service infrastructure	Your app or your backend
Credentials	Often provider-managed or provider-linked	Controlled by your app
Authorization UX	Provider-defined or limited	Fully application-defined
Local data access	Usually indirect/upload-based	Can access local resources directly
Portability	Often provider-specific	Can be provider-neutral
Retry semantics	Partly provider-defined	Application-defined
Observability	Depends on provider events	Full local instrumentation possible
Latency	Fewer client round trips in some flows	Extra model-tool-model round trips
Privacy boundary	Data may move through provider tool infrastructure	Can keep execution local, depending on tool
Failure recovery	Provider-dependent	Application-controlled

Neither column is automatically better.

The right choice depends on the tool.

Search is a good example

Suppose a model needs current web information.

A hosted search tool can let the provider perform the search and return grounded results within the same response lifecycle.

Advantages can include:

• simpler integration;
• fewer client-side components;
• provider-native source annotations;
• tighter integration with the model’s context pipeline.

A client-executed search tool gives you more control over:

• search provider;
• query logging;
• result filtering;
• caching;
• domain restrictions;
• user privacy;
• source normalization;
• multi-provider reuse.

The model may see similar text either way, but the trust and execution boundaries differ.

Local files strongly favor client execution

A local-first desktop or mobile app may want a tool such as:

search_local_files(query)

The client can execute that against a local index without uploading the entire library to a provider-managed file service.

Only the selected result content needs to enter model context.

That can be a major privacy and architecture advantage.

It also means the client now owns indexing, authorization, deletion, and result quality.

Hosted code execution is a different security boundary

Code execution illustrates the opposite tradeoff.

Running arbitrary generated code directly on a user’s machine is dangerous unless the environment is strongly sandboxed.

A hosted code environment can isolate execution away from local files and credentials.

That can be safer for some workloads, provided the user understands that inputs and generated artifacts are processed in the hosted environment.

A local client can also sandbox code, but implementing a trustworthy sandbox is substantially harder than exposing a narrow domain-specific tool.

If your app only needs to calculate a checksum, parse JSON, or resize an image, a dedicated client tool may be safer and simpler than a generic code interpreter.

State ownership matters

Hosted tools may keep provider-side state that the client does not fully own.

Examples include:

• uploaded file identifiers;
• provider-managed indexes;
• execution containers;
• background tool jobs;
• provider-native search state;
• response continuation identifiers.

Client-executed tools can keep state inside the application:

type ToolExecution = {
  id: string
  conversationId: string
  toolName: string
  argumentsDigest: string
  status: "pending" | "running" | "completed" | "failed"
  resultRef?: string
}

This is easier to include in local backups and restore logic.

But it also becomes your responsibility to version and migrate that state.

Portability is one of the largest differences

Suppose you switch a conversation from Provider A to Provider B.

If the previous turn used a provider-hosted search artifact or provider-specific file reference, Provider B may not understand it.

You may need to convert the useful part into portable conversation content:

provider-specific tool artifact
        ↓
client extracts durable result
        ↓
portable text / structured result
        ↓
new provider context

Client-executed tools are easier to make provider-neutral because the application controls the canonical representation.

For example:

type ToolResult = {
  callId: string
  toolName: string
  status: "success" | "error"
  content: ToolContent[]
  metadata?: Record<string, unknown>
}

Each provider adapter can encode that structure into its native format.

This connects directly to How to Build a Provider-Neutral AI Message Model.

Hosted tools can reduce round-trip latency

A client-executed tool loop often looks like:

client -> provider
provider -> client with tool call
client -> tool
client -> provider with result
provider -> client with answer

A provider-hosted tool may be able to keep more of that work inside one provider-side response lifecycle.

That can reduce network round trips between the client and provider.

But latency still depends on:

• the tool itself;
• provider scheduling;
• network location;
• streaming behavior;
• whether the tool launches a background operation;
• whether multiple tool rounds are needed.

Do not assume “hosted” always means faster.

Measure the actual workflow.

Client tools can work on a private network

A mobile app may connect to:

• a local database;
• a private home server;
• a LAN-only AI service;
• a corporate internal endpoint;
• an on-device index.

A hosted provider usually cannot directly reach those resources, and allowing it to do so would create a new network exposure anyway.

A client-executed tool can reach resources already accessible to the device.

That is useful, but it also means your client becomes responsible for preventing unsafe network access.

Read How to Secure a Local OpenAI-Compatible Endpoint for the network-side principles.

Credentials should stay with the executor

A clean architecture keeps each credential with the component that actually needs it.

For a client tool:

model sees account reference
client resolves secret
client calls service

For a hosted tool:

provider/service manages linked authorization
model/provider invokes hosted capability

Avoid moving secrets through model-visible text merely to bridge architectures.

For example, do not take a local GitHub token and put it in tool arguments so a hosted browser can use it.

Authorization is easier to centralize with client execution

A local tool executor can enforce one consistent policy layer:

type ToolPolicy =
  | { mode: "disabled" }
  | { mode: "ask" }
  | { mode: "allow" }

The same policy can apply regardless of which model provider proposed the call.

That is a major benefit for multi-provider clients.

If hosted tools have their own independent permission systems, your app may need to map multiple authorization models into one understandable UI.

Hosted tools still need application-level policy

It is a mistake to assume:

“The provider operates the tool, so I do not need to think about authorization.”

If enabling a hosted tool can expose files, perform external actions, or spend significant resources, your application still needs to decide:

• whether the capability should be enabled;
• what data it may receive;
• whether the user understands the boundary;
• whether the operation should require confirmation.

The enforcement mechanism may differ, but product policy still matters.

Tool schemas differ too

Client-executed tools usually expose a schema you define.

Hosted tools may use provider-defined configuration objects rather than ordinary function schemas.

That means a provider-neutral capability layer should distinguish:

type Capability =
  | { kind: "clientTool"; definition: ToolDefinition }
  | { kind: "hostedSearch"; providerConfig: unknown }
  | { kind: "hostedFiles"; providerConfig: unknown }

Do not pretend every capability is the same just because the UI labels all of them “tools.”

Error handling differs

For a client-executed tool, your app can return structured errors:

{
  "status": "error",
  "code": "permission_denied",
  "message": "User denied calendar write access"
}

The model can then decide how to proceed.

For hosted tools, failure behavior may arrive as:

• provider stream events;
• partial response metadata;
• terminal response errors;
• provider-specific tool status objects.

Your adapter should normalize enough information for UI and analytics without erasing provider-specific detail needed for recovery.

Cancellation differs

Client tool cancellation can often be tied to your own task handle:

runningTools[callId]?.cancel()

You can decide whether cancellation means:

• abort local computation;
• close a network request;
• leave a durable background job running;
• mark the model turn interrupted.

Hosted tools depend on whatever cancellation semantics the provider exposes.

This matters especially for long-running operations.

See How Long-Running AI Tasks Work for durable background-job patterns.

Observability can be richer for client tools

A client owns exact timestamps for:

• tool request received;
• approval requested;
• approval granted;
• execution started;
• first result byte;
• execution completed;
• tool result sent back to model.

That enables useful diagnostics:

model decision: 620 ms
tool approval: 3.4 s
tool execution: 180 ms
second model turn: 1.2 s

Hosted tools may expose only part of this timeline.

Use provider events when available, but distinguish measured client timing from inferred provider timing.

Reliability responsibilities move with execution

For client tools, you own:

• retries;
• idempotency;
• duplicate suppression;
• persistence;
• timeout policy;
• concurrency;
• crash recovery.

For hosted tools, the provider may own some of those concerns, but your application still has to handle uncertain outcomes from the outer request.

For example:

provider request disconnects

does not necessarily tell you whether a hosted side effect occurred.

Unknown outcomes still need careful recovery.

Side effects deserve a stronger preference for client control

For read-only tasks, provider-hosted execution can be a convenient abstraction.

For high-impact writes, application-controlled execution is often easier to reason about because you can bind the action to explicit local policy and approval.

Examples:

• send email;
• publish content;
• delete records;
• create payment;
• modify repository;
• change account settings.

A client can show the exact proposed action before execution.

That does not mean hosted writes are inherently unsafe. It means the product needs equally clear authorization semantics even when execution is remote.

Hybrid systems are normal

Most serious AI clients should expect both architectures.

For example:

hosted web search
client-side local file search
client-side calendar write
hosted code execution
remote MCP tool

The useful abstraction is not “all tools are interchangeable.”

It is:

type ToolExecutionLocation =
  | "provider"
  | "client"
  | "remote-server"

Then let policy, UI, persistence, and analytics account for the location explicitly.

Remote MCP tools are a third category

MCP introduces another execution placement:

model -> client -> MCP server -> external system

The client still mediates the call, but the actual operation may occur on a remote MCP server.

That combines characteristics of both models:

• client-visible tool discovery and approval;
• remote execution;
• server-controlled credentials or OAuth;
• provider-neutral tool semantics.

This is why execution location should be modeled separately from tool protocol.

A practical capability model

A multi-provider client might represent capabilities like this:

type ToolCapability = {
  id: string
  displayName: string
  execution: "client" | "provider" | "remote"
  sideEffect: "none" | "reversible" | "external" | "destructive"
  permissionMode: "disabled" | "ask" | "allow"
  portable: boolean
}

That gives the application enough information to make better decisions than a single supportsTools: true flag.

When hosted tools are a good fit

Hosted tools are attractive when:

• the provider offers a high-quality capability you do not want to reimplement;
• provider-native grounding or source metadata matters;
• the workload belongs naturally in provider infrastructure;
• you want to minimize client orchestration;
• the data is already being processed by that provider;
• the tool does not need local/private-network access;
• the provider’s permission and retention behavior matches your product requirements.

When client-executed tools are a good fit

Client tools are attractive when:

• the capability should work across providers;
• local data must remain local until selected;
• the operation requires custom authorization;
• you need private-network access;
• your app already owns the relevant credential;
• you need exact retry/idempotency control;
• you need detailed local observability;
• the tool has high-impact side effects;
• you need durable execution state independent of the model provider.

Avoid capability laundering

One architectural mistake is to expose a client tool that simply forwards arbitrary model-selected requests to a provider-hosted or external execution system.

For example:

http_request(url, method, headers, body)

can become an accidental bridge around all your carefully designed provider and tool boundaries.

If the real capability is “search documentation,” expose that capability directly instead of laundering it through a generic network primitive.

Test both execution paths separately

A robust test suite should not stop at “model requested the tool.”

For hosted tools, test:

• capability encoding;
• provider events;
• provider error mapping;
• cancellation;
• unsupported-model behavior;
• state cleanup.

For client tools, test:

• schema validation;
• authorization;
• approval binding;
• retries;
• idempotency;
• timeout behavior;
• application restart;
• duplicate calls.

For hybrid workflows, test transitions between them.

A decision checklist

Before choosing execution location, ask:

• Who should hold the credential?
• Does the tool need local data?
• Does it need private-network access?
• Is the result portable across providers?
• Does it have side effects?
• Who enforces authorization?
• Who owns retry and idempotency?
• What state must survive app restart?
• What telemetry do we need?
• What happens if the model provider changes?
• What is the privacy boundary?

If those questions are unanswered, “hosted vs client” is not merely an implementation detail yet.

Where BYOKchat fits

A multi-provider BYOK client benefits from provider-neutral client tools because the same permission and execution model can work across providers. It can also support provider-native capabilities where those capabilities are valuable.

The important design principle is to keep the boundary explicit.

A tool’s UI label tells you what it does. Its execution location tells you who you are trusting to do it.