OpenAI-Compatible Does Not Mean OpenAI-Identical

An API can call itself OpenAI-compatible and still differ from OpenAI in ways that matter to a real application.

Usually the phrase means some subset of the following works:

POST /v1/chat/completions
Authorization: Bearer ...
model: "..."
messages: [...]
stream: true | false

That is useful. It lets existing SDKs and clients connect with minimal changes.

But compatibility is rarely a promise that every endpoint, request field, response field, streaming event, tool-calling rule, model capability, error code, or edge case is identical.

For client developers, the safe interpretation is:

OpenAI-compatible is a transport and schema starting point, not a complete behavioral contract.

Compatibility has layers

It helps to separate five different kinds of compatibility.

Layer	Question
Endpoint compatibility	Are familiar paths such as `/v1/chat/completions` available?
Request compatibility	Do familiar JSON fields have the same names and meanings?
Response compatibility	Are response objects shaped the same way?
Streaming compatibility	Are streamed chunks/events framed and ordered the same way?
Semantic compatibility	Do tools, reasoning, limits, finish states, and errors behave the same way?

An endpoint may be highly compatible at the first two layers and very different at the last three.

The happy path hides most differences

A minimal request is easy to imitate:

{
  "model": "some-model",
  "messages": [
    { "role": "user", "content": "Hello" }
  ]
}

If the server returns:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Hi"
      }
    }
  ]
}

many clients will appear to work.

The differences become visible when you add:

tool calling;
reasoning controls;
multimodal input;
structured output;
log probabilities;
usage accounting;
prompt caching;
provider-specific headers;
model discovery;
cancellation;
streaming tool arguments;
hosted tools;
response continuation;
background execution.

That is where an adapter architecture earns its keep.

Request fields can be accepted but ignored

One of the most dangerous compatibility failures is silent acceptance.

A server may accept a field such as:

{
  "temperature": 0.2,
  "reasoning_effort": "high"
}

without rejecting it, even if the selected model ignores one or both controls.

This is worse than a clear error because the application may assume the requested behavior occurred.

Clients should distinguish:

field accepted
field supported by endpoint
field supported by selected model
field actually meaningful in this mode

Those are not the same thing.

Model IDs are not portable contracts

Two OpenAI-compatible servers can expose completely different model namespaces:

openai/gpt-...
meta-llama/...
qwen/...
local-model
my-fine-tune

Even when the same underlying model family appears on two services, the surrounding runtime may differ:

context limits;
quantization;
tool support;
image support;
reasoning output;
default sampling;
stop behavior;
maximum output;
prompt template;
token accounting.

Do not infer capabilities from a model-name substring alone.

`/v1/models` often tells you less than you need

A models endpoint commonly returns IDs and basic metadata. That does not automatically answer:

Does this model support tools?
Does it support image input?
Does it support structured output?
Can it stream reasoning?
What context window is active on this deployment?

Some providers expose richer capability metadata. Others expose only IDs. Local servers may expose just the models currently loaded.

Treat model discovery and capability discovery as related but separate problems.

See How AI Model Discovery APIs Work.

Chat Completions is not the whole OpenAI API

A server that implements /v1/chat/completions is not necessarily claiming support for:

/v1/responses
/v1/embeddings
/v1/audio
/v1/images
/v1/files
/v1/batches

Even a server that implements several of these may implement only a subset of fields.

Therefore this assumption is unsafe:

chat completions works
→ every OpenAI endpoint works

Capability checks should be endpoint-specific.

Streaming exposes implementation differences quickly

Chat-completion streaming often resembles Server-Sent Events with JSON chunks, but clients still encounter differences in:

whether a terminal sentinel is sent;
whether usage arrives in a final chunk;
whether the first delta contains a role;
how empty deltas are represented;
how tool-call IDs are streamed;
how reasoning fields are streamed;
whether errors arrive as HTTP errors or in-stream objects;
whether the connection closes immediately after the final event.

A robust parser should normalize semantic events rather than make the UI depend directly on one provider’s chunk object.

See How AI Streaming Works and How to Parse Server-Sent Events Correctly.

Tool calling is a major compatibility fault line

The familiar shape is something like:

{
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "parameters": {
          "type": "object",
          "properties": {
            "city": { "type": "string" }
          },
          "required": ["city"]
        }
      }
    }
  ]
}

But compatible providers can differ on:

supported JSON Schema keywords;
whether parallel calls are supported;
whether tool choice controls are honored;
how call IDs are generated;
whether arguments stream incrementally;
whether tool results require exact prior call ordering;
whether reasoning state must be replayed with tool results.

The application should validate tool arguments regardless of provider claims.

Reasoning is even less portable

There is no single universal reasoning representation across OpenAI-compatible APIs.

A compatible service may expose:

reasoning_content
reasoning
reasoning_effort
thinking
provider-specific nested fields

Another service may expose no reasoning data at all.

The same setting name can also have a different accepted enum or effect.

Do not put reasoning implementation details into the canonical chat model. Keep them behind provider/model capability metadata and adapter-owned continuation state.

Structured output can differ by schema subset

Two endpoints may both advertise JSON or structured output while supporting different contracts.

Questions to verify include:

Is plain JSON mode supported?
Is JSON Schema supported?
Is strict adherence supported?
Which JSON Schema keywords are accepted?
Does streaming work with structured output?
Which models support it?

A successful HTTP response does not prove the output satisfies your application’s schema. Validate locally.

See Structured AI Output Explained.

Error responses are not uniform

OpenAI-style errors often resemble:

{
  "error": {
    "message": "...",
    "type": "...",
    "code": "..."
  }
}

Compatible services may differ in:

nesting;
error code strings;
HTTP status choices;
retry headers;
quota metadata;
request IDs;
whether upstream-provider errors are wrapped.

Do not switch your entire UX on one provider-specific error string.

Normalize into application-level categories such as:

authentication
permission
rate_limit
invalid_request
unsupported_capability
provider_unavailable
timeout
network
unknown

Base URLs need careful normalization

Users often paste one of these:

https://example.com
https://example.com/v1
https://example.com/v1/
https://example.com/api/openai/v1

If the client blindly appends /v1/chat/completions, it can create:

/v1/v1/chat/completions

or target the wrong path entirely.

A custom-provider UI should define exactly what base URL means and show the final endpoint before saving.

See How to Design a Custom OpenAI-Compatible Provider Connection.

Header behavior can differ

Most services accept:

Authorization: Bearer <token>
Content-Type: application/json

But some compatible endpoints need additional headers for:

account/project selection;
API versioning;
routing preferences;
application attribution;
custom gateway authentication.

Store protected headers as credentials, not ordinary display metadata.

Never log them with request diagnostics.

Compatibility shims should be explicit

A weak implementation accumulates scattered conditions:

if (provider === "x") { ... }
if (provider === "y") { ... }
if (model.includes("reasoner")) { ... }

A stronger architecture centralizes differences:

The adapter should own wire-format differences. The capability layer should decide whether a requested feature is valid before sending.

Build a capability matrix, not a compatibility boolean

Avoid:

isOpenAICompatible: true

as the only metadata.

Prefer a shape closer to:

interface ModelCapabilities {
  streaming: boolean;
  tools: boolean;
  parallelTools?: boolean;
  imageInput?: boolean;
  structuredOutput?: "none" | "json" | "jsonSchema";
  reasoning?: "none" | "summary" | "content" | "opaqueState";
  maxContextTokens?: number;
  maxOutputTokens?: number;
}

Not every value needs to be known. unknown is often more honest than false.

Probe carefully

Capability probing can help for custom endpoints, but probes have costs:

they consume rate limits;
they may cost money;
unsupported requests may be noisy;
model behavior can change;
some features cannot be proven reliably from one request.

Prefer documented metadata where available. Use probes for narrow questions and cache the result with an expiry.

Test against multiple implementations

A client that only tests against OpenAI has not really tested OpenAI compatibility.

Use fixtures or simulators for variations such as:

missing usage
unknown finish reason
empty first delta
streamed tool arguments split mid-UTF-8
429 with Retry-After
429 without Retry-After
unknown response fields
unsupported parameter accepted silently
unsupported parameter rejected
model list without capabilities

Then run integration tests against representative real endpoints.

Prefer tolerant readers and strict writers

A useful protocol principle is:

strict when sending: send only fields the selected capability profile supports;
tolerant when reading: ignore unknown additive fields and preserve useful metadata where appropriate.

Do not crash because a provider added one response property.

Do not send every field merely because one SDK type exposes it.

The practical definition of compatibility

For a production client, compatibility means more than “the SDK did not throw.”

A connection is usable only when the features the user selected actually work:

endpoint reachable
+ authentication valid
+ selected model exists
+ requested capabilities supported
+ stream parseable
+ finish state understandable
+ errors recoverable
= usable connection

That is a stronger and more useful definition than a branding label.

Where BYOKchat fits

A multi-provider BYOK client should treat OpenAI-compatible servers as first-class connections without pretending they are all the same provider. The client can share a large amount of transport and schema logic while still keeping capability detection, custom headers, local-network policy, reasoning, tools, and provider-specific fields behind explicit boundaries.

That gives users the convenience of compatibility without forcing the application to lie about what a server can do.