What is a unified LLM API? One interface for every model
A unified LLM API gives applications one interface for working with models from multiple providers. Teams can keep the same authentication flow and request structure when testing or switching models, reducing the provider-specific integration code they need to maintain. The interface can normalize common behavior such as streaming, tool calls, structured outputs, embeddings, retries, and usage data, but provider-specific parameters and model capabilities still require validation before a production change.
Braintrust Gateway gives teams one endpoint for OpenAI, Anthropic, Google, AWS, and custom providers, then connects those model calls to the evaluation workflow used to assess release quality. Gateway requests can be logged as traces, production failures can be added to datasets, and candidate models can be compared through experiments before approving a model change for production. Start free with Braintrust Gateway.
What is a unified LLM API?
A unified LLM API sits between the application and the model provider. Each request passes through the same sequence. The API identifies the requested model, authenticates the caller, translates the request, sends it to the appropriate provider, normalizes the response, and records the resulting usage data.
The request body specifies the model, and the application keeps the same client structure for every model. The unified API resolves differences in endpoint URLs, headers, message schemas, parameter names, and response formats before sending each request to the selected provider.
Many unified LLM APIs use the OpenAI request format as a common interface because many applications and SDK integrations already support it. Anthropic, Google, and hosted inference providers also offer OpenAI-compatible endpoints, which gives gateways a broadly supported schema for translating requests across providers.
What a unified LLM API replaces
A unified LLM API removes several integration layers that otherwise have to be maintained separately for each model provider:
Separate SDKs: Teams using GPT, Claude, and Gemini directly have to maintain multiple clients, each with its own release cycle and type definitions. A unified interface lets the application use one client structure across supported providers.
Separate credentials: Direct integrations require managing provider keys across services and environments. A unified layer centralizes those credentials and gives the application a single authentication path, reducing the credential changes required when switching providers.
Divergent request formats: Providers differ in message roles, content blocks, tool schemas, parameter names, and stop-sequence rules. Without a unified layer, application code must account for those differences wherever it calls each provider.
Provider-specific streaming: Streaming responses use different chunk formats, delta structures, and terminal events. A unified layer converts them into a single stream format at the chunk level.
Separate error and retry handling: Providers expose different status codes, error bodies, rate-limit headers, and retry behavior. A unified API can map those to a shared set of response types, which lets one retry policy cover every provider.
Split usage and billing views: Direct provider integrations distribute usage data across separate accounts and dashboards. A unified layer can consolidate model usage and cost data, although the underlying provider billing arrangement depends on how the service handles provider credentials.
Unified LLM API vs direct provider integration
Direct provider integration makes sense when an application depends primarily on one model provider and relies heavily on provider-native capabilities. Features such as Anthropic prompt caching, OpenAI Responses API state, or Gemini-specific multimodal inputs are generally easiest to use through the provider's native SDK, where the full parameter set and response format remain available.
A unified LLM API becomes more useful when an application runs multiple models, evaluates replacements for models already in production, or needs centralized control over credentials, usage, and request logs. The application can keep a consistent integration surface across providers, but teams still need to verify any provider-specific behavior that the shared interface cannot represent cleanly.
How a unified LLM API request works
One client, one base URL, and one credential reach every provider because the unified layer handles authentication, translation, routing, normalization, and logging.
Each request follows a predictable sequence through the unified layer, and every step below maps to one stage of that path.
Client request and model identifier
The application sends a request to the unified endpoint through a supported SDK, and the model identifier determines which provider should receive it. The unified API uses that identifier to select the required credentials and translation rules. Changing the model becomes a configuration change, with no edits to provider-specific application logic.
Authentication and credential storage
The application authenticates with the unified API, while the gateway stores provider credentials. Braintrust supports provider credentials at both the organization and project level, which gives individual projects separate control over provider accounts, billing, and rate limits without requiring a different gateway endpoint.
Request translation to the provider format
Before forwarding the request, the unified API converts the shared schema into the format the selected provider expects. Translation can include mapping message roles, converting tool schemas, renaming parameters, and restructuring content blocks. Compatibility is not always exact. Anthropic's OpenAI compatibility layer, for example, concatenates system and developer messages into the single initial system message expected by the Claude API.
Provider execution, routing, and failover
After translation, the request is sent to the selected provider endpoint and the gateway applies its routing and retry rules. Braintrust Gateway can retry eligible failures against configured fallback providers when the original endpoint is unavailable, rate-limited with a 429, or returns a 5xx server error. Response headers such as x-bt-used-endpoint, x-bt-failover-from, and x-bt-failover-to identify which endpoint ultimately handled the request.
Response normalization and streaming
The provider response is translated back into the format the client expects, including content blocks, tool calls, and finish reasons. Streaming chunks are normalized in the same way, using the single stream format described above. Provider-specific fields with no equivalent in the shared response format may still be omitted or returned empty, and normalization should not be treated as complete feature parity.
Logging, usage metadata, and cost attribution
The unified API can record the model and provider used for each request, along with token usage, latency, cost, and cache status. Braintrust Gateway can also connect a request to a distributed trace through x-bt-parent and return the logged span ID in x-bt-span-id. An engineer can later attach scores or user feedback to that same model call.
What a unified LLM API normalizes
A unified LLM API standardizes the parts of model communication that map reasonably well across providers. The goal is to give application code a consistent interface without implying that every provider supports the same capabilities or implements them the same way.
Text generation: Chat and completion requests generally translate cleanly across providers, making text generation one of the most portable parts of a unified interface.
Streaming: Provider-specific events and response chunks are converted into a consistent streaming format. One parser handles every supported model.
Tool calling: A unified schema can translate tool definitions, tool calls, and tool results into the format each provider expects. Differences can still appear in schema enforcement, tool selection, and parallel execution.
Structured outputs: JSON schema requests can be mapped to the structured-output mechanism supported by the selected provider, although the level of schema enforcement can differ between models.
Embeddings: Unified APIs commonly expose an OpenAI-compatible embeddings route and translate requests for supported providers. Braintrust can also proxy embedding requests to the selected model while converting the request format as needed.
Usage metadata: Token usage, cached tokens, latency, and cost can be returned in a common schema, giving teams a consistent basis for comparing model usage across providers.
Errors and retries: Provider errors are mapped to a consistent set of response types and status codes. Shared retry logic then applies across models without per-provider branches.
Where unified LLM API compatibility breaks
Compatibility breaks when the shared schema cannot represent a provider's native behavior exactly. A request can still succeed, even when translation changes how the provider interprets part of the request or what it returns to the application.
Provider-specific parameters
Sampling and reasoning controls are often handled differently across compatibility layers. Anthropic's OpenAI compatibility layer ignores fields such as presence_penalty, frequency_penalty, logit_bias, seed, logprobs, top_logprobs, response_format, service_tier, store, user, metadata, and reasoning_effort. It also requires n to equal 1 and caps temperature values above 1.
Unsupported parameters are not handled consistently across unified layers. LiteLLM raises an exception by default when a parameter is unsupported for the target model, while setting drop_params=True drops unsupported fields. Before switching, confirm that the selected provider accepts every parameter the application sends.
Modality and input format differences
Image, audio, video, and document support varies across providers and models. Anthropic's compatibility layer accepts image_url content but ignores the detail field, while input_audio and file blocks are discarded. A request can complete successfully even when part of the original input never reaches the model.
Context windows and token accounting
Context limits and tokenization methods differ by model, so the same prompt can produce different token counts and truncation points across providers. A retrieval pipeline sized for one model can exceed another model's context window, and cost estimates based on one tokenizer may no longer hold after switching models.
Tool calling semantics
A shared tool schema does not guarantee identical execution behavior. Anthropic's compatibility layer ignores the strict field on function definitions, so tool arguments may not match the supplied schema even when the field is accepted. Support for server-side tools such as web search and code execution can also differ, as can tool-choice enforcement and parallel tool calls.
Response detail and metadata
Normalized responses can omit provider-specific information when the shared schema has no equivalent field. Through Anthropic's compatibility layer, logprobs, system_fingerprint, refusal content, and token detail objects return empty. Thinking can be enabled without the reasoning output being returned, and citations and safety annotations may be lost during normalization.
Rate limits, quotas, and regional availability
Provider limits still apply after requests pass through a unified endpoint. Account-level quotas, tokens-per-minute limits, and regional model availability remain tied to the underlying provider, and Claude requests through the OpenAI compatibility layer continue to follow Anthropic's standard limits for /v1/messages.
Provider-specific escape hatches
A unified interface will not cover every provider capability, and applications need a controlled way to access features outside the shared schema. Most implementations use one of four patterns, depending on whether the exception belongs in the SDK, request body, headers, or provider configuration.
Native SDKs against the unified endpoint
Some gateways accept each provider's native request format alongside a shared format. This lets teams keep one SDK for standard calls and use another when a provider-specific capability requires its native request structure. Braintrust Gateway supports using any supported provider SDK to call models from any supported provider.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://gateway.braintrust.dev",
api_key=os.environ["BRAINTRUST_API_KEY"],
)
response = client.responses.create(
model="gpt-5-mini",
input=[{"role": "user", "content": "Say hello!"}],
)
print(response.output_text)
Passthrough parameters
Parameters that do not exist in the shared schema can sometimes be included directly in the request body and forwarded to the provider without translation. Anthropic, for example, documents enabling thinking through the OpenAI SDK with extra_body against its own OpenAI-compatible endpoint.
response = client.chat.completions.create(
model="claude-sonnet-4-6",
messages=[{"role": "user", "content": "Who are you?"}],
extra_body={
"thinking": {
"type": "enabled",
"budget_tokens": 2000,
}
},
)
Request headers
Gateway-specific behavior that does not belong in a provider request schema can be controlled through headers. Braintrust Gateway uses headers such as x-bt-use-cache, x-bt-cache-ttl, x-bt-fallback-providers, x-bt-project-id, and x-bt-parent to control caching, fallback behavior, project scoping, and tracing without changing the provider request itself.
Custom providers
Self-hosted models, fine-tuned models, and proprietary endpoints can be registered as custom providers and called through the same interface as commercial models. The platform owner assigns the endpoint a model name and configures its request format, authentication headers, and other provider settings, which keeps internal models on the same model-selection path as everything else.
Escape hatches are easier to maintain when provider-specific logic stays isolated from the rest of the application. Keeping passthrough parameters, native requests, and other exceptions in a dedicated adapter makes provider dependencies easier to identify when models change.
Unified LLM API vs AI gateway, model router, proxy, and model marketplace
When teams compare model-access products, the terminology can blur because one platform may provide several of these functions together. The useful distinction is the responsibility each layer takes on between the application and the model provider.
| Layer | Primary responsibility | What it does not cover on its own |
|---|---|---|
| Unified LLM API | Translating requests and normalizing responses so one client structure reaches every supported model | Production controls such as caching, quotas, failover policy, and request logging |
| AI gateway | Applying production controls to model traffic: credential storage, caching, rate limits, failover, audit logs, and cost attribution | Deciding which model should answer a given request |
| Model router | Selecting the model or provider for each request based on rules, cost, latency, availability, or evaluation scores | The schema translation work a unified API performs beneath it |
| LLM proxy | Forwarding requests to a provider endpoint, often with minimal translation, to centralize network egress and keys | Cross-provider normalization, routing policy, and quality measurement |
| Model marketplace | Aggregating catalog access and billing so many models are reachable through one account | Evaluation, release gating, and deployment control over the serving layer |
Most production platforms combine several rows. Braintrust Gateway, for example, provides the unified API and the gateway controls in the same endpoint, then connects both to the evaluation workflow.
Also read:
- Compare AI gateways across caching, routing, failover, and observability
- Best LLM routers and model routing platforms
Benefits of a unified LLM API
Faster model switching: When model selection is handled through configuration, teams can test a candidate model without rebuilding the application around another provider SDK. The integration work stays largely unchanged, so engineering effort can focus on validating how the new model behaves.
Reduced provider lock-in: Provider-specific authentication, request formats, and response handling are concentrated in the unified layer. A provider change still requires compatibility testing, but it does not require the integration work of replacing a direct provider implementation.
Centralized credentials and spend: Provider keys can be managed in one system with shared access controls, while usage data from different models can be recorded in a consistent format. Finance and platform owners get a single view of where model spend comes from, without checking a separate provider dashboard for every request path.
Consistent logs and traces: Requests made through the same interface can be recorded with the same metadata for latency, cost, token usage, and provider selection. That consistency makes it easier to investigate failures and compare model behavior across providers without maintaining separate observability pipelines.
Regional routing and failover: A gateway can send requests through regional endpoints and retry eligible failures against another provider or endpoint without changing the application request. Braintrust Gateway supports a latency-routed global endpoint along with regional endpoints in US East, US West, EU West, and Asia Pacific Southeast.
Lower experimentation costs: Response caching can reuse identical model outputs during development and repeated evaluation runs, reducing unnecessary provider calls. Braintrust Gateway supports request-level cache controls and configurable TTLs, with cached responses encrypted and scoped to the requesting user by default. The LLM gateway caching guide explains where caching is useful and where reused responses need additional quality checks.
Limitations of a unified LLM API
Another dependency in the request path: Every model call passes through an additional service, so gateway availability becomes part of application reliability. Teams with stricter infrastructure or data-control requirements can use a self-hosted Braintrust data plane, which shifts patching, scaling, and uptime of that plane onto the team running it.
Abstraction can hide provider differences: Unsupported parameters, unenforced tool schemas, or missing response fields may change model behavior without producing an obvious request error. Features that affect application behavior still need testing against each target provider before they are treated as portable.
New provider features may take time to reach the shared interface: Model providers can release new reasoning controls, modalities, or API capabilities before a unified layer supports them. Applications that need a provider feature immediately may still require a native request path until support is available.
Latency and debugging become more complex: An additional routing hop can add latency, and failures can originate from either the gateway or the underlying provider. Response headers such as x-bt-used-endpoint, x-bt-error-origin, and x-bt-failover-from identify which endpoint handled the request, which separates gateway failures from provider failures during investigation.
Pricing depends on the gateway's billing model: Some services add a markup to provider token prices or resell model capacity, which can make the final cost different from the provider's published rate. Bring-your-own-key setups preserve the direct provider billing relationship and make those costs easier to reconcile.
Data handling requires its own review: A gateway processes prompts and model responses as they pass between the application and provider. Retention policies, encryption, storage location, and regional deployment options must be evaluated alongside the corresponding provider requirements.
Model switching still changes model behavior
Changing the model identifier removes much of the integration work involved in a provider switch, but it does not preserve application quality. Models that accept the same request can still differ in instruction following, tool-call accuracy, refusal behavior, formatting consistency, and latency. Prompts tuned for one model may perform differently when the underlying model changes.
The production question is whether the candidate model meets the application's existing quality requirements. An engineer can run the same evaluation dataset against the current and candidate models, then compare experiments in Braintrust to identify score changes and inspect the test cases that improved or regressed. Adding production failures to the dataset over time gives future model comparisons stronger coverage of behavior that has already affected real users.
Comparing a base model against candidate models surfaces score changes, cost, and latency differences before a switch reaches production.
Using experiment scores and regression results as release criteria keeps a model change from reaching production solely because the integration succeeds.
Route model calls through Braintrust Gateway
Braintrust Gateway provides teams with a single endpoint for the supported providers, including self-hosted and fine-tuned endpoints. Existing SDKs can point to the Gateway URL, while Braintrust keeps provider credentials and model selection centralized.
Braintrust becomes more useful when model access and model evaluation need to support the same release decision. The traces, datasets, and experiments described in the previous section attach directly to Gateway requests, and online scoring can be applied to live traces after release to track quality alongside cost and latency.
Teams comparing unified model-access options can use the unified LLM API provider comparison to review Braintrust Gateway alongside other providers across model access, caching, observability, evaluation, and release control.
Route model calls and evaluate changes with Braintrust Gateway on the free tier →
FAQs about unified LLM APIs (2026)
What is the difference between a unified LLM API and an LLM gateway?
A unified LLM API standardizes how an application sends requests to different model providers and receives their responses, so the same client structure can work across supported models. An LLM gateway sits around that API layer and manages production concerns such as provider credentials, caching, rate limits, failover, and request logging. Braintrust Gateway and Portkey are examples of gateways that include a unified API and add the production controls on top.
Can I call OpenAI, Claude, and Gemini through a single SDK?
With a compatible gateway, the same SDK can call models from different providers by changing the model identifier and gateway endpoint. Braintrust Gateway supports cross-provider access across its supported AI providers, including OpenAI, Anthropic, and Gemini, and a single client integration covers every standard model call. Provider-native capabilities may still require additional configuration or a native request format.
Which unified LLM APIs support geographic routing across regions?
Braintrust Gateway supports geographic routing through a global endpoint with latency-based DNS routing and health checks, along with region-specific endpoints in US East, US West, EU West, and Asia Pacific Southeast. Teams can use a regional endpoint when they need requests to enter through a specific geography, or the global endpoint when they want routing based on latency and endpoint health.
Does a unified LLM API remove provider lock-in?
A unified API reduces the amount of provider-specific code an application depends on, but it does not make models interchangeable. Provider-specific features, model behavior, account limits, and pricing can still affect a migration, so portability depends on how much application logic relies on capabilities outside the shared schema. Using your own provider credentials and keeping provider-specific code isolated can make future changes easier to manage.
How do I use a provider-specific feature through a unified API?
Use the narrowest escape hatch that preserves the provider capability you need. You can often pass a provider-specific parameter through the request body, configure gateway behavior through headers, and use the provider SDK for features that require a native request structure against a compatible gateway endpoint. Keeping those exceptions in one adapter also makes provider dependencies easier to find when models or APIs change.
Do unified LLM APIs support embeddings and multimodal inputs?
Embeddings are commonly exposed through a shared endpoint because their request and response structures translate relatively well across providers. Multimodal compatibility is less predictable because image, audio, video, and document handling varies by model. Teams should verify each required input type against the target model rather than assuming that acceptance by the unified API means the provider received every part of the request.
Does a unified LLM API change model output quality?
A unified API does not make different models produce equivalent results, and request translation can introduce additional differences when parameters or content don't map exactly. Before replacing a production model, teams can evaluate candidate models against representative test cases and measure whether the new configuration meets the application's existing quality requirements.