The three first-class provider classes, which one to reach for, how OpenAI-compatible backends fit, and the per-model output-token caps that dendrux handles for you.

Models and providers

A provider is the thin adapter between dendrux's universal types and a vendor's HTTP API. You construct one, hand it to Agent(provider=...), and the loop calls it. Providers do shape translation only — no agent-loop awareness, no business logic.

Four first-class provider classes ship in the box. Everything below is read from the dendrux source.

from dendrux import Agent
from dendrux.llm.anthropic import AnthropicProvider
from dendrux.llm.openai import OpenAIProvider
from dendrux.llm.openai_responses import OpenAIResponsesProvider
 
agent = Agent(prompt="…", provider=AnthropicProvider(model="claude-opus-4-8"), strict_tenancy=False)  # single-tenant example; see Multi-tenancy

Picking a provider

There are two OpenAI providers because OpenAI runs two different request/response APIs, and they differ in what they can surface.

• OpenAI, current models → OpenAIResponsesProvider. The Responses API is OpenAI's current direction and the only one that returns reasoning summaries (the model's thinking, as text) for gpt-5 / o-series. See Reasoning and thinking.
• Legacy OpenAI models, or any OpenAI-compatible server → OpenAIProvider. Chat Completions is the universal endpoint. It returns reasoning token counts but no reasoning text.
• Anthropic → AnthropicProvider.
• Open-source or aggregated premium models via OpenRouter → OpenRouterProvider. A preset over OpenAIProvider that adds a native-tools capability guard and a queryable model catalog. See the OpenRouter recipe.

The Responses API is OpenAI-only. Pointing OpenAIResponsesProvider at a non-OpenAI backend will not work — that is exactly the case OpenAIProvider covers.

Responses: phase replay and cache keys

• Assistant phase is replayed. Every assistant message dendrux records carries a phase ("commentary" for text alongside tool calls, "final_answer" otherwise; see Message.meta["phase"]). When history goes back to the Responses API, each assistant input item carries that phase, as OpenAI asks for manual history replay (from gpt-5.3-codex on, dropping it can degrade performance). User items never get a phase.
• prompt_cache_key is per customer, not per run. The provider sends a key only when it has a prefix: the constructor cache_key_prefix= (OpenAI recommends one per user or customer), otherwise the runner's {agent_name}:{model}. It never falls back to the run id. On GPT-5.6 and later the key is unnecessary. See Prompt caching.

OpenAI-compatible backends

OpenAIProvider speaks plain Chat Completions, so it drives any server that implements that wire format — vLLM, SGLang, Groq, Together, Ollama, LM Studio — by setting base_url. There is no separate provider class per backend.

# vLLM / SGLang / local
OpenAIProvider(model="meta-llama/Llama-3-70B", base_url="http://localhost:8000/v1", api_key="not-needed")
 
# Groq
OpenAIProvider(model="llama-3.3-70b", base_url="https://api.groq.com/openai/v1", api_key="gsk-…")

When base_url points away from the official endpoint, dendrux drops OpenAI-specific request fields (the prompt-cache key, the OpenAI-only output-token parameter) that some compatible backends reject. You get the universal request shape automatically.

Provider errors and deadlines

Since 0.5.0 every failed request raises a ProviderError subclass (ProviderAuthError, ProviderRateLimitError, ProviderTimeoutError, ProviderUnavailableError, ProviderStreamIncompleteError, ProviderResponseError, ModelRefusalError, ContentFilteredError, ContextLengthExceededError, or the base class for credits, rejected and unknown). All four providers map their SDK and HTTP errors the same way and chain the SDK exception on __cause__. A stream that ends without its terminal event, a Responses error or response.failed event, a refusal and a content filter all raise instead of producing an empty answer. See Handling provider errors for the taxonomy and what to do with each.

Every provider accepts deadlines and local stream bounds:

max_retries keeps its meaning: the SDK still retries connection resets, 408, 409, 429 and 5xx responses. But when call_timeout fires the call is abandoned and not retried again, raising ProviderTimeoutError, so a long generation is paid for at most once. A stream retries once inside the provider only if nothing has been yielded yet. Pass None to turn a deadline off. With call_timeout=None nothing bounds the SDK's own retries either, so a slow or failing request can be retried after it already produced billed output, and a paid call can run more than once; the network read timeout is still set a little above the longest deadline so a dead connection cannot hang for hours. The old timeout= argument is a deprecated shorthand that sets both call_timeout and stream_idle_timeout and prints a DeprecationWarning.

Exceeding a local stream bound closes upstream and raises ProviderResponseError(category="rejected"), with code stream_size_limit or stream_tool_call_limit. A completed response with neither text nor tool calls raises the same class with code empty_completion; token-limit truncation and tool-only output keep their existing behavior.

The duration and any timeout reason are recorded with each LLM call.

Provider clients are rebuilt per event loop, so using one provider from a second loop (a test client, a second asyncio.run) no longer fails on a connection from the first.

Request errors on OpenAI-compatible endpoints

Compatible servers report problems in slightly different ways. OpenAIProvider (and therefore OpenRouterProvider) classifies API status errors and in-stream error events for batch calls, stream creation, and stream iteration. There are three outcomes for the classified errors, checked in this order:

1. Context window exceeded raises ContextLengthExceededError.
2. Streaming not supported (streaming calls only) raises NotImplementedError, the same as before.
3. Any other status error raises the matching ProviderError subclass (auth, rate limit, unavailable and so on), with the SDK error on __cause__.

from dendrux import ContextLengthExceededError
 
try:
    result = await agent.run(user_input, history=history)
except ContextLengthExceededError as exc:
    # exc.status_code, exc.provider_code, exc.model; the SDK error is exc.__cause__
    history = history[-6:]          # trim and try again, or tell the user

ContextLengthExceededError is a ProviderError (and so a RuntimeError), importable from dendrux and dendrux.errors. It carries status_code (None for an error event inside a stream), provider_code (for example "context_length_exceeded") and model. The message names the model and never contains prompt content. As with any provider exception, the run row ends in error before the exception reaches you.

What counts as a context overflow:

• The provider code context_length_exceeded when the status is absent (an in-stream event) or is 400, 403, 413, or 422. A response with any other status, such as 500, is returned as the original SDK status error.
• Otherwise a 400, 403, 413 or 422 whose text matches the wording servers use for an oversized request ("maximum context length", "input is too long", and similar, matched case-insensitively). On a 403 only the one documented formula is accepted, because an auth error can describe a model's context length without being one.
• Errors with a rate-limit, quota, permission or authentication code are never overflows, even if their text mentions tokens. A complaint about the max_tokens setting alone is a request-configuration error, not an overflow, unless the text also counts the prompt's input tokens.

Two other gateway fixes ride along. A streaming request that fails with an ordinary 400 (a bad parameter, say) used to be reported as "this endpoint does not support streaming"; now only a 400 that says streaming is unsupported (or whose param is stream) is mapped that way, and every other error surfaces as itself. And a backend that ends a stream without ever sending a finish_reason no longer loses its buffered tool calls: they are flushed when the stream ends.

Stop reasons and custom providers

Every built-in provider sets LLMResponse.stop_reason (a normalized StopReason) and raw_stop_reason (the vendor string) on batch and streaming calls. The final answer's value lands on RunResult.stop_reason; "max_tokens" marks a run as truncated. See Truncated answers. A custom provider should set both fields when its backend reports why generation stopped, and leave both None otherwise.

Anthropic strict mode

AnthropicProvider sends @tool(strict=True) tools, and structured output whose schema fits, with "strict": true. Pass AnthropicProvider(..., strict=False) to send tools and structured output exactly as before 0.4.1. Details and limits are in Strict mode on Anthropic.

The output-token cap is per-API

Every provider takes a constructor default for the maximum output tokens per call and lets you override it per call via kwargs. The wire parameter name differs by API, and dendrux picks the right one for you:

The OpenAI Chat Completions case is the subtle one. gpt-5 and o-series models reject max_tokens on Chat Completions with an HTTP 400 (Unsupported parameter: 'max_tokens' … Use 'max_completion_tokens' instead); max_tokens is deprecated for gpt-4o-era models too. So on the official endpoint OpenAIProvider always sends max_completion_tokens, which works for gpt-4o, gpt-5, and o-series alike. Compatible backends that still expect the older max_tokens keep getting it. The selection is gated on base_url, the same way prompt-cache routing is — you do not configure it.

You keep using the friendly constructor name regardless of endpoint:

# Constructor default applies to every call …
provider = OpenAIProvider(model="gpt-5.5", max_tokens=4_000)
 
# … and a per-call kwarg overrides it for one call.
await agent.run("…", max_tokens=512)

Per-call overrides

Anything you pass to agent.run(...) / agent.stream(...) is forwarded to the provider for that call and overrides the constructor default. Common knobs: model, max_tokens, temperature, and the reasoning controls (thinking, effort) covered in Reasoning and thinking. Unknown kwargs are ignored rather than forwarded blindly, so a kwarg meant for one provider will not break another.