Provider System

crates/provider gives the agent one interface for API-backed model backends. The rest of Pioneer talks to Provider, ChatRequest, ChatMessage, ToolDefinition, and ProviderToolCall; provider adapters translate those into OpenAI, Anthropic, Gemini, Ollama, Bedrock, direct cloud, local API, or OpenAI-compatible API calls. The provider layer hides API differences from the rest of the system. The agent loop works with one vocabulary: messages, tools, streaming chunks, usage, reasoning, and normalized tool calls.

​

Why this layer exists

LLM providers disagree about almost everything: streaming shape, tool-call deltas, system prompt placement, attachment support, file upload APIs, model listing, auth, base URLs, and error formats. Pioneer cannot let those differences leak into every runtime layer. The provider crate is the boundary. Above it, Pioneer speaks ChatRequest; below it, each adapter handles its own API. Prompt compilation, tool routing, task execution, and persistence can then remain independent of provider wire formats.

​

Provider trait

Every provider implements: The agent chooses streaming when capabilities().streaming is true and the turn is not forced into non-streaming mode.

​

Provider registry

The gateway creates a ProviderRegistry with a workspace-aware key resolver backed by GatewaySecrets and pioneer-keystore. The registry lazily creates provider instances and caches them by workspace id plus provider name. When a provider key changes, the gateway invalidates cached instances for that provider so the next turn uses fresh credentials. Provider API keys are workspace-scoped. The same gateway can have one OpenAI key in a work workspace and another OpenAI key in a personal workspace. Raw key values still live in the gateway keystore; the workspace id is part of the secret id. Provider construction is centralized in create_provider. That function maps provider aliases to concrete adapters. It also includes a large set of OpenAI-compatible endpoints where the only difference is base URL and auth style. The provider registry is deliberately gateway-side. Clients can list providers, set/delete keys, and request model lists, but they should not instantiate provider adapters or hold raw provider credentials. Provider network routing is workspace-scoped as well. The gateway can store an HTTP or SOCKS proxy for a provider, include that proxy in the provider-registry cache key, and build the provider HTTP client with the selected route. Changing a proxy invalidates the cached provider instance so later model discovery and turns use the new route. Proxy values are validated and stored with gateway secrets rather than being treated as ordinary client state. Model listing returns normalized ProviderModelInfo records. Besides ids, names, limits, pricing, lifecycle state, and modalities, capabilities can include reasoning metadata: whether reasoning is supported, which effort values are available, the default effort, whether effort is mandatory, whether a reasoning token budget is supported, and where the metadata came from. Shared client code turns that metadata into model selector rows.

​

Current adapter families

CLI-backed runtimes such as Codex CLI are excluded from this crate. They are process/session integrations documented in CLI Runtime Architecture.

​

Request shape

The common ChatRequest includes:

• model id
• ordered chat messages
• temperature and max token controls
• optional tool definitions
• optional tool choice
• optional parallel tool call flag
• optional compiled prompt payload

ChatMessage supports text content, reasoning content, tool-call messages, tool-result messages, and structured content parts for files, images, audio, and video.

​

Tool calls

Provider adapters normalize tool calls into ProviderToolCall: id, name, and JSON arguments. The agent does not need to know whether a provider streamed tool-call fragments, returned a final tool-call array, or encoded function calls in a provider-specific shape. Parsing lives under crates/provider/src/tools. Provider-specific code should preserve the common invariant: by the time the agent sees a tool call, the tool name and arguments are ready for the tool router.

​

Attachments

The attachment pipeline normalizes input parts before provider submission. It handles byte sources, paths, URLs, and references. Provider capabilities decide whether a given input can be sent natively, uploaded as a provider file, inlined as a data URL, or converted to text fallback. Gateway configuration controls attachment size limits, total request limits, allowed path roots, URL-source behavior, redirect limits, private-network rules, MIME checks, retry behavior, circuit breaker behavior, and upload registry TTL. Attachment handling is split under crates/provider/src/attachments: Do not bypass this pipeline from a provider adapter. If a new provider has special file-upload semantics, add a transport plan or adapter hook rather than making the agent loop provider-aware.

​

Provider failures

Provider calls can fail before the first chunk, between chunks, during tool-call parsing, or in non-streaming calls. The agent/provider layer classifies these failures so the gateway recovery coordinator can decide whether to schedule a recovery attempt, mark the turn failed, or continue with available state. Context-length failures are detected from provider error messages and treated specially because they usually require context compression or prompt reduction rather than a simple retry. Streaming output is also covered by gateway resilience timeouts. [gateway.resilience.provider_stream_items] configures lease, idle, and hard deadlines for provider stream items that may need reconciliation after interruption or restart. These deadlines are separate from provider transport timeouts such as first_chunk_timeout_secs and inter_chunk_idle_timeout_secs. Error normalization is a central responsibility of this crate. A raw HTTP error tells the user little and gives the gateway little to recover from. A classified provider failure can identify whether the problem occurred before generation, during streaming, while parsing tool calls, or because the context was too large.

​

Adding a provider

Add a provider by implementing Provider, declaring accurate capabilities, mapping ChatRequest into the provider API, normalizing stream chunks and tool calls, and registering the adapter in create_provider. If the provider supports model listing or file uploads, implement those paths in the provider crate rather than leaking provider-specific logic into the agent or gateway. Implementation checklist:

1. Add adapter code under crates/provider/src/providers.
2. Register aliases/base URL behavior in create_provider.
3. Declare capabilities truthfully, especially streaming, tool calls, vision, and input type support.
4. Normalize streaming chunks into StreamChunk.
5. Normalize tool calls into ProviderToolCall with valid JSON arguments.
6. Add model-listing support when the upstream API supports it.
7. Add attachment support through the attachment pipeline when needed.
8. Update gateway/client provider lists, docs, icons, and generated client contracts if the provider becomes user-selectable.

If the provider exposes reasoning controls, populate ProviderModelReasoningCapabilities rather than hard-coding picker behavior in a client shell. Use provider metadata when available; otherwise use static registry or config override data only when the behavior is stable enough for users.

​

Related pages

• Prompt And Context explains the compiled_prompt payload that providers receive.
• Agent Loop explains when providers are called and how streaming chunks become turn events.
• Tools System explains what happens after normalized tool calls leave the provider layer.
• Gateway explains provider key storage and provider cache invalidation.
• CLI Runtime Architecture explains the separate process/session model for Codex-style runtimes.