crates/tools is the runtime layer that turns model tool calls into real actions. The agent loop asks it to build a tool runtime for a turn, then routes each provider tool call through that runtime.
Tools currently execute as the OS user running the gateway. There is not yet a separate sandbox for tool runs, so every tool implementation must be treated as security-sensitive.
The tools layer is where Pioneer crosses from language into side effects. Reading a file, running a command, calling an MCP server, applying a patch, or using computer control is no longer just “model output.” It is gateway-host execution.
Tool code must be written as privileged runtime code. A remote gateway means the tool runs on the remote host. A local gateway means it runs as the local OS user. Until sandboxing lands, tool policy and output policy are the main safety boundaries.
Why This Layer Exists
Tools need a common runtime because Pioneer has several capability sources: built-ins, MCP servers, skills, and task orchestration. The model should not need to know which subsystem owns a tool. It should see a clean tool name, JSON schema, and result.
At the same time, Pioneer needs consistent behavior after execution: timeline events, model-visible output, storage output, recovery evidence, retries, and diagnostics. If every subsystem returned raw text directly to the model, context windows would explode and recovery would be unreliable.
Runtime Pieces
| Piece | Responsibility |
|---|
| Tool registry | Stores tool specs and handlers by name. |
| Tool router | Selects the handler for a raw tool call and controls which tools are visible to the model. |
| Tool runtime | Executes calls, emits tool events, applies output projection, and returns model-visible results. |
| Extension bundles | Add dynamic tools from MCP, skills, and tasks to the same runtime as built-ins. |
| Output policy | Separates what the model sees from what the timeline, storage, and recovery systems retain. |
| Retry classifier | Classifies tool failures as recoverable, fatal, partial, or successful so the agent can decide whether to retry. |
exec_commandwrite_stdinread_filelist_dirgrep_filesapply_patchweb_searchweb_fetchdownload_urlread_skillrequest_tools
Large domain tools are registered in the same router but hidden until turn preflight or an explicit request_tools call makes them visible.
| Domain | Tools |
|---|
memory | memory_search, memory_list, memory_get, memory_remember, memory_forget |
task | task_create, task_wait, task_cancel, task_update, task_detach, task_list, task_get, task_reschedule, task_pause, task_resume |
artifact | artifact_prepare, artifact_register |
computer_use | computer_use |
artifact_prepare gives the model a safe turn-scoped staging path for a user-visible output file. artifact_register imports a completed regular file into the workspace artifact store and binds it to the current turn. Tool authors should not invent filesystem scanning or tool-specific discovery paths; all agent-created user results use this explicit registration contract.
Shell execution uses unified sessions. exec_command can start a process and return either final output or a session id. write_stdin can continue an existing session. Active sessions are bounded and buffered so long-running commands do not grow memory without limit.
Filesystem tools are intentionally direct: read files, list directories, search with grep-style behavior, and apply patches. Manual patch edits in Pioneer are routed through the patch tool path rather than ad hoc file rewriting.
Web tools are optional at runtime and controlled by gateway configuration. They can search, fetch, and download with limits.
Computer use uses native automation/screenshot dependencies where available. It is part of the same tool router, not a special client-side channel, but the computer_use schema is hidden until preflight or request_tools reveals the computer_use domain.
Lazy Visibility
The router stores all specs for a turn, but the model only receives the currently visible subset.
Turn preflight is the first visibility selector. It receives a compact PreflightToolIndex: coreTools plus concrete hidden-domain candidateTools with name, domain, short summary, and mutation flag. It does not receive full JSON schemas. The preflight output tools.visibleTools must be exact tool names from candidateTools, never domain names.
request_tools is the in-turn expansion path. Its schema accepts:
{
"domains": ["task", "artifact"],
"reason": "Need to delegate work and register generated files."
}
domains is an enum: memory, task, artifact, or computer_use. The handler expands each requested domain to every registered and available tool in that domain, reports already-visible and unavailable tools, and returns a compact result without embedding hidden schemas. The agent loop applies the result before the next provider round and keeps visibility monotonic for the rest of the turn.
Dynamic tools are exposed through ToolExtensionBundle.
MCP tools are created from live MCP server catalogs. Pioneer maps a server tool into a callable tool descriptor, then calls back into McpService when the model invokes it.
Skill tools are declared by installed skills and validated by the skills runtime. They can represent shell, HTTP, or function-proxy style tools depending on the skill declaration and policy.
Task tools are injected when task orchestration is available. They let an agent create attached tasks, wait for them, cancel them, detach them, and observe terminal child task results.
The important architectural choice is that dynamic tools do not bypass the router. They are added to the same runtime, use the same output projection path, and emit the same kind of tool events. This is what lets the agent loop treat a built-in shell command, an MCP call, and a skill-provided function as variations of the same execution contract.
Dynamic extension tools that are not in the built-in domain map are independent of preflight domain selection and request_tools. If an MCP server or skill contributes a turn-specific tool, including through an explicit composer capability, the final visibility pass keeps it available as a dynamic extension when it is registered and has a handler.
Output Projection
A tool call produces more than one view of output. The model-visible view may be shorter or structured differently from the timeline view. Storage may keep full output, a summary, metadata only, or nothing depending on policy. Recovery evidence can include exit status, error class, retry hints, excerpts, and fingerprints.
This matters because a large shell output or MCP response should not automatically flood the LLM context window. Tool authors should define output policies deliberately.
Retry And Recovery
Tool failures are normalized into ToolOutcome with status and error class. Recoverable errors can trigger another provider round with a retry instruction. Fatal errors are surfaced to the model or fail the turn depending on context.
The agent does not blindly retry forever. Retry decisions are constrained by per-episode and per-tool budgets, and the tool loop itself has a separate round/call budget.
What Not To Put Here
Provider-specific tool-call parsing belongs in Provider System. The tools crate should receive normalized tool calls.
Prompt prose about when to use tools belongs in Prompt And Context. The tools crate should define capabilities and execution behavior, not assistant personality.
MCP server lifecycle belongs in MCP Architecture. Skills installation and trust policy belong in Skills Architecture. The tools layer only receives their already-materialized tool descriptors and handlers.
Related Pages
