> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getpioneer.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Secret Storage

> How Pioneer stores provider keys, MCP secrets, gateway bearer tokens, and JWT signing material.

Pioneer stores runtime secret values in a local SQLite keystore file named `keystore.db`.

For the gateway, the file lives under the gateway runtime home next to `gateway.db`. Run `pioneer status` to see the runtime home path for the current installation.

The secret architecture has two goals:

1. keep raw secret values out of ordinary domain tables, config files, protocol read responses, and client registry files;
2. keep the non-secret metadata needed for UX, auditing, and cleanup in the owning domain.

Pioneer therefore stores a redacted row or secret ref in `gateway.db`; the raw value lives in `keystore.db`.

## Layers

| Layer                  | Code                                                     | Responsibility                                                                                           |
| ---------------------- | -------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| Keystore facade        | `crates/keystore`                                        | Stable secret ids, `db-keystore` integration, permission hardening, in-memory test backend.              |
| Gateway secret service | `crates/gateway/src/secrets.rs`                          | Provider keys, MCP secrets, remote-access key, superuser JWT material, status reports, GC, JWT rotation. |
| Desktop secret helpers | `crates/desktop`, `crates/client` gateway secret helpers | Gateway bearer token refs and platform-specific client token storage.                                    |
| Domain tables          | `gateway.db` via `pioneer-crud`                          | Redacted metadata, install rows, secret refs, audit rows, provider key configured state.                 |
| Protocol               | `crates/protocol`                                        | Secret write DTOs and redacted read DTOs.                                                                |

## What goes in the keystore

The keystore stores secret values that should not be written into ordinary config or domain tables:

| Secret kind                | Stored value                                                                                    |
| -------------------------- | ----------------------------------------------------------------------------------------------- |
| Workspace provider API key | API keys and tokens used by gateway provider adapters, scoped by workspace id and provider name |
| MCP secret                 | Secret `env` values, HTTP headers, tokens, and authorization values from MCP install config     |
| Remote-access key          | Rathole relay token used by the gateway-owned remote-access supervisor                          |
| Superuser JWT material     | Signing material for the current singleton superuser bearer token flow                          |
| Desktop gateway auth token | Bearer tokens saved by the desktop app for gateway connections                                  |
| User JWT token             | Reserved namespace for future non-superuser token support                                       |

The current implemented JWT flow has one superuser signing material entry. Future user tokens are separate from the superuser namespace.

## What does not store raw secrets

`gateway-settings.toml` stores gateway-scoped runtime settings. It can contain ordinary non-secret settings such as memory switches, thread-context switches, or model-selection preferences. It can also contain the secret-backend selector:

```toml theme={null}
[secrets]
backend = "keystore"
```

It does not contain raw secret values. There should be no `jwt_secret`, provider key tables, MCP secret tables, bearer tokens, API keys, or raw authorization headers in this file.

`gateway.db` stores normal gateway domain state. For MCP, it stores installation rows, catalog snapshots, audit data, redacted transport/source data, and secret refs. Provider key metadata is resolved by workspace id and provider name. It does not store raw provider or MCP secret values.

The desktop gateway registry stores `auth_token_ref` values for gateway endpoints. It rejects the old raw `auth_token` field.

Remote-access settings store enabled/service/transport state and a secret ref. The raw remote-access key is stored through the gateway secret service and is only read back by gateway runtime code that starts the tunnel supervisor.

## Secret refs

Secret refs are stable handles that let domain code refer to a secret without carrying the secret itself.

| Domain                 | Ref shape                                                       |
| ---------------------- | --------------------------------------------------------------- |
| Provider keys          | workspace id plus provider name                                 |
| MCP secrets            | generated refs attached to redacted install/source data         |
| Remote access          | default ref `remote_access` unless settings specify another ref |
| Desktop gateway tokens | endpoint-derived auth token refs in the desktop registry        |

Refs should be treated as identifiers, not credentials. A ref may reveal that a secret exists, but it must not let a client reconstruct the secret value.

## Write flow

Provider, MCP, remote-access, and gateway-token writes follow the same pattern:

1. Validate and normalize the user-provided secret value.
2. Store the raw value through the appropriate keystore writer.
3. Persist only metadata, redacted values, configured flags, or refs in the domain table/settings file.
4. Return a redacted snapshot to the client.

If step 2 succeeds and step 3 fails, the owning flow must either roll back or make the orphan detectable. MCP has explicit orphan reporting and garbage collection because install/update flows can leave old refs behind after config changes.

## Read flow

Read APIs should not return raw secrets. They return:

* configured/unconfigured booleans;
* redacted previews;
* secret refs;
* counts by kind;
* permission health;
* diagnostics.

Runtime code that actually needs a secret reads it server-side. Examples: provider adapters resolve API keys through `GatewaySecrets`; MCP server startup materializes env/header secrets; remote access resolves its relay key before applying desired state; JWT token issuance reads signing material.

## Security model

The current keystore is not encrypted at rest. Pioneer opens `db-keystore` with encryption disabled.

Pioneer does harden filesystem permissions for the runtime directory and keystore SQLite files, including `keystore.db`, `keystore.db-wal`, and `keystore.db-shm` when present. This limits normal access to the OS user that owns the gateway or desktop runtime.

Filesystem permissions are not encryption. Any OS user, administrator, backup process, malware, or service account that can read the runtime home can read `keystore.db`. Encrypting keystore contents is a separate planned task.

## Permission health

The gateway and CLI status paths inspect runtime directory and keystore file permissions. Health covers the keystore file and SQLite sidecar files such as `keystore.db-wal` and `keystore.db-shm` when present.

Permission hardening is best-effort. It improves local process isolation for the current OS user, but it is not a cryptographic boundary. Do not describe it as encryption or secure enclave storage.

## Garbage collection

MCP secrets can become orphaned when an MCP install is updated or uninstalled. Pioneer compares active MCP installation refs in `gateway.db` with MCP secret entries in the keystore and can report or delete orphaned MCP secrets.

Garbage collection is intentionally scoped. It does not delete provider keys, remote-access keys, desktop gateway tokens, or JWT material because those have different ownership and lifecycle rules.

## JWT material

The gateway currently has singleton superuser JWT signing material. `pioneer issue-superuser-token` signs a privileged bearer token with that material. Rotating superuser JWT material invalidates existing superuser tokens and requires issuing a new one.

JWT signing material is not printed during rotation. Token issuance prints a bearer token because that is the explicit purpose of the command; users should treat it like a password.

## Maintenance commands

Inspect keystore status without printing secret values:

```bash theme={null}
pioneer secrets status
pioneer secrets status --json
```

The report includes the storage path, encryption mode, counts by secret kind, permission health, and MCP orphan status. If `gateway.db` does not exist yet, MCP orphan status is reported as unavailable.

Clean orphan MCP secret values:

```bash theme={null}
pioneer secrets garbage-collection --dry-run
pioneer secrets garbage-collection
pioneer secrets garbage-collection --json
```

Garbage collection only applies to MCP secret values whose refs are no longer present in `gateway.db`. It does not delete workspace provider keys, desktop gateway tokens, or JWT material. The command refuses to run when `gateway.db` is missing.

Rotate the singleton superuser JWT signing material:

```bash theme={null}
pioneer secrets rotate-jwt-token superuser
pioneer secrets rotate-jwt-token superuser --json
```

Rotation does not print token material. If signing material already existed, existing superuser bearer tokens become invalid. Issue a new bearer token after rotation:

```bash theme={null}
pioneer issue-superuser-token
```

## Related pages

* [CLI Commands](/cli/commands) lists the command syntax.
* [Persistence Layer](/architecture/persistence) explains how `gateway.db` and `keystore.db` split state.
* [MCP Architecture](/architecture/mcp) explains MCP secret refs and redaction.
* [Remote Access Architecture](/architecture/remote-access) explains how the relay key is resolved and passed to the tunnel supervisor.
* [Protocol Layer](/architecture/protocol) explains why read responses must be redacted.