Signer State Backend

The signer state backend stores Containment Chamber control-plane state. It is separate from validator key storage and anti-slashing storage, even when all three use DynamoDB.

Use this page when enabling DynamoDB-backed keys, runtime auth policies and tokens, or multiple signer replicas.

When You Need It

Deployment shape	State backend required?	Why
Filesystem-only signer with static auth	No	Keys and auth policy are loaded from local config at boot
Runtime auth policies and tokens	Yes	Policy and token changes must survive restarts and propagate across replicas
DynamoDB key source	Yes	The master key must survive restarts; KMS auto-unseal and ceremony reconcile require durable state
Multiple signer replicas	Yes for stateful features	Replicas use the backend to converge on seal state, auth changes, and reconcile progress

signer_state:
  backend: dynamodb
  table: containment-state
  refresh_interval_seconds: 1
  max_watcher_staleness_seconds: 3

Use a distinct DynamoDB table for signer state. Do not reuse the validator-key table or the anti-slashing table.

What It Stores

The state backend holds six row types. Validator private keys and slashing-protection records are stored elsewhere — see DynamoDB Key Source and Anti-Slashing.

Row PK	Contents
MASTER_KEY	KMS-wrapped Shamir shares, master-key commitment hash, share-binding HMAC, Shamir threshold, ceremony generation counter, and ceremony config hash. Written once at auto-init; updated in place by reconcile when the ceremony static changes.
SEAL_OVERRIDE	Break-glass seal latch. Presence prevents KMS auto-unseal at boot and signals every live replica to clear its in-memory master key and enter Sealed. HMAC’d by a master-key subkey so it cannot be forged by a DynamoDB-write-only attacker. Cleared by an IAM DeleteItem on PK SEAL_OVERRIDE followed by a restart.
ROOT_TOKEN_BOOTSTRAP	age-encrypted root management token ciphertext, written at auto-init and overwritten on re-create. Read out-of-band by an authorized operator with age -d; never read by the server at runtime.
AUTH_GENERATION	Monotonic auth-generation counter and current root token accessor. Guards root re-create atomicity and lets a re-create delete the prior root without a full table scan.
AUTH_TOKEN#{accessor}	Runtime auth tokens, HMAC’d by a master-key-derived secret.
AUTH_POLICY#{name}	Runtime auth policies.

How It Differs From Other Storage

Storage	Holds	Used by
Key source	Validator private keys or encrypted key material	Signing and key-management operations
Signer state backend	Seal state, KMS-wrapped master-key shares, root-token bootstrap, and runtime auth records	Chamber control-plane operations
Anti-slashing backend	Slashing-protection history	Ethereum signing safety checks

The signer state backend does not store validator private keys. Those live in the configured key source. It does not store slashing-protection records. Those live in the anti-slashing backend.

Runtime Behavior

The background StateWatcher task polls the state backend on signer_state.refresh_interval_seconds. At boot and on each tick it:

1. Checks SEAL_OVERRIDE first. If the row is present and its HMAC verifies against the installed master key, the replica transitions to Sealed, zeroizes the in-memory master key, and clears the auth plane. Every subsequent tick is a no-op — Sealed is terminal until the process restarts without the latch.

2. Unseals / reconciles when Unsealed. On the first successful tick: reconstructs the master key from KMS-wrapped Shamir shares (attested KMS decrypt on Nitro), verifies the master-key commitment and share-binding HMAC, and transitions to Unsealed. On a fresh table, auto-init runs first (generate → Shamir-split → KMS-wrap → write MASTER_KEY). Subsequent ticks compare the row’s ceremony_config_hash and generation against the compiled-in (Nitro) or config-block (non-Nitro) ceremony static; if the static is ahead in generation, the watcher re-wraps the shares under the new KMS key set in a single DynamoDB transaction and re-creates the root token when recipients changed.

Set refresh_interval_seconds low enough for operational changes to propagate at an acceptable speed.

signer_state.max_watcher_staleness_seconds (default 3) is a separate guard. If a replica’s watcher snapshot grows older than this bound — because the watcher stalls on a DynamoDB outage or a panic-recovery loop — the signing handler starts refusing requests with 503 rather than signing against possibly-stale seal state. refresh_interval_seconds sets how fast a replica observes a cluster-wide seal; max_watcher_staleness_seconds sets how fast signing fails if the watcher itself stops advancing. Set it to 0 to disable the freshness check.

State Machine

The signer has exactly two states:

State	Meaning
Unsealed	Fully operational. The master key is resident in memory; signing, key management, and auth all work.
Sealed	Break-glass: a SEAL_OVERRIDE latch is present. Terminal until the latch is removed and every replica restarts.

Boot always resolves to Unsealed — auto-init on a fresh table, KMS auto-unseal on a warm table. → Sealed is break-glass, triggered by containment-chamber operator seal or observed from a SEAL_OVERRIDE row written by another replica. There is no Sealed → Unsealed transition: recovery requires an IAM DeleteItem on SEAL_OVERRIDE followed by a restart of every replica.

Troubleshooting

Symptom	Check
Signer fails to reach Unsealed at boot	Check that a ceremony: block is configured (non-Nitro) or the binary has ceremony compiled in (Nitro); verify IAM and DynamoDB access.
Auth edits do not propagate	Check refresh_interval_seconds and DynamoDB read permissions
KMS auto-unseal fails at boot	Confirm the IAM role has kms:Decrypt on every key in the ceremony set and the MASTER_KEY row exists
Signing returns 503 (watcher stale)	Check DynamoDB connectivity; the watcher has not successfully ticked within max_watcher_staleness_seconds

Note

This page documents operator-facing behavior. Use the source code when changing row names or backend implementation details.

Next Steps

• DynamoDB Key Source — configure the key table that works alongside state
• Seal State Machine — state-machine transitions, break-glass seal, and recovery
• Troubleshooting — diagnose state-backend propagation delays