Seal State Machine

The seal state machine protects the master key that encrypts validator private keys in the DynamoDB key source. The master key is never stored in plaintext: it lives Shamir-split M-of-N across N AWS KMS keys, with each share individually KMS-wrapped and stored in the MASTER_KEY DynamoDB row. At runtime, shares are KMS-decrypted and combined to reconstruct the master key in memory, then zeroized again when the signer seals or stops.

This system only applies to the DynamoDB key source. Filesystem keystores load at startup and do not use the chamber master key.

For operator procedures, see the Seal & Unseal Runbook. For request bodies and response schemas, see the API Reference.

States

There are exactly two states:

State	Signing	Description
unsealed	enabled	Master key is reconstructed in memory. The signer is fully operational.
sealed	disabled	Break-glass terminal: a SEAL_OVERRIDE latch row is present in DynamoDB. No key material is resident. There is no automatic recovery — a restart is required after the latch is manually removed.

State Machine

direction: down

unsealed: "Unsealed\nsigning enabled"
sealed: "Sealed — terminal\nsigning disabled"

unsealed.style.fill: "#CAF2E6"
unsealed.style.stroke: "#13A477"
unsealed.style.font-color: "#170206"
sealed.style.fill: "#FEEC8C"
sealed.style.stroke: "#D35F0A"
sealed.style.font-color: "#170206"

unsealed -> sealed: "break-glass: SEAL_OVERRIDE latch" {style.stroke: "#D35F0A"}

Recovery from sealed is not a state transition within the running process: delete the SEAL_OVERRIDE DynamoDB row via IAM and restart every replica. There is no unseal endpoint and no automatic resume.

Danger

No unseal endpoint exists. Once sealed, the only recovery path is a manual IAM DeleteItem on the SEAL_OVERRIDE row (partition key pk = SEAL_OVERRIDE) followed by a restart of every replica. See the Runbook for the step-by-step procedure.

Boot Flow

Every boot resolves state before any replica begins serving traffic. The boot sequence:

1. Check SEAL_OVERRIDE first. If the latch row is present, the replica starts directly in Sealed — no auto-init, no auto-unseal, no key material in RAM. This check happens before any other work.
2. Fresh table (no MASTER_KEY row) + ceremony configured → auto-init: generate a random 256-bit master key → Shamir-split → KMS-wrap each share under the ceremony key set → self-test (KMS decrypt + Shamir-combine + commitment verify) before any durable write → write MASTER_KEY row atomically with a freshly-created root token (age-encrypted to root_token_recipients, stored in ROOT_TOKEN_BOOTSTRAP) → Unsealed.
3. Existing MASTER_KEY row → KMS auto-unseal: reconstruct the master key from the KMS-wrapped shares (attested decrypt on Nitro), verify the master-key commitment and the share-binding HMAC → Unsealed.

A stateful deployment requires a ceremony. If none resolves — no ceremony: block on non-Nitro, no compiled-in static on Nitro — boot fails fast with a clear error instead of coming up non-operational. Auto-init does not require a DynamoDB key store; the master key also backs auth, anti-slashing, and the seal latch, so a filesystem or import key source works too. A failed pre-write self-test (for example a KMS policy that has not yet admitted this PCR0) aborts boot with no row written, and a restart retries once the cause is fixed.

The custody parameters — KMS key set (kms_keys), threshold (kms_threshold), root-token recipients (root_token_recipients), and ceremony generation — come from the ceremony static: compiled-in and PCR0-measured on Nitro builds, or the trusted ceremony: config block on non-Nitro deployments. They are never operator-supplied at runtime.

Note

On a lost init race — two replicas both seeing an empty table and racing to auto-init — the losing replica discards its candidate key and reconstructs from the winner’s row via KMS auto-unseal. The fleet converges on a single master key with no split-brain.

SEAL_OVERRIDE Latch

The break-glass seal is a DynamoDB row (partition key pk = SEAL_OVERRIDE) written by POST /api/v1/chamber/seal (scope chamber_seal). Its properties:

• HMAC-authenticated: the row carries an HMAC computed over (sealed_at, sealed_by_replica, reason) by a master-key subkey. The watcher verifies this HMAC against the live key on observation — a forged or tampered row fails the watcher tick rather than triggering a seal (RC8 tamper detection).
• Cluster-wide: the row is visible to every replica sharing the DynamoDB table.
• Observed by the background watcher: on each tick, the watcher checks for the row first. When present and authenticated, it transitions the local replica to Sealed, zeroizes the master key from memory, and revokes all auth tokens.
• Honoured at boot: every boot checks for the latch before attempting auto-init or auto-unseal.
• Restart-surviving: the row persists in DynamoDB, so restarting a replica does not clear the seal.

Seal Verb

containment-chamber operator seal --auth-token env:VAR (scope chamber_seal):

1. Writes the HMAC-stamped SEAL_OVERRIDE row with attribute_not_exists guard. If another replica’s seal races and wins, the guard conflict is treated as success — the desired end-state is achieved.
2. Transitions the local replica to Sealed.
3. Zeroizes the master key from the in-memory key cell.
4. Revokes all auth tokens.

The fleet-wide effect follows within one watcher-tick interval on every other replica.

Recovery

To recover from Sealed:

1. Delete the SEAL_OVERRIDE DynamoDB row via IAM:

   aws dynamodb delete-item \
     --table-name <signer-state-table> \
     --key '{"pk": {"S": "SEAL_OVERRIDE"}}'

2. Restart every replica. Boot re-checks for the latch; finding none, it auto-unseals from the MASTER_KEY row.

There is no API route to remove the latch. The IAM delete-item is the only unlock mechanism.

Rotation

Rotation is declarative: there are no rotate CLI commands. To change the KMS key set, threshold, or root-token recipients:

1. Edit the ceremony static: bump generation, update kms_keys / kms_threshold / root_token_recipients, and move the old keys into retired_kms_keys (keeping them decryptable across the rollout).
2. Redeploy.
3. After auto-unseal, the background watcher’s Unsealed arm runs the reconcile pass: it hashes the static, compares it to the row’s ceremony_config_hash, and — when the static is ahead in generation — re-wraps the master-key shares under the new key set in a single DynamoDB transact. It proves the new keys can decrypt before retiring the old ones.

The replica stays Unsealed and serving throughout. Reconcile is generation-gated and idempotent — a retry on the next watcher tick is always safe (H5).

Security Properties

Property	Mechanism
Master key at rest	Never stored plaintext; 256-bit key Shamir-split M-of-N, each share KMS-wrapped and stored in the MASTER_KEY row
Master key in memory	Held only while Unsealed; zeroized on seal, process exit, or SEAL_OVERRIDE observation
Custody parameters	Compiled-in PCR0-measured static (Nitro) or trusted ceremony: config block (non-Nitro) — not operator-supplied at runtime
SEAL_OVERRIDE authenticity	HMAC over (sealed_at, sealed_by_replica, reason) by a master-key subkey; tampered row fails the watcher tick (RC8)
Break-glass scope	chamber_seal scope required; latch is cluster-wide and restart-surviving
Recovery from Sealed	IAM DeleteItem on SEAL_OVERRIDE row + restart every replica — no auto-resume, no unseal endpoint
Attested KMS decrypts (Nitro)	KMS RecipientAttestation enforcement; key material recovered only inside enclave memory

For day-to-day procedures, see the Seal & Unseal Runbook.