Store encrypted validator keys in DynamoDB with Shamir shares protected by AWS KMS
The DynamoDB key source persists encrypted BLS validator private keys in a single DynamoDB table. Each key is encrypted with AES-256-GCM under a 256-bit master key; the master key itself never lives in plaintext on disk — it is reconstructed at unseal time from Shamir shares wrapped by AWS KMS.
This is one of three pluggable key sources: Filesystem, DynamoDB, and Memory.
When to use
Section titled “When to use”- AWS-native deployment with managed DynamoDB
- Need durable key storage that survives signer restarts
- Want multi-account custody — KMS keys across multiple AWS accounts, no single account holds the master key
- Need runtime key generation (
/api/v1/chamber/keys/generate) or import (/api/v1/chamber/keys)
Generated keys are persisted as inactive first. Activate them with the Chamber key lifecycle API after the validator is ready to use the key for signing.
For non-AWS deployments, prefer Filesystem with EIP-2335 keystores.
Storage layout
Section titled “Storage layout”One DynamoDB table holds encrypted validator keys. Each row is keyed by validator public key and includes:
- The encrypted BLS secret key.
- An
activeorinactivestatus. status_shard, used by the requiredstatus-shard-indexglobal secondary index.created_at, cipher-version metadata, and a row HMAC that covers the key material, status, shard, timestamp, cipher version, and optional mnemonic backup.- An optional age-encrypted mnemonic backup for keys generated by the chamber.
The table must expose a GSI named status-shard-index with status_shard as its hash key. Startup and background refresh query this index across 16 shards for every status in status_filter.
Configuration
Section titled “Configuration”key_sources: dynamodb: table: containment-keys status_filter: [active] refresh_interval_seconds: 1 max_concurrent_reads: 16Field reference
Section titled “Field reference”| Field | Default | Description |
|---|---|---|
table |
required | DynamoDB table name for encrypted validator keys |
status_filter |
["active"] |
Which key statuses to load into the signer |
refresh_interval_seconds |
1 |
Background reload interval. Set to 0 to disable refresh |
max_concurrent_reads |
16 |
Parallel read workers for startup and refresh loading |
These knobs configure the storage only. The HTTP surface that reads from and writes to it (generate, import, list, patch, delete) has its own per-verb gating under chamber.keys.* — see Chamber Key Management.
Master-key lifecycle
Section titled “Master-key lifecycle”Validator keys are encrypted under a single 256-bit master key. The master key itself is reconstructed during seal/unseal from KMS-wrapped Shamir shares and held only in memory.
The master-key metadata is not part of the key-source table. It lives in the state backend. The key-source table holds encrypted validator keys and optional mnemonic backups.
Adjacent configuration
Section titled “Adjacent configuration”A working DynamoDB key source requires three independent subsystems, each with its own configuration root:
| Subsystem | Config root | This page covers? |
|---|---|---|
| Key source — encrypted validator keys | key_sources.dynamodb |
yes |
| State backend — master-key shares, ceremony config, and auth records | signer_state.* |
no, see Signer State Backend |
| Anti-slashing — EIP-3076 slashing protection (any backend works) | anti_slashing.* |
no, see Anti-Slashing |
The three are independent. You can mix freely — DynamoDB key source + PostgreSQL anti-slashing + DynamoDB state backend is a perfectly valid configuration. The all-AWS pattern below picks DynamoDB for all three because it inherits managed durability for free.
Minimum required adjacent config
Section titled “Minimum required adjacent config”When the DynamoDB key source is active, the chamber refuses to boot without a state backend configured (the master key has to live somewhere). A minimal valid config:
key_sources: dynamodb: table: containment-keys
signer_state: backend: dynamodb table: containment-stateAnti-slashing is independent — pick any backend (noop, sqlite, postgres, dynamodb) from the anti-slashing page. For an all-DynamoDB stack:
anti_slashing: backend: dynamodb table: containment-slashingThree distinct tables — the chamber refuses to start if key_sources.dynamodb.table == signer_state.table or any pair collides.
Multi-account KMS
Section titled “Multi-account KMS”The Shamir threshold can split across KMS keys in separate AWS accounts. A 2-of-3 setup means any two accounts can reconstruct the master key, but no single account holds enough material to do it alone — the strongest custody story the DynamoDB key source supports.
The KMS key ARNs, Shamir threshold, and root-token recipients are declared in the static ceremony: config block (non-Nitro deployments) or compiled into the binary (Nitro). The chamber reads them at boot — no operator-supplied parameters at runtime. On first boot with a fresh state table, the master key is auto-initialized: the chamber generates it, Shamir-splits it, KMS-wraps each share, and writes the MASTER_KEY row. See Signer State Backend for the full row layout.
For IAM setup, see AWS IAM Permissions. Ready-to-use Terraform examples live in terraform/examples/single-account/ and terraform/examples/multi-account/. (These repository links resolve once the project is public.)
Disaster recovery
Section titled “Disaster recovery”age-encrypted mnemonic backup
Section titled “age-encrypted mnemonic backup”When chamber.keys.backup.recipients is non-empty, every key generated by POST /api/v1/chamber/keys/generate or POST /api/v1/chamber/keys/generate-keys has its BIP-39 mnemonic encrypted with the configured age public keys and stored in the same DynamoDB row as the encrypted secret key. This is your last-resort recovery path that doesn’t depend on AWS:
chamber: keys: generate: enabled: true backup: recipients: - "age1..."To recover a key when KMS is lost:
- Read
encrypted_mnemonicfrom the DynamoDB row - Decrypt with your age private key (stored offline)
- Re-derive the BLS keypair via EIP-2333
- Import into any compatible signer
A non-empty chamber.keys.backup.recipients list enables the backup; an empty or absent list disables it.
What happens when KMS is lost
Section titled “What happens when KMS is lost”If fewer than the configured Shamir threshold of KMS keys are accessible, the signer cannot reconstruct the master key and will refuse to unseal. It logs a clear error indicating which KMS keys failed. If you cannot restore access to enough KMS keys, use the age backup to recover the mnemonics and re-import the keys.
Troubleshooting
Section titled “Troubleshooting”KMS access denied
Section titled “KMS access denied”Error: KMS Decrypt failed: AccessDeniedExceptionThe signer’s IAM role lacks kms:Decrypt on one or more KMS keys. Check:
- The IAM role attached to the instance or pod has the correct policy
- The KMS key resource policy grants access to the role
- For cross-account: both the role’s policy AND the target KMS key’s resource policy grant access
aws kms describe-key --key-id arn:aws:kms:...See AWS IAM Permissions for the full policy reference.
Insufficient KMS shares
Section titled “Insufficient KMS shares”Error: Shamir reconstruction failed: insufficient sharesThe signer needs at least the threshold’s worth of KMS keys. Check:
- Network connectivity to the KMS endpoints in each region (especially for cross-region multi-account setups)
- IAM access to every KMS key in the set, not just one
If you can’t restore access to enough KMS keys, use the age backup to recover the mnemonics and re-import the keys.
DynamoDB throttling
Section titled “DynamoDB throttling”Error: ProvisionedThroughputExceededExceptionThe DynamoDB table is throttled. Options:
- Switch to
PAY_PER_REQUESTbilling mode (recommended for variable workloads) - Lower
key_sources.dynamodb.max_concurrent_readsto reduce burst read load - Increase provisioned RCUs/WCUs if you must stay on provisioned billing
HMAC mismatch on unseal
Section titled “HMAC mismatch on unseal”Error: Master key HMAC verification failedThe reconstructed master key did not match the stored hash. This means either:
- The KMS keys accessible to the signer are not the ones that wrapped the shares in the
MASTER_KEYrow — for example, the ceremony static was updated but a reconcile has not yet run, or the IAM role lacks access to some keys - The
MASTER_KEYrow was tampered with
Don’t proceed if the HMAC fails. Restore the original KMS keys, or use the age backup to recover and re-import keys against a fresh master key.
Key row HMAC mismatch
Section titled “Key row HMAC mismatch”Error: key row HMAC mismatchThe validator-key row failed integrity verification. The HMAC covers the public key, encrypted key, active/inactive status, status shard, creation timestamp, cipher version, and optional encrypted mnemonic. Treat this as table corruption or tampering. Restore the row from a trusted DynamoDB backup before making the key active again.
Next Steps
Section titled “Next Steps”- Chamber Key Management — generate, import, list, patch, delete endpoints
- Signer State Backend — where the master key, auth policies, and auth tokens live
- Anti-Slashing — EIP-3076 slashing protection, independent of this key source
- Seal State Machine — state-machine transitions, KMS auto-unseal, and break-glass seal

