Encrypting Cost Data at Rest with KMS Envelope Encryption

Stored cost records reveal a company’s spend shape, tenant footprints, and capacity plans, and this page envelope-encrypts them with AWS KMS — a per-record data key from KMS, AES-256-GCM sealing the payload, and the wrapped key stored alongside the ciphertext.

Back to: Security & Access Control for Cost Data

Once a pipeline has read billing data under a least-privilege Cost Explorer grant, the records it writes to S3, DynamoDB, or a warehouse become a standing liability: a chargeback ledger is a map of exactly where the money goes. Server-side encryption at the bucket level protects against stolen disks, but it leaves plaintext visible to anyone with read access to the store. Envelope encryption moves the boundary into the application: the record is ciphertext everywhere except the microsecond it is being processed, and decrypting it requires a live kms:Decrypt call that IAM and CloudTrail both govern.

The mechanism is deliberately simple. KMS holds a customer master key (CMK) that never leaves the HSM. For each record (or batch) you ask KMS to mint a data encryption key (DEK); it hands back that key twice — once in plaintext and once encrypted under the CMK. You encrypt the payload with the plaintext DEK using AES-256-GCM locally, then throw the plaintext DEK away and persist only the ciphertext and the encrypted DEK. Nobody can read the record without first calling KMS to unwrap the DEK, and KMS enforces that call against the same access-control policies that guard the cost estate. This is why “envelope” — the small key is sealed in an envelope only KMS can open, and the bulk data rides along cheaply.

Why not just send every record to kms:Encrypt directly? KMS caps direct encryption at 4 KB and every call is a network round trip you pay for and get throttled on. Envelope encryption calls KMS once per DEK and does the bulk symmetric work locally at gigabytes per second, so it scales to millions of cost records while keeping the root key in hardware.

Prerequisites

Before encrypting a single record, confirm the following.

  • A KMS symmetric CMK in the same region as your data store, with automatic key rotation enabled. Note its key ID or alias (e.g. alias/cost-data-cmk).

  • IAM permissions for the pipeline principal: kms:GenerateDataKey on the write path and kms:Decrypt on the read path — nothing more. Grant these on the specific key ARN, following the least-privilege posture from the companion IAM page.

    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "EnvelopeEncryptCostData",
      "Effect": "Allow",
      "Action": ["kms:GenerateDataKey", "kms:Decrypt"],
      "Resource": "arn:aws:kms:us-east-1:111111111111:key/abcd-1234"
    }
  ]
}

  • Python 3.10+ with boto3 and the cryptography library (which provides AES-GCM via OpenSSL bindings — do not hand-roll a cipher).

    pip install "boto3>=1.34" "cryptography>=42.0"

Step-by-Step Implementation

You will request a data key, seal a cost record with AES-256-GCM, persist a self-describing envelope, implement the decrypt path, and finally rotate keys without touching existing ciphertext.

Step 1 — Request a data key from KMS

generate_data_key returns both representations of a fresh 256-bit DEK. Pass an encryption context — a set of non-secret key/value pairs KMS binds to the ciphertext as additional authenticated data; the identical context must be supplied at decrypt time, which cryptographically ties each DEK to the tenant and record type it was minted for.

import boto3

kms = boto3.client("kms", region_name="us-east-1")
CMK_ID = "alias/cost-data-cmk"


def new_data_key(tenant_id: str) -> tuple[bytes, bytes]:
    """Return (plaintext_dek, encrypted_dek) for one tenant's cost records."""
    resp = kms.generate_data_key(
        KeyId=CMK_ID,
        KeySpec="AES_256",                       # 32-byte key for AES-256-GCM
        EncryptionContext={                      # bound as AAD; not secret
            "tenant_id": tenant_id,
            "data_class": "cost-record",
        },
    )
    return resp["Plaintext"], resp["CiphertextBlob"]


# Expected shapes:
#   plaintext  -> 32 raw bytes (never persisted)
#   encrypted  -> ~184-byte opaque blob (safe to store)

Step 2 — Seal the payload with AES-256-GCM

AES-GCM is authenticated encryption: it produces ciphertext plus a tag that detects any tampering on decrypt. Generate a fresh 96-bit nonce per record — reusing a nonce under the same key is catastrophic for GCM — and bind the same encryption context as associated data so the record cannot be silently moved to another tenant.

The diagram below shows how the plaintext DEK is used once and discarded while the wrapped DEK and ciphertext are what actually get stored.

KMS envelope encryption data flow: GenerateDataKey, local AES-256-GCM seal, stored envelope, and the KMS Decrypt read pathKMS GenerateDataKey emits a plaintext DEK and an encrypted DEK. The plaintext DEK, the cost record, and a nonce feed an AES-256-GCM seal producing ciphertext and a tag; the plaintext DEK is then wiped. The stored envelope holds the encrypted DEK, nonce, ciphertext, and tag. To read, the encrypted DEK goes to KMS Decrypt, which returns the plaintext DEK, and AES-256-GCM open recovers the record.KMSGenerateDataKeyDEK fork:plaintext + encryptedAES-256-GCM sealplaintext DEK + record+ nonce -> ct + tagthen wipe plaintext DEKStored envelopeenc DEK + nonce+ ciphertext + tagKMS Decryptunwrap -> plaintext DEKencrypted DEK persisted as-isread path: recover record
import os
from cryptography.hazmat.primitives.ciphers.aead import AESGCM


def seal(plaintext_dek: bytes, payload: bytes, aad: bytes) -> tuple[bytes, bytes]:
    """Encrypt payload with a one-time nonce; return (nonce, ciphertext||tag)."""
    aesgcm = AESGCM(plaintext_dek)
    nonce = os.urandom(12)                       # 96-bit nonce, unique per record
    ct = aesgcm.encrypt(nonce, payload, aad)     # tag is appended to ciphertext
    return nonce, ct

Step 3 — Persist a self-describing envelope

Store everything the decrypt path needs except the plaintext DEK. Version the envelope so a future algorithm or format change is a field, not a migration. Base64-encode the binary parts for JSON stores; keep them raw for binary columns.

import base64
import json
from decimal import Decimal


def encrypt_record(record: dict, tenant_id: str) -> str:
    """Envelope-encrypt one cost record into a storable JSON string."""
    plaintext_dek, encrypted_dek = new_data_key(tenant_id)
    try:
        payload = json.dumps(record, default=str).encode("utf-8")
        aad = f"{tenant_id}|cost-record".encode("utf-8")   # mirrors EncryptionContext
        nonce, ciphertext = seal(plaintext_dek, payload, aad)
    finally:
        # Best-effort wipe; do not let the plaintext key linger in the frame.
        del plaintext_dek

    envelope = {
        "v": 1,
        "alg": "AES-256-GCM",
        "tenant_id": tenant_id,
        "encrypted_dek": base64.b64encode(encrypted_dek).decode(),
        "nonce": base64.b64encode(nonce).decode(),
        "ciphertext": base64.b64encode(ciphertext).decode(),
    }
    return json.dumps(envelope)


if __name__ == "__main__":
    rec = {"service": "RDS", "cost": Decimal("184.20"), "day": "2026-06-15"}
    blob = encrypt_record(rec, tenant_id="team-payments")
    print(blob[:120] + " ...")

Expected output (truncated):

{"v": 1, "alg": "AES-256-GCM", "tenant_id": "team-payments", "encrypted_dek": "AQIDAHh...", "nonce": "3k9f...", ...

Step 4 — The decrypt path

Reverse the process: unwrap the encrypted DEK via kms:Decrypt with the same encryption context, then open the AES-GCM ciphertext with the same AAD. If either the context or the tag fails, decryption raises rather than returning garbage — that is the authenticity guarantee doing its job.

from cryptography.exceptions import InvalidTag


def decrypt_record(blob: str) -> dict:
    """Recover the original cost record from a stored envelope."""
    env = json.loads(blob)
    resp = kms.decrypt(
        CiphertextBlob=base64.b64decode(env["encrypted_dek"]),
        EncryptionContext={                       # MUST match the encrypt call
            "tenant_id": env["tenant_id"],
            "data_class": "cost-record",
        },
    )
    plaintext_dek = resp["Plaintext"]
    try:
        aesgcm = AESGCM(plaintext_dek)
        aad = f"{env['tenant_id']}|cost-record".encode("utf-8")
        payload = aesgcm.decrypt(
            base64.b64decode(env["nonce"]),
            base64.b64decode(env["ciphertext"]),
            aad,
        )
    except InvalidTag:
        raise ValueError("ciphertext failed authentication — tampered or wrong key")
    finally:
        del plaintext_dek
    return json.loads(payload)


# Expected:
#   decrypt_record(blob) -> {'service': 'RDS', 'cost': '184.20', 'day': '2026-06-15'}

Step 5 — Rotate keys without re-encrypting everything

KMS automatic rotation rotates the CMK’s backing material yearly while keeping the same key ID, so kms:Decrypt transparently unwraps DEKs sealed under any past CMK version — old envelopes keep working untouched. What rotation does not do is change the per-record DEKs; each record was sealed with its own DEK, so their exposure is already isolated. If you must force a full re-key (a suspected DEK leak, or a compliance re-wrap), stream the envelopes through decrypt-then-re-encrypt, minting a fresh DEK per record:

def rewrap(blob: str) -> str:
    """Re-encrypt a record under a brand-new data key (e.g. after a leak)."""
    record = decrypt_record(blob)               # unwrap old DEK, open payload
    return encrypt_record(record, json.loads(blob)["tenant_id"])  # fresh DEK

Because each record carries its own encrypted_dek, re-wrapping is embarrassingly parallel and resumable — batch it the same way you would an idempotent historical backfill so an interrupted re-key resumes without double-processing.

Verification

Prove round-trip integrity and the security properties before trusting the store.

  1. Round-trip assertion. Encrypt then decrypt and confirm equality:

    src = {"service": "DynamoDB", "cost": "12.44", "day": "2026-06-20"}
assert decrypt_record(encrypt_record(src, "team-search")) == src

  2. Tamper detection. Flip one byte of the stored ciphertext and confirm decrypt_record raises ValueError — GCM’s tag must reject the modified record rather than return corrupted cost data.

  3. Wrong-context rejection. Call kms.decrypt with a mismatched tenant_id in the encryption context and confirm KMS raises InvalidCiphertextException. This proves a record sealed for one tenant cannot be unwrapped as another’s.

  4. Audit the key usage. Every unwrap is a Decrypt event in CloudTrail carrying the encryption context. Grep the trail to confirm only the pipeline role calls Decrypt, and feed anomalies into your cost-and-access alert routing.

Gotchas & Edge Cases

  • Never reuse an AES-GCM nonce under the same key. Two records sealed with the same DEK and the same nonce leak the XOR of their plaintexts and destroy the tag’s integrity guarantee. Generating a fresh 12-byte os.urandom nonce per record — and a fresh DEK per record or small batch — keeps you far from the birthday bound.
  • Persist the encrypted DEK, never the plaintext. The single fatal mistake is logging or storing resp["Plaintext"]. Scrub it from debug logs and delete the reference as soon as the seal completes; a plaintext DEK next to its ciphertext defeats the entire scheme.
  • Encryption context is authenticated, not confidential. It is stored and logged in cleartext, so never put secrets in it — use it for binding identifiers (tenant, data class) that you want recorded in CloudTrail for audit.
  • KMS throttles GenerateDataKey and Decrypt. The shared account quota (region-dependent, often a few thousand requests/second) means a per-record DEK on a million-row batch will throttle. Batch multiple records under one DEK, or cache a DEK for a short window, and apply retry-with-backoff on the KMS calls.
  • Cross-region reads need a multi-region key or re-wrap. A single-region CMK cannot decrypt in another region. If cost records replicate across regions for DR, use a KMS multi-region key so the replica region can unwrap locally, or re-wrap on replication.
  • Decimal survives the round trip only as a string. json.dumps(..., default=str) serializes Decimal cost as a string; the decrypted record carries "184.20", not a float. Re-coerce to Decimal on read to preserve the precision the billing schema contract depends on.

Frequently Asked Questions

Why not encrypt each record with kms:Encrypt directly?

Direct KMS encryption is capped at 4 KB per call and every call is a throttled network round trip you pay for. Envelope encryption calls KMS once to mint a data key, then does the bulk symmetric work locally with AES-256-GCM at memory speed. It scales to millions of records while keeping the root key inside the KMS HSM, which is exactly the trade-off cost pipelines need.

What is the encryption context actually for?

It is a set of non-secret key/value pairs KMS binds to the wrapped DEK as additional authenticated data. The identical context must be supplied at decrypt time, so a record sealed with tenant_id=A cannot be unwrapped as tenant_id=B even by a caller with full Decrypt permission. It also lands in CloudTrail, giving you a per-decrypt audit trail keyed by tenant and data class.

Does enabling KMS automatic rotation re-encrypt my stored records?

No. Automatic rotation changes the CMK’s backing key material yearly while keeping the same key ID, and Decrypt transparently unwraps DEKs sealed under any prior version, so existing envelopes keep working with zero migration. It does not touch the per-record data keys. Forcing a full re-key is a separate, deliberate decrypt-then-re-encrypt pass you run only after a suspected key leak or for compliance.

How do I handle a suspected data-key leak?

Because every record carries its own encrypted DEK, a single leaked DEK exposes only that one record. Re-wrap the affected records by decrypting and re-encrypting them under fresh data keys; the operation is parallel and resumable per record. If the CMK itself is suspected, disable it, rotate to a new CMK, and re-wrap the whole store as an idempotent batch job.

Can I use the same pattern outside AWS?

Yes. The envelope pattern is provider-neutral: Azure Key Vault and GCP Cloud KMS both expose a wrap/unwrap or generate-data-key equivalent, and the local AES-256-GCM step is identical. Only the KMS client calls change, which makes envelope encryption a natural fit for records normalized across providers in a multi-cloud cost model.

Back to: Security & Access Control for Cost Data