Security & Access Control for Cost Data

Engineering least-privilege identities, short-lived credentials, and tamper-evident audit trails so the automation that reads billing data and enforces quotas can never be turned against the accounts it protects.

Back to: Cloud Database Cost Fundamentals & Architecture

Cost telemetry is a high-value target hiding inside a low-priority pipeline. The same service account that pulls a tenant’s month-to-date spend usually also holds — one policy statement away — the ability to raise a quota, resize an Aurora cluster, or delete a preview database. That adjacency is the whole security problem: a leaked cost-reader token is only dangerous if the identity behind it can do anything beyond reading. This page treats access control for cost data as an engineering surface with the same rigor the rest of the Cloud Database Cost Fundamentals & Architecture reference applies to attribution and enforcement — how the identities are scoped, how credentials are minted and expired, the Python that keeps secrets out of code, and how every read and mutation lands in an audit trail you can later prove was not tampered with.

The sequence below traces how a cost pipeline obtains temporary credentials and reads KMS-encrypted billing exports under strict role separation, with the infrastructure operator explicitly denied access to the financial data plane.

How a cost pipeline mints scoped, expiring credentials to read encrypted billing dataA FinOps cost pipeline requests temporary credentials from workload identity STS, which assumes a scoped read-only role and returns a time-bound token. The pipeline reads cost telemetry from the encrypted billing export; the export requests a KMS decrypt and the billing key grants it only to the allowed role, then the decrypted cost data is returned to the pipeline. The pipeline exposes budget and chargeback views to a FinOps analyst, while access to the financial data plane is explicitly denied to the infra operator.FinOps costpipelineWorkloadidentity STSScoped read-only roleEncryptedbilling exportKMS billingkeyFinOpsanalystInfraoperator1 · request temp credentials2 · assume scoped role3 · time-bound token4 · read cost telemetry5 · request decrypt6 · decrypt: allowed role only7 · decrypted cost data8 · budget + chargeback views9 · DENY financial data plane

Trust Model & Access Boundaries

Before any SDK call, the design question is: which identities exist, what can each one touch, and what is the blast radius if one is compromised? Financial telemetry sits at the seam between three teams with genuinely different needs, and collapsing them into a single “cost” role is the most common failure — the direct analogue of collapsing disaggregated compute and storage cost dimensions into one instance line item. Each principal scales on its own axis of privilege and must be bounded independently.

Read-only cost consumers — dashboards, chargeback exporters, anomaly detectors — need ce:GetCostAndUsage, budgets:Describe*, and decrypt on the billing export key, and nothing else. They must never hold a write action. Enforcement identities — the automation that acts on a quota decision — need a small, explicit set of mutating actions (rds:ModifyDBCluster, budgets:CreateNotification) gated by IAM conditions, and they should be a different role from the reader so a compromised dashboard token cannot resize a database cluster. Human operators get federated, MFA-backed sessions scoped to their team’s cost-allocation tags; platform operators managing the orchestration layer are deliberately denied the financial data plane so operational and billing scopes never cross-contaminate.

The reason short-lived credentials matter is quantifiable. If a token has time-to-live $T_{ttl}$ and your mean time to detect a leak is $T_{detect}$, the window in which a stolen credential is usable is bounded by:

$$W_{exposure} = \min(T_{ttl},\ T_{detect})$$

Long-lived access keys make $T_{ttl}$ effectively infinite, so exposure collapses to how fast you notice — usually too slow. Cross-account role assumption with a 15-to-60-minute session caps $W_{exposure}$ at $T_{ttl}$ regardless of detection speed, which is why every automated reader below assumes a role rather than embedding a key.

Two edge cases dominate real incidents. Untagged resources have no tenant key, so their spend cannot be bound to any per-team access policy and falls into an unattributed bucket that everyone can see but no one owns — a boundary can only protect what attribution can name. Cross-account fan-out multiplies the trust surface: a management account reading spend from fifty member accounts needs a role in each, and one over-broad trust policy (a wildcard principal, a missing ExternalId) turns a single compromise into fifty. The role separation and per-role scoping are the hardest part of the design, so they are worth drawing explicitly.

Least-privilege separation: which identity may take which cost actionA permission matrix with four identity columns and five action rows. Read identities (cost reader, human operator) may run ce:GetCostAndUsage and kms:Decrypt but no write action. The enforcement actor may run rds:ModifyDBCluster, rds:DeleteDBCluster and budgets:CreateNotification but neither read action. The infra operator is denied all five actions, so the operational and billing scopes never cross. A leaked reader token therefore cannot mutate a database cluster, and an enforcement token cannot read raw billing data.Cost readerdashboardsEnforcementactorHumanoperatorInfraoperatorce:GetCostAndUsagekms:Decryptrds:ModifyDBClusterrds:DeleteDBClusterbudgets:CreateNotificationallowdenyreads and writes never share one identity

Secure Telemetry Extraction & Credential Handling

Once the boundaries are drawn, extraction has to honor them at runtime: mint a scoped session, decrypt only what the role is granted, and stay within the low steady-state rate limits the cost APIs enforce. This is the same pull the rest of the pipeline performs, but with the credential lifecycle made explicit rather than assumed.

The reader assumes a purpose-built role via STS, passing an ExternalId so a confused-deputy attack cannot borrow the trust relationship, and receives credentials that expire on their own. Those credentials feed a boto3 session directly — they are never written to disk, an environment file, or a log line:

import boto3

def scoped_cost_session(role_arn: str, external_id: str,
                        session_name: str = "cost-reader",
                        ttl_seconds: int = 900) -> boto3.Session:
    """Assume a read-only cost role and return a session bound to short-lived creds.

    ttl_seconds is capped to the role's MaxSessionDuration; 900s (15m) keeps the
    exposure window small for an automated, frequently-run reader.
    """
    sts = boto3.client("sts")
    assumed = sts.assume_role(
        RoleArn=role_arn,
        RoleSessionName=session_name,
        ExternalId=external_id,          # blocks confused-deputy role borrowing
        DurationSeconds=ttl_seconds,
    )
    creds = assumed["Credentials"]
    return boto3.Session(
        aws_access_key_id=creds["AccessKeyId"],
        aws_secret_access_key=creds["SecretAccessKey"],
        aws_session_token=creds["SessionToken"],
    )

Billing exports written to S3 are server-side encrypted under a customer-managed KMS key, and the export objects carry an encryption context that binds each ciphertext to its tenant. Decrypting with the matching context means a role granted decrypt for tenant=acme cannot silently read tenant=globex, even if it can list every object:

def decrypt_export_blob(session: boto3.Session, ciphertext: bytes,
                        tenant: str) -> bytes:
    """Decrypt a billing export payload, binding the tenant into the KMS context.

    The EncryptionContext must match what was supplied at encrypt time or KMS
    raises InvalidCiphertextException — this is the per-tenant cryptographic fence.
    """
    kms = session.client("kms")
    resp = kms.decrypt(
        CiphertextBlob=ciphertext,
        EncryptionContext={"tenant": tenant, "purpose": "billing-export"},
    )
    return resp["Plaintext"]

With a session in hand, the actual cost pull is the familiar paginated Cost Explorer call, grouped by a cost-allocation tag so each record is already attributable. Because these providers name the same physical dimension differently, the raw rows are mapped into the canonical shape described under normalizing provider billing exports into a unified schema, and every record is checked against strict typing and schema validation on billing data before it is trusted — a row with a missing tenant tag is a security event, not a value to silently coerce. The extraction, validation, and aggregation that surround this call live under the broader metric extraction and aggregation pipelines reference; here the point is only that the client carrying the request is a scoped, expiring identity:

def read_tenant_spend(session: boto3.Session, start: str, end: str,
                      tag_key: str = "Tenant") -> dict:
    """Pull unblended per-tenant spend using an already-scoped session.

    start/end are YYYY-MM-DD with end EXCLUSIVE (Cost Explorer convention).
    UnblendedCost keeps account-level credits/RIs from distorting per-tenant views.
    """
    ce = session.client("ce", region_name="us-east-1")
    out: dict[str, float] = {}
    token = None
    while True:
        params = {
            "TimePeriod": {"Start": start, "End": end},
            "Granularity": "DAILY",
            "Metrics": ["UnblendedCost"],
            "GroupBy": [{"Type": "TAG", "Key": tag_key}],
        }
        if token:
            params["NextPageToken"] = token
        resp = ce.get_cost_and_usage(**params)
        for day in resp["ResultsByTime"]:
            for group in day["Groups"]:
                tenant = group["Keys"][0].split("$", 1)[-1] or "unattributed"
                amount = float(group["Metrics"]["UnblendedCost"]["Amount"])
                out[tenant] = out.get(tenant, 0.0) + amount
        token = resp.get("NextPageToken")
        if not token:
            break
    return out

Python Automation Patterns

The pattern that keeps a long-running pipeline both secure and cheap is credential caching keyed on expiry: assume the role once, reuse the session until it is close to expiring, and re-assume just before it lapses. This avoids hammering STS (itself throttled) while never letting a session outlive its intended window. A thin provider wraps the assume-role call and hands out a live session:

from datetime import datetime, timedelta, timezone

class ScopedSessionProvider:
    """Cache an assumed-role session and refresh it just before expiry."""

    def __init__(self, role_arn: str, external_id: str, ttl_seconds: int = 900):
        self._role_arn = role_arn
        self._external_id = external_id
        self._ttl = ttl_seconds
        self._session: boto3.Session | None = None
        self._expires_at = datetime.min.replace(tzinfo=timezone.utc)

    def session(self) -> boto3.Session:
        # refresh with a 60s safety margin so an in-flight call never expires mid-request
        if self._session is None or datetime.now(timezone.utc) >= self._expires_at - timedelta(seconds=60):
            self._session = scoped_cost_session(self._role_arn, self._external_id,
                                                ttl_seconds=self._ttl)
            self._expires_at = datetime.now(timezone.utc) + timedelta(seconds=self._ttl)
        return self._session

STS and Cost Explorer both throttle, so every credential-bearing call is wrapped in a retry that backs off only on throttling and re-raises AccessDenied immediately — a permission error is a policy fact to surface, never something to retry into a rate-limit. This mirrors the retry logic for failed metric pulls discipline used elsewhere in the pipeline, but with an explicit carve-out so authorization failures fail fast:

import functools
import random
import time
from botocore.exceptions import ClientError

RETRYABLE = {"ThrottlingException", "Throttling", "TooManyRequestsException"}

def retry_throttled(max_attempts: int = 5, base: float = 0.5):
    """Retry throttling with full-jitter backoff; re-raise AccessDenied etc. at once."""
    def decorator(fn):
        @functools.wraps(fn)
        def wrapper(*args, **kwargs):
            for attempt in range(max_attempts):
                try:
                    return fn(*args, **kwargs)
                except ClientError as exc:
                    code = exc.response["Error"]["Code"]
                    if code not in RETRYABLE or attempt == max_attempts - 1:
                        raise            # AccessDenied surfaces immediately
                    time.sleep(random.uniform(0, base * (2 ** attempt)))
        return wrapper
    return decorator

When a management account fans out across many member accounts, the per-account assume-role and pull run concurrently under the async semaphore-controlled parsing workflow pattern. The semaphore does double duty here: it keeps the global STS and Cost Explorer request rate under the throttle ceiling, and it bounds how many decrypted tenant payloads are resident in memory at once, shrinking the window in which sensitive data is exposed to a process compromise:

import asyncio
import aioboto3

async def read_account_spend(session: aioboto3.Session, account: dict,
                             period: tuple[str, str], sem: asyncio.Semaphore) -> dict:
    async with sem:  # cap concurrent assume-role + decrypt across the whole fan-out
        async with session.client("sts") as sts:
            assumed = await sts.assume_role(
                RoleArn=account["role_arn"],
                RoleSessionName="cost-reader",
                ExternalId=account["external_id"],
                DurationSeconds=900,
            )
        creds = assumed["Credentials"]
        async with session.client(
            "ce",
            region_name="us-east-1",
            aws_access_key_id=creds["AccessKeyId"],
            aws_secret_access_key=creds["SecretAccessKey"],
            aws_session_token=creds["SessionToken"],
        ) as ce:
            resp = await ce.get_cost_and_usage(
                TimePeriod={"Start": period[0], "End": period[1]},
                Granularity="DAILY",
                Metrics=["UnblendedCost"],
                GroupBy=[{"Type": "TAG", "Key": "Tenant"}],
            )
            return {"account_id": account["id"], "raw": resp["ResultsByTime"]}


async def read_all_accounts(accounts: list[dict], period: tuple[str, str]) -> list[dict]:
    session = aioboto3.Session()
    sem = asyncio.Semaphore(5)   # honour STS + Cost Explorer rate limits globally
    tasks = [read_account_spend(session, a, period, sem) for a in accounts]
    return await asyncio.gather(*tasks)

Quota Enforcement Integration

Access control is what makes an enforcement decision trustworthy. The quota boundary design that translates normalized cost signals into hard and soft limits produces actions — throttle a database cluster, tighten a scaling envelope, tear down an expired preview database — and every one of those actions is a privileged mutation that must be gated, logged, and provably un-tampered. The security layer contributes three things to that loop.

First, separation of read and write identities. The reader that evaluates a boundary and the actor that applies it are different roles, so the decision path can be reviewed independently and a leaked reader token cannot enact enforcement. Second, policy-as-code guardrails in front of every cost mutation: budget adjustments, reservation purchases, and tag edits are evaluated by an Open Policy Agent (or an IAM condition) before they commit, so an automated actor cannot raise its own ceiling. Third, and most important for later trust, a tamper-evident audit record for every decision.

That audit record is where cost security becomes provable rather than assumed. Each enforcement event is serialized deterministically and chained with an HMAC over the previous entry’s signature, so any later edit or deletion breaks the chain and is detectable — the same chain-of-custody guarantee financial systems require:

import hmac
import hashlib
import json

def sign_audit_event(event: dict, prev_signature: str, secret: bytes) -> str:
    """Return an HMAC that chains this event to the previous one.

    Canonical JSON (sorted keys) makes the signature reproducible; chaining on
    prev_signature makes any reordering or deletion of earlier events detectable.
    """
    payload = json.dumps(event, sort_keys=True, separators=(",", ":"))
    message = f"{prev_signature}:{payload}".encode("utf-8")
    return hmac.new(secret, message, hashlib.sha256).hexdigest()


def append_event(chain: list[dict], event: dict, secret: bytes) -> dict:
    """Append a signed, chained audit event and return it."""
    prev_sig = chain[-1]["signature"] if chain else "genesis"
    signed = {**event, "prev": prev_sig,
              "signature": sign_audit_event(event, prev_sig, secret)}
    chain.append(signed)
    return signed

Verification walks the chain and recomputes each signature; a single mismatch localizes exactly where the log was altered, which is what turns “we log everything” into “we can prove the log is intact.” Those chained events also feed anomaly detection — a spend curve that deviates from its baseline and an enforcement action that fired without a matching decision are both visible in the same stream. Because the enforcement path can join back to query execution cost modeling, a compute overrun that triggered a throttle can be traced from the audit event all the way to the unbounded scan that caused it, without ever widening the reader’s permissions to do so.

Failure Modes & Troubleshooting

Security failures in a cost pipeline have distinctive signatures, and the signature is most of the fix.

  • AccessDenied on a Cost Explorer or Budgets call. The assumed role is missing an action, or the STS session was minted from a principal the trust policy does not allow. Resolution: never wrap AccessDenied in throttling retries (it will only delay the real error); diff the role’s effective policy against the exact action in the error and add only that action — resist the urge to grant ce:*.
  • InvalidCiphertextException from KMS decrypt. The EncryptionContext passed at decrypt time does not match what was supplied at encrypt time — usually a tenant key mismatch or a purpose typo. Resolution: treat the context as part of the schema, assert it on both sides, and remember a context mismatch is often a correct denial (a role reaching for another tenant’s data), not a bug to loosen.
  • ThrottlingException from STS during fan-out. The concurrency cap was raised or removed and the assume-role storm exceeded the STS rate. Resolution: keep the semaphore ceiling and the session cache — re-assuming per call is both a throttle risk and a needless widening of the exposure window; reuse the cached session until its 60-second refresh margin.
  • Expired token mid-request (ExpiredTokenException). A cached session outlived its TTL because a long batch ran past the refresh margin, or clock skew shrank the effective window. Resolution: refresh on a margin (not on expiry), pin the container clock to NTP, and make the provider idempotent so a refresh mid-batch is transparent to callers.
  • Spend appears in unattributed / a tenant with no owner. A resource shipped without a cost-allocation tag, so its cost cannot be bound to any team’s access scope; because tags backfill only from activation date, freshly tagged resources also lag up to 24 hours. Resolution: enforce tags at provision time and alert on unattributed spend as a governance violation rather than dropping it silently.
  • Billing API unreachable, reader returns zeros. If the extractor reads zeros during an outage, downstream boundaries falsely clear and audit gaps open. Resolution: degrade to the last good cached read via the fallback routing pattern for cost APIs, applying the same graceful degradation when billing APIs are down discipline the extraction layer uses — a stale-but-signed reading is safer than a fresh zero.

Back to: Cloud Database Cost Fundamentals & Architecture