← Back to the library
OperationsApplied · 8 min

The cloud bill suddenly spikes

“Finance forwards an invoice two or three times larger than last month. Traffic looks normal, and no single service obviously changed.”

THE PRINCIPLE

Break cost down by usage type, not by service, and alert on the rate of change. A daily anomaly alert catches in a day what the invoice shows you a month later.

FIRST MOVES

  1. In Cost Explorer or your provider's equivalent, group daily cost by usage type and find the first day the line moved. Line it up with deploys and config changes from that day.
  2. Check NAT gateway bytes: traffic to S3, ECR, or DynamoDB through NAT pays per GB processed. Gateway or interface VPC endpoints and same-AZ placement usually remove most of it.
  3. Check log and metric ingestion: debug logging left on, a log line per request in a hot loop, or high-cardinality metrics labels.
  4. Check capacity that never shrank: an HPA or ASG pinned at max, orphaned volumes and snapshots, a retrying job or a recursive function trigger.
  5. Add guardrails: AWS Budgets with forecast alerts, Cost Anomaly Detection to email or Slack, cost-allocation tags per team, and max replica caps on autoscalers.

TOOLS: THEN → NOW

Reading the monthly invoiceDaily cost by usage type, with budgets and anomaly detection alerting within about a day
Private subnets sending all AWS traffic through NATVPC gateway endpoints for S3 and DynamoDB, and interface endpoints for heavy services

PATTERN SNAPSHOT

# Daily cost by usage type for the last 14 days: the spike's line item stands out.
# (GNU date syntax; each Cost Explorer API request is billed.)
aws ce get-cost-and-usage \
  --time-period Start=$(date -u -d '14 days ago' +%F),End=$(date -u +%F) \
  --granularity DAILY --metrics UnblendedCost \
  --group-by Type=DIMENSION,Key=USAGE_TYPE \
  --query 'ResultsByTime[].[TimePeriod.Start, Groups[].[Keys[0], Metrics.UnblendedCost.Amount]]' \
  --output json

# Typical culprits in USAGE_TYPE:
#   *NatGateway-Bytes             NAT data processing
#   *DataTransfer-Regional-Bytes  cross-AZ traffic
#   *DataProcessing-Bytes         CloudWatch Logs ingestion

CLOSE THE AI. EXPLAIN THIS.

Your private-subnet workers pull container images and S3 objects all day. Which line item grows, and what change removes most of it?

SOURCES

Guide reviewed