Asterrr's Handbook

Secrets management

Secrets Manager versus Parameter Store SecureString, rotation with Lambda and managed rotation, single-user and alternating-user strategies, cross-account secrets, caching, and keeping secrets out of code, user data and environment variables.

Exam tasks: 5.3.1 (design management and rotation of secrets for workloads)

The decision: where does each credential live, how does it rotate without an outage, who can read it (and from which account), and how does the application fetch it without the value ending up in code, images or logs?

Where to store it

Secrets ManagerParameter Store SecureString
Built-in rotationYes: managed rotation or a Lambda function on a scheduleNo; build your own with EventBridge and Lambda
Max value size64 KB4 KB standard, 8 KB advanced tier
EncryptionAlways, with aws/secretsmanager or a customer managed keyKMS, with aws/ssm or a customer managed key
Resource-based policyYes, per secretNo; advanced parameters can be shared through AWS RAM
Cross-account readResource policy + customer managed keyRAM share of an advanced parameter + key access
Multi-RegionReplica secrets that stay in syncNo; copy per Region
VersioningStaging labels (AWSCURRENT, AWSPENDING, AWSPREVIOUS)Numbered versions and labels
CostPer secret per month, plus API callsStandard tier free
Good forDatabase, API and third-party credentials that must rotateConfig values, feature flags, low-change secrets
  • Parameter Store can reference a Secrets Manager secret (/aws/reference/secretsmanager/<name>), so tools that only read parameters can still get rotated secrets.
  • Prefer no secret at all: IAM roles for AWS APIs, IAM database authentication for RDS and Aurora, and Secrets Manager only where a password is unavoidable.

Rotation

Managed rotation

  • Services that own the credential rotate it for you, with no Lambda function: for example RDS and Aurora with managed master user passwords, Redshift admin passwords, and some partner-held secrets.
  • For RDS, set ManageMasterUserPassword and the password lives in Secrets Manager, rotating every 7 days by default. Nobody ever sees it in the console or in CloudFormation.

Rotation with a Lambda function

The function is called four times per rotation, one per step:

createSecret: generate a new value and store it as version AWSPENDING.
setSecret: change the credential in the database or service to the pending value.
testSecret: log in with the pending value to prove it works.
finishSecret: move AWSCURRENT to the new version; the old one becomes AWSPREVIOUS.
  • Schedules can be as frequent as every 4 hours, with a rotation window during which the rotation may run.
  • The function needs a network path to both the database and the Secrets Manager endpoint. In a private subnet with no NAT gateway, add an interface VPC endpoint for Secrets Manager. This is the most common cause of "rotation times out".
  • It also needs kms:Decrypt and kms:GenerateDataKey on the secret's key.
StrategyHow it worksTrade-off
Single userChanges the password of the one user the app usesBrief window where connections using the old password fail; simplest
Alternating usersTwo users (for example app and app_clone); each rotation updates the idle one and switchesNo downtime; needs a separate admin secret with rights to change the other users

Exam signal

"Rotate database credentials automatically with no downtime for a high-traffic application" means the alternating users strategy, or managed master user passwords for the master account. "Rotation fails with a timeout" in a private subnet points to a missing Secrets Manager VPC endpoint or NAT route.

Rotating the secret but not the app

Rotation only helps if the application reads the secret at run time. Applications that copy the password into a config file at deploy time break at the next rotation. Use the caching client, the Lambda extension or native integrations so the app picks up AWSCURRENT on its own.

Cross-account access

To let account B read a secret in account A:

  1. Resource policy on the secret in A allows the role in B secretsmanager:GetSecretValue.
  2. The secret is encrypted with a customer managed key, and its key policy allows the role in B kms:Decrypt (usually with kms:ViaService set to Secrets Manager in that Region).
  3. The IAM policy on the role in B allows both actions on the secret and key ARNs.
  • Secrets encrypted with aws/secretsmanager can't be read cross-account, because you can't edit that key's policy. Re-encrypt with a customer managed key first.
  • Turn on BlockPublicPolicy so a resource policy that grants broad access is rejected, and let IAM Access Analyzer flag secrets shared outside the organization.

Getting secrets into workloads safely

WorkloadRecommended way
LambdaAWS Parameters and Secrets Lambda Extension (local cache over HTTP), not environment variables
ECSsecrets in the task definition referencing a Secrets Manager or Parameter Store ARN; injected at start, the task execution role needs access
EKSSecrets Store CSI Driver with the AWS provider (ASCP), with EKS Pod Identity or IRSA for access
EC2Instance role plus SDK or Secrets Manager Agent; never in user data
CloudFormationDynamic references (resolve:secretsmanager:...) or ManageMasterUserPassword, never plaintext parameters
Any appClient-side caching libraries to cut latency, cost and throttling

Places secrets must never be:

  • Source code and repositories. Run secret scanning (Amazon Inspector code security, or pre-commit hooks such as git-secrets), and treat anything already committed as compromised: rotate it.
  • EC2 user data. Anyone with ec2:DescribeInstanceAttribute can read it, and it's visible from inside the instance through the metadata service.
  • Lambda environment variables. With the default key, anyone allowed lambda:GetFunctionConfiguration sees them in plaintext, and they can't rotate. Fetch the secret at run time instead.
  • Container images and AMIs. Anyone who pulls them gets the secret.
  • Logs. Mask them with CloudWatch Logs data protection.
64 KB
Largest Secrets Manager secret value.
4 KB / 8 KB
Parameter Store standard and advanced parameter size limits.
4 hours
Most frequent rotation schedule in Secrets Manager.
7 days
Default rotation for RDS managed master user passwords.

Scenarios

Scenario
A ticketing platform stores its Aurora PostgreSQL application password in Secrets Manager with a rotation Lambda function in a private subnet. After enabling rotation, every attempt fails with a timeout, and the secret still has only an AWSCURRENT version. The subnet has no NAT gateway. What should the security engineer do?
Scenario
An analytics account needs to read a partner API key stored as a Secrets Manager secret in a shared-services account. The secret uses the default aws/secretsmanager key. The engineer added a resource policy on the secret allowing the analytics role, and an IAM policy on the role, but GetSecretValue still fails. What must the engineer also do?
Scenario · choose 2
A development team passes a database password to EC2 instances through user data and to Lambda functions as a KMS-encrypted environment variable. A security review requires that the password rotate every 30 days and that no one with read-only console access can see it. Which TWO changes meet the requirements?

Further reading

On this page