AWS IAM and Security: Hardening Runbook
AWS IAM and Security: Hardening Runbook
The reference sheet turned into work. Twelve phases take an organization from root lockdown through Organizations guardrails, Identity Center, account baselines, workload roles, Cognito, Directory Service, KMS, the WAF and Shield edge and RAM sharing, to the recurring reviews that keep it clean. Every step is one decision or one command block; tick as you go. Four incident playbooks and an AccessDenied triage table close the sheet. Concepts and diagrams live in bubim-awsiam.
Prepare: decisions and inventory
0 / 0Paper first. These answers feed every later command; the variables set here are used in the code blocks below.
export AWS_REGION=eu-central-1 export MGMT=111111111111 SECURITY=222222222222 LOGS=333333333333 NETWORK=444444444444 WORKLOAD=555555555555 export ORG_ID=o-a1b2c3d4e5
aws iam generate-credential-report >/dev/null; sleep 5 aws iam get-credential-report --query Content --output text | base64 -d > cred-report-$(date +%F).csv aws iam list-users --query 'Users[].[UserName,PasswordLastUsed]' --output table aws iam list-roles --query 'Roles[].[RoleName,Path]' --output table # /aws-service-role/ = service-linked
Root user lockdown
0 / 0The root user ignores IAM and SCP limits in its own account. Use it only for billing and the root-only tasks, and in member accounts remove it altogether in step 2.4.
aws iam get-account-summary --query 'SummaryMap.[AccountAccessKeysPresent,AccountMFAEnabled]' # expect [0, 1]; delete any root key in the console under Security credentials
for T in SECURITY OPERATIONS BILLING; do
aws account put-alternate-contact --alternate-contact-type $T \
--name "Cloud $T" --title "Team mailbox" \
--email-address "aws-$(echo $T | tr A-Z a-z)@example.com" --phone-number "+359000000000"
doneaws iam create-account-alias --account-alias example-mgmt
aws sns create-topic --name root-activity
aws events put-rule --name root-activity --event-pattern \
'{"detail-type":["AWS Console Sign In via CloudTrail","AWS API Call via CloudTrail"],"detail":{"userIdentity":{"type":["Root"]}}}'
aws events put-targets --rule root-activity \
--targets Id=sns,Arn=arn:aws:sns:$AWS_REGION:$MGMT:root-activityOrganizations and guardrails
0 / 0Run from the management account. Attach every new SCP or RCP to the Sandbox OU first, watch for a day, then widen.
aws organizations create-organization --feature-set ALL export ROOT_ID=$(aws organizations list-roots --query 'Roots[0].Id' --output text) for T in SERVICE_CONTROL_POLICY RESOURCE_CONTROL_POLICY TAG_POLICY; do aws organizations enable-policy-type --root-id $ROOT_ID --policy-type $T done
for OU in Security Infrastructure Workloads Sandbox Suspended; do
echo "$OU $(aws organizations create-organizational-unit --parent-id $ROOT_ID --name $OU \
--query 'OrganizationalUnit.Id' --output text)"
done
aws organizations move-account --account-id $SECURITY \
--source-parent-id $ROOT_ID --destination-parent-id ou-a1b2-securityfor SP in iam.amazonaws.com access-analyzer.amazonaws.com cloudtrail.amazonaws.com; do aws organizations enable-aws-service-access --service-principal $SP done for SP in iam.amazonaws.com access-analyzer.amazonaws.com; do aws organizations register-delegated-administrator --service-principal $SP --account-id $SECURITY done
aws iam enable-organizations-root-credentials-management aws iam enable-organizations-root-sessions CREDS=$(aws sts assume-root --region $AWS_REGION --target-principal $LOGS \ --task-policy-arn arn=arn:aws:iam::aws:policy/root-task/IAMDeleteRootUserCredentials \ --duration-seconds 900 --query Credentials --output json) export AWS_ACCESS_KEY_ID=$(echo "$CREDS" | jq -r .AccessKeyId) export AWS_SECRET_ACCESS_KEY=$(echo "$CREDS" | jq -r .SecretAccessKey) export AWS_SESSION_TOKEN=$(echo "$CREDS" | jq -r .SessionToken) aws iam delete-login-profile # root password aws iam list-access-keys # then delete-access-key --access-key-id ... aws iam list-mfa-devices # then deactivate-mfa-device --serial-number ... unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN
{
"Version": "2012-10-17",
"Statement": [
{"Sid": "StayInOrg", "Effect": "Deny", "Action": "organizations:LeaveOrganization", "Resource": "*"},
{"Sid": "KeepSecurityOn", "Effect": "Deny",
"Action": ["cloudtrail:StopLogging", "cloudtrail:DeleteTrail", "cloudtrail:UpdateTrail",
"guardduty:DeleteDetector", "guardduty:DisassociateFromAdministratorAccount",
"config:StopConfigurationRecorder", "config:DeleteConfigurationRecorder", "config:DeleteDeliveryChannel",
"securityhub:DisableSecurityHub", "access-analyzer:DeleteAnalyzer"],
"Resource": "*",
"Condition": {"ArnNotLike": {"aws:PrincipalArn": "arn:aws:iam::*:role/SecurityBreakGlass"}}},
{"Sid": "NoRootUser", "Effect": "Deny", "Action": "*", "Resource": "*",
"Condition": {"StringLike": {"aws:PrincipalArn": "arn:aws:iam::*:root"}}},
{"Sid": "ImdsV2Only", "Effect": "Deny", "Action": "ec2:RunInstances", "Resource": "arn:aws:ec2:*:*:instance/*",
"Condition": {"StringNotEquals": {"ec2:MetadataHttpTokens": "required"}}}
]
}SCP=$(aws organizations create-policy --type SERVICE_CONTROL_POLICY --name baseline-guardrails \ --description "Org baseline" --content file://scp-baseline.json \ --query 'Policy.PolicySummary.Id' --output text) aws organizations attach-policy --policy-id $SCP --target-id ou-a1b2-sandbox # then $ROOT_ID
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "OrgPrincipalsOnly",
"Effect": "Deny",
"Principal": "*",
"Action": ["s3:*", "sqs:*", "kms:*", "secretsmanager:*"],
"Resource": "*",
"Condition": {
"StringNotEqualsIfExists": {"aws:PrincipalOrgID": "o-a1b2c3d4e5"},
"BoolIfExists": {"aws:PrincipalIsAWSService": "false"}
}
}]
}RCP=$(aws organizations create-policy --type RESOURCE_CONTROL_POLICY --name data-perimeter \ --description "Org principals only" --content file://rcp-perimeter.json \ --query 'Policy.PolicySummary.Id' --output text) aws organizations attach-policy --policy-id $RCP --target-id ou-a1b2-sandbox
aws cloudtrail create-trail --name org-trail --s3-bucket-name org-trail-$LOGS \ --is-organization-trail --is-multi-region-trail --enable-log-file-validation aws cloudtrail start-logging --name org-trail
Workforce access with IAM Identity Center
0 / 0People get access through groups, permission sets and account assignments. Nobody needs an IAM user after this phase.
export INST=$(aws sso-admin list-instances --query 'Instances[0].InstanceArn' --output text) export IDS=$(aws sso-admin list-instances --query 'Instances[0].IdentityStoreId' --output text)
aws organizations register-delegated-administrator --service-principal sso.amazonaws.com --account-id $SECURITY
PS=$(aws sso-admin create-permission-set --instance-arn $INST --name ReadOnly \ --session-duration PT8H --query 'PermissionSet.PermissionSetArn' --output text) aws sso-admin attach-managed-policy-to-permission-set --instance-arn $INST \ --permission-set-arn $PS --managed-policy-arn arn:aws:iam::aws:policy/ReadOnlyAccess
GID=$(aws identitystore get-group-id --identity-store-id $IDS \
--alternate-identifier '{"UniqueAttribute":{"AttributePath":"displayName","AttributeValue":"aws-readonly"}}' \
--query GroupId --output text)
aws sso-admin create-account-assignment --instance-arn $INST --permission-set-arn $PS \
--target-type AWS_ACCOUNT --target-id $WORKLOAD --principal-type GROUP --principal-id $GIDaws configure sso # session name corp, start URL, home Region, scope sso:account:access aws sso login --sso-session corp aws sts get-caller-identity --profile dev-admin
aws iam delete-login-profile --user-name alice aws iam update-access-key --user-name alice --access-key-id AKIAEXAMPLEKEY01 --status Inactive
Account baseline
0 / 0Run in every account, or bake it into the account vending pipeline.
aws iam update-account-password-policy --minimum-password-length 14 \ --require-uppercase-characters --require-lowercase-characters --require-numbers --require-symbols \ --allow-users-to-change-password --password-reuse-prevention 24
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "DenyAllWithoutMFA",
"Effect": "Deny",
"NotAction": ["iam:CreateVirtualMFADevice", "iam:EnableMFADevice", "iam:ListMFADevices",
"iam:ListVirtualMFADevices", "iam:ResyncMFADevice", "iam:ChangePassword",
"iam:GetUser", "sts:GetSessionToken"],
"Resource": "*",
"Condition": {"BoolIfExists": {"aws:MultiFactorAuthPresent": "false"}}
}]
}aws accessanalyzer create-analyzer --analyzer-name org-external --type ORGANIZATION
aws accessanalyzer create-analyzer --analyzer-name org-unused --type ORGANIZATION_UNUSED_ACCESS \
--configuration '{"unusedAccess":{"unusedAccessAge":90}}'aws accessanalyzer validate-policy --policy-type IDENTITY_POLICY --policy-document file://policy.json \ --query 'findings[].[findingType,issueCode]' --output table
{
"Version": "2012-10-17",
"Statement": [
{"Sid": "CreateRolesOnlyWithBoundary", "Effect": "Allow",
"Action": ["iam:CreateRole", "iam:PutRolePermissionsBoundary", "iam:AttachRolePolicy", "iam:PutRolePolicy"],
"Resource": "arn:aws:iam::555555555555:role/app-*",
"Condition": {"StringEquals": {"iam:PermissionsBoundary": "arn:aws:iam::555555555555:policy/DevBoundary"}}},
{"Sid": "PassOnlyAppRoles", "Effect": "Allow", "Action": "iam:PassRole",
"Resource": "arn:aws:iam::555555555555:role/app-*"},
{"Sid": "BoundaryIsFixed", "Effect": "Deny",
"Action": ["iam:DeleteRolePermissionsBoundary", "iam:CreatePolicyVersion", "iam:DeletePolicy",
"iam:DeletePolicyVersion", "iam:SetDefaultPolicyVersion"],
"Resource": ["arn:aws:iam::555555555555:role/*", "arn:aws:iam::555555555555:policy/DevBoundary"]}
]
}aws ec2 modify-instance-metadata-defaults --http-tokens required --http-put-response-hop-limit 2
Roles for workloads and cross-account access
0 / 0No access keys on instances, in containers or in pipelines. One role per workload, scoped to what it touches.
cat > trust-ec2.json <<'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF
aws iam create-role --role-name app-reports-ec2 --assume-role-policy-document file://trust-ec2.json \
--permissions-boundary arn:aws:iam::$WORKLOAD:policy/DevBoundary
aws iam attach-role-policy --role-name app-reports-ec2 --policy-arn arn:aws:iam::$WORKLOAD:policy/reports-read
aws iam create-instance-profile --instance-profile-name app-reports-ec2
aws iam add-role-to-instance-profile --instance-profile-name app-reports-ec2 --role-name app-reports-ec2
aws ec2 associate-iam-instance-profile --instance-id i-0abc1234def567890 --iam-instance-profile Name=app-reports-ec2aws iam create-open-id-connect-provider --url https://token.actions.githubusercontent.com \ --client-id-list sts.amazonaws.com
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"Federated": "arn:aws:iam::555555555555:oidc-provider/token.actions.githubusercontent.com"},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {"token.actions.githubusercontent.com:aud": "sts.amazonaws.com"},
"StringLike": {"token.actions.githubusercontent.com:sub": "repo:example-org/infra:ref:refs/heads/main"}
}
}]
}aws sts assume-role --role-arn arn:aws:iam::111111111111:role/ReportsRead \ --role-session-name smoke-test --query 'Credentials.Expiration' --output text aws sts get-caller-identity --profile reports
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::999999999999:root"},
"Action": "sts:AssumeRole",
"Condition": {"StringEquals": {"sts:ExternalId": "7f3c9e1a-2b4d-4c6e-8f00-vendor"}}
}]
}Customer identity with Cognito
only with a web or mobile app0 / 0User pool for sign-in, identity pool for AWS credentials, one tightly scoped role behind it. Fig 3 of the reference shows the flow.
POOL=$(aws cognito-idp create-user-pool --pool-name app-users \
--policies 'PasswordPolicy={MinimumLength=12,RequireUppercase=true,RequireLowercase=true,RequireNumbers=true,RequireSymbols=false}' \
--username-attributes email --auto-verified-attributes email \
--query 'UserPool.Id' --output text)
aws cognito-idp set-user-pool-mfa-config --user-pool-id $POOL \
--software-token-mfa-configuration Enabled=true --mfa-configuration OPTIONALCLIENT=$(aws cognito-idp create-user-pool-client --user-pool-id $POOL --client-name web \ --no-generate-secret --explicit-auth-flows ALLOW_USER_SRP_AUTH ALLOW_REFRESH_TOKEN_AUTH \ --query 'UserPoolClient.ClientId' --output text)
IDP=$(aws cognito-identity create-identity-pool --identity-pool-name app_identities \ --no-allow-unauthenticated-identities \ --cognito-identity-providers ProviderName=cognito-idp.$AWS_REGION.amazonaws.com/$POOL,ClientId=$CLIENT \ --query IdentityPoolId --output text) aws cognito-identity set-identity-pool-roles --identity-pool-id $IDP \ --roles authenticated=arn:aws:iam::$WORKLOAD:role/app-cognito-auth
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"Federated": "cognito-identity.amazonaws.com"},
"Action": ["sts:AssumeRoleWithWebIdentity", "sts:TagSession"],
"Condition": {
"StringEquals": {"cognito-identity.amazonaws.com:aud": "eu-central-1:00000000-0000-0000-0000-000000000000"},
"ForAnyValue:StringLike": {"cognito-identity.amazonaws.com:amr": "authenticated"}
}
}]
}{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:PutObject"],
"Resource": "arn:aws:s3:::app-uploads/${cognito-identity.amazonaws.com:sub}/*"
}]
}Directory Service
only with AD-dependent workloads0 / 0Choose with fig 7 of the reference, then do either the Managed AD steps or the AD Connector step. Passwords come from the vault through environment variables, never typed into shell history.
aws ds create-microsoft-ad --name corp.example.com --short-name CORP --edition Standard \
--password "$AD_ADMIN_PASSWORD" \
--vpc-settings '{"VpcId":"vpc-0abc1234","SubnetIds":["subnet-0aaa1111","subnet-0bbb2222"]}'aws ds create-trust --directory-id d-1234567890 --remote-domain-name onprem.example.com \ --trust-password "$TRUST_PASSWORD" --trust-direction "Two-Way" --trust-type Forest \ --conditional-forwarder-ip-addrs 10.10.0.10 10.10.0.11
aws ds connect-directory --name onprem.example.com --short-name ONPREM --size Small \
--password "$CONNECTOR_SVC_PASSWORD" \
--connect-settings '{"VpcId":"vpc-0abc1234","SubnetIds":["subnet-0aaa1111","subnet-0bbb2222"],"CustomerDnsIps":["10.10.0.10","10.10.0.11"],"CustomerUserName":"svc-adconnector"}'aws ssm create-association --name AWS-JoinDirectoryServiceDomain \
--parameters '{"directoryId":["d-1234567890"],"directoryName":["corp.example.com"],"dnsIpAddresses":["10.0.1.10","10.0.2.10"]}' \
--targets Key=tag:DomainJoin,Values=trueKeys with KMS, and CloudHSM where mandated
0 / 0One customer managed key per data domain, administrators separated from users, rotation on, deletion guarded.
{
"Version": "2012-10-17",
"Statement": [
{"Sid": "AccountDelegation", "Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::555555555555:root"}, "Action": "kms:*", "Resource": "*"},
{"Sid": "KeyAdmins", "Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::555555555555:role/KeyAdmin"},
"Action": ["kms:Create*", "kms:Describe*", "kms:Enable*", "kms:List*", "kms:Put*", "kms:Update*",
"kms:Disable*", "kms:Get*", "kms:TagResource", "kms:UntagResource", "kms:RotateKeyOnDemand",
"kms:ScheduleKeyDeletion", "kms:CancelKeyDeletion"],
"Resource": "*"},
{"Sid": "KeyUsers", "Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::555555555555:role/app-reports-ec2"},
"Action": ["kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey"],
"Resource": "*"}
]
}KEY=$(aws kms create-key --description "reports data" --key-usage ENCRYPT_DECRYPT \ --key-spec SYMMETRIC_DEFAULT --policy file://key-policy.json --query KeyMetadata.KeyId --output text) aws kms create-alias --alias-name alias/reports --target-key-id $KEY
aws kms enable-key-rotation --key-id $KEY --rotation-period-in-days 365 aws kms get-key-rotation-status --key-id $KEY
{"Sid": "NoKeyDeletion", "Effect": "Deny",
"Action": ["kms:ScheduleKeyDeletion", "kms:DeleteImportedKeyMaterial"], "Resource": "*",
"Condition": {"ArnNotLike": {"aws:PrincipalArn": "arn:aws:iam::*:role/SecurityBreakGlass"}}}aws cloudhsmv2 create-cluster --hsm-type hsm2m.medium --subnet-ids subnet-0aaa1111 subnet-0bbb2222 aws cloudhsmv2 create-hsm --cluster-id cluster-abc123 --availability-zone eu-central-1a aws cloudhsmv2 create-hsm --cluster-id cluster-abc123 --availability-zone eu-central-1b aws cloudhsmv2 describe-clusters --filters clusterIds=cluster-abc123 \ --query 'Clusters[0].Certificates.ClusterCsr' --output text > cluster.csr # sign cluster.csr with your issuing CA, then aws cloudhsmv2 initialize-cluster --cluster-id cluster-abc123 \ --signed-cert file://cluster.crt --trust-anchor file://ca.crt
Edge: WAF, Shield and Firewall Manager
0 / 0Managed rules in Count mode first, look at what they would block, then enforce. CloudFront web ACLs use scope CLOUDFRONT in us-east-1.
[
{"Name": "aws-common", "Priority": 0,
"Statement": {"ManagedRuleGroupStatement": {"VendorName": "AWS", "Name": "AWSManagedRulesCommonRuleSet"}},
"OverrideAction": {"Count": {}},
"VisibilityConfig": {"SampledRequestsEnabled": true, "CloudWatchMetricsEnabled": true, "MetricName": "aws-common"}},
{"Name": "aws-bad-inputs", "Priority": 1,
"Statement": {"ManagedRuleGroupStatement": {"VendorName": "AWS", "Name": "AWSManagedRulesKnownBadInputsRuleSet"}},
"OverrideAction": {"Count": {}},
"VisibilityConfig": {"SampledRequestsEnabled": true, "CloudWatchMetricsEnabled": true, "MetricName": "aws-bad-inputs"}},
{"Name": "aws-sqli", "Priority": 2,
"Statement": {"ManagedRuleGroupStatement": {"VendorName": "AWS", "Name": "AWSManagedRulesSQLiRuleSet"}},
"OverrideAction": {"Count": {}},
"VisibilityConfig": {"SampledRequestsEnabled": true, "CloudWatchMetricsEnabled": true, "MetricName": "aws-sqli"}},
{"Name": "aws-ip-reputation", "Priority": 3,
"Statement": {"ManagedRuleGroupStatement": {"VendorName": "AWS", "Name": "AWSManagedRulesAmazonIpReputationList"}},
"OverrideAction": {"Count": {}},
"VisibilityConfig": {"SampledRequestsEnabled": true, "CloudWatchMetricsEnabled": true, "MetricName": "aws-ip-reputation"}},
{"Name": "rate-limit", "Priority": 4,
"Statement": {"RateBasedStatement": {"Limit": 2000, "AggregateKeyType": "IP"}},
"Action": {"Block": {}},
"VisibilityConfig": {"SampledRequestsEnabled": true, "CloudWatchMetricsEnabled": true, "MetricName": "rate-limit"}}
]ACL_ARN=$(aws wafv2 create-web-acl --name app-acl --scope REGIONAL --default-action Allow={} \
--rules file://waf-rules.json \
--visibility-config SampledRequestsEnabled=true,CloudWatchMetricsEnabled=true,MetricName=app-acl \
--query Summary.ARN --output text)
aws logs create-log-group --log-group-name aws-waf-logs-app-acl
aws wafv2 put-logging-configuration --logging-configuration \
ResourceArn=$ACL_ARN,LogDestinationConfigs=arn:aws:logs:$AWS_REGION:$WORKLOAD:log-group:aws-waf-logs-app-aclaws wafv2 associate-web-acl --web-acl-arn $ACL_ARN \ --resource-arn arn:aws:elasticloadbalancing:$AWS_REGION:$WORKLOAD:loadbalancer/app/web/0123456789abcdef
ACL_ID=${ACL_ARN##*/}
LOCK=$(aws wafv2 get-web-acl --name app-acl --scope REGIONAL --id $ACL_ID --query LockToken --output text)
aws wafv2 update-web-acl --name app-acl --scope REGIONAL --id $ACL_ID --lock-token $LOCK \
--default-action Allow={} --rules file://waf-rules-enforce.json \
--visibility-config SampledRequestsEnabled=true,CloudWatchMetricsEnabled=true,MetricName=app-aclaws shield create-subscription --region us-east-1 aws shield create-protection --region us-east-1 --name web-alb \ --resource-arn arn:aws:elasticloadbalancing:$AWS_REGION:$WORKLOAD:loadbalancer/app/web/0123456789abcdef aws shield associate-drt-role --region us-east-1 --role-arn arn:aws:iam::$WORKLOAD:role/ShieldResponseTeam
aws fms associate-admin-account --region us-east-1 --admin-account $SECURITY
Sharing with RAM
0 / 0Central network account owns the VPCs; workload accounts launch into shared subnets.
aws ram enable-sharing-with-aws-organization
aws ram create-resource-share --name shared-subnets \ --resource-arns arn:aws:ec2:$AWS_REGION:$NETWORK:subnet/subnet-0aaa1111 arn:aws:ec2:$AWS_REGION:$NETWORK:subnet/subnet-0bbb2222 \ --principals arn:aws:organizations::$MGMT:ou/$ORG_ID/ou-a1b2-workload \ --no-allow-external-principals
aws ram get-resource-shares --resource-owner OTHER-ACCOUNTS --query 'resourceShares[].[name,status]' --output table aws ec2 describe-subnets --filters Name=owner-id,Values=$NETWORK --query 'Subnets[].[SubnetId,AvailabilityZone]' --output table
Operate: detection and recurring reviews
0 / 0Prevention drifts. These are the checks that catch it; put them in a calendar.
aws guardduty enable-organization-admin-account --admin-account-id $SECURITY aws securityhub enable-organization-admin-account --admin-account-id $SECURITY
aws iam generate-credential-report >/dev/null; sleep 5
aws iam get-credential-report --query Content --output text | base64 -d | \
awk -F, 'NR>1 && $9=="true" && $11=="N/A" {print $1": access key 1 active but never used"}'
aws accessanalyzer list-findings-v2 --analyzer-arn $UNUSED_ARN --filter '{"status":{"eq":["ACTIVE"]}}' \
--query 'findings[].[resource,findingType]' --output tableaws iam create-access-key --user-name ci-legacy # deploy the new key, confirm it is in use aws iam get-access-key-last-used --access-key-id AKIANEWKEYEXAMPLE1 aws iam update-access-key --user-name ci-legacy --access-key-id AKIAOLDKEYEXAMPLE1 --status Inactive aws iam delete-access-key --user-name ci-legacy --access-key-id AKIAOLDKEYEXAMPLE1
aws organizations list-policies-for-target --target-id ou-a1b2-workload --filter SERVICE_CONTROL_POLICY aws sso-admin list-account-assignments --instance-arn $INST --account-id $WORKLOAD --permission-set-arn $PS aws iam generate-service-last-accessed-details --arn arn:aws:iam::$WORKLOAD:role/app-reports-ec2
Playbooks
For the day something goes wrong. Ticks here do not count toward the runbook progress.
PB1 Leaked access key
when a key shows up in a repository, a paste site, an AWS Health or GuardDuty notice, or unexpected API calls
aws iam update-access-key --user-name ci-legacy --access-key-id AKIALEAKEDEXAMPLE --status Inactive
aws cloudtrail lookup-events --lookup-attributes AttributeKey=AccessKeyId,AttributeValue=AKIALEAKEDEXAMPLE \ --max-results 50 --query 'Events[].[EventTime,EventName,EventSource]' --output table
aws iam put-user-policy --user-name ci-legacy --policy-name revoke-older-sessions --policy-document \
'{"Version":"2012-10-17","Statement":[{"Effect":"Deny","Action":"*","Resource":"*","Condition":{"DateLessThan":{"aws:TokenIssueTime":"2026-10-19T12:00:00Z"}}}]}'PB2 A bucket policy locks everyone out
when a bucket in a member account denies all principals, including administrators
CREDS=$(aws sts assume-root --region $AWS_REGION --target-principal $WORKLOAD \ --task-policy-arn arn=arn:aws:iam::aws:policy/root-task/S3UnlockBucketPolicy \ --query Credentials --output json) # export the three values as in step 2.4
aws s3api delete-bucket-policy --bucket reports unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN
PB3 Identity provider down or admins locked out
when nobody can reach the access portal, or the last admin permission set was removed
PB4 Onboard a new account
when a team needs a new environment
aws organizations create-account --email [email protected] --account-name reports-prod aws organizations list-create-account-status --states SUCCEEDED --query 'CreateAccountStatuses[].[AccountName,AccountId]' --output table aws organizations move-account --account-id 666666666666 --source-parent-id $ROOT_ID --destination-parent-id ou-a1b2-workload
aws organizations list-policies-for-target --target-id 666666666666 --filter SERVICE_CONTROL_POLICY aws organizations list-parents --child-id 666666666666
AccessDenied triage
CloudTrail and most error messages now name the policy type that denied the call. Walk the evaluation order of fig 1 in the reference: deny, RCP, SCP, resource, identity, boundary, session.
| Symptom | Layer | Fix | Step |
|---|---|---|---|
| “explicit deny in a service control policy” | SCP | list the SCPs on the account and every parent OU; look for Region deny and the baseline statements | 2.5, 2.6 |
| “explicit deny in a resource control policy” | RCP | caller outside the org, or a service call without aws:PrincipalIsAWSService; add a scoped exception | 2.7 |
| “no identity-based policy allows the action” | identity | add the Allow to the role or permission set; check the exact action and resource ARN | 3.4, 5.1 |
| “no resource-based policy allows” across accounts | resource | cross-account needs both sides: the bucket, key or queue policy must name the caller too | 5.4 |
| “explicit deny in a permissions boundary” or boundary has no Allow | boundary | the action is outside DevBoundary; widen the boundary deliberately or use another role | 4.5 |
| “…because no session policy allows” | session | the caller passed a session policy, or the permission set has an inline scope-down | 3.4 |
| S3 GetObject denied on an SSE-KMS object | KMS | the role can read the bucket but not use the key: add it to KeyUsers in the key policy | 8.1 |
| AssumeRole denied | trust | trust policy principal, missing ExternalId or MFA, or the caller lacks sts:AssumeRole on the role ARN | 5.4, 5.5 |
| iam:PassRole denied when launching EC2, Lambda or ECS | identity | allow iam:PassRole on that role ARN, scoped with iam:PassedToService | 4.5 |
| Works in one Region, denied in another | SCP | Region deny; global services must be in the NotAction list | 2.6 |
| Identity Center user sees no accounts | assignment | no assignment for the user’s group, or SCIM has not provisioned the user or group yet | 3.3, 3.5 |
| Identity pool returns NotAuthorized or InvalidIdentityPoolConfiguration | trust | role trust aud does not match the pool ID, or the amr condition says unauthenticated | 6.4 |
| Container on EC2 gets no role credentials | IMDS | IMDSv2 hop limit is 1; set it to 2 | 4.6 |
| WAF blocks legitimate users | WAF | read sampled requests and logs, set the single rule to Count, add a scope-down statement | 9.4 |
# simulate before guessing: which statement allows or denies, for a given principal and action
aws iam simulate-principal-policy --policy-source-arn arn:aws:iam::$WORKLOAD:role/app-reports-ec2 \
--action-names s3:GetObject --resource-arns arn:aws:s3:::reports/2026/q3.csv \
--query 'EvaluationResults[].[EvalActionName,EvalDecision]' --output tableLimits of the simulator Resource policies count only when you pass them with –resource-policy, and key policies and controls outside the account are not evaluated. A clean simulation next to a real AccessDenied points there.
0 comments