AWS Systems Manager: Hardening Runbook
Runbook · AWS · Systems Manager
AWS Systems Manager: Hardening Runbook
Fifteen phases take one account and Region from a default Systems Manager setup to a hardened one: DHMC and IMDSv2 for node identity, private endpoints, a KMS key and encrypted log destinations, an OS baseline for ssm-user, hardened session preferences, least-privilege operator access and just-in-time approval, safe documents, patching, Parameter Store, organization guardrails, detection and hybrid nodes, then a verification pass. Tick as you go. Concepts, the nine diagrams and the control matrix live in bubim-awsssm.
aws ssm get-service-setting cpx · bub.im
0 / 0
P0
Prepare
0 / 0Run as an administrator of the workload account. JSON files below use account 111122223333 and Region eu-central-1; step 0.3 swaps in your values.
0.1 Check the CLI and the Session Manager pluginThe plugin is needed for every start-session call; verify its package signature where your policy requires it.
aws --version session-manager-plugin --version jq --version
0.2 Export the account, Region, VPC and private subnets
export AWS_REGION=eu-central-1 export ACCOUNT=$(aws sts get-caller-identity --query Account --output text) export ORG_ID=$(aws organizations describe-organization --query Organization.Id --output text) export VPC=REPLACE_WITH_VPC_ID export PRIV_SUBNETS="REPLACE_WITH_SUBNET_A REPLACE_WITH_SUBNET_B" export PRIV_RT=REPLACE_WITH_PRIVATE_ROUTE_TABLE_ID export VPC_CIDR=$(aws ec2 describe-vpcs --vpc-ids $VPC --query 'Vpcs[0].CidrBlock' --output text) export LOG_BUCKET=ssm-session-logs-$ACCOUNT-$AWS_REGION echo $ACCOUNT $ORG_ID $VPC $VPC_CIDR
0.3 Work in one folder; fill placeholders in saved JSON filesSave every JSON block from this runbook into the folder under the name shown above it, then rerun the sed line. On macOS use sed -i ”.
mkdir -p ~/ssm-hardening && cd ~/ssm-hardening sed -i "s/111122223333/$ACCOUNT/g; s/eu-central-1/$AWS_REGION/g; s/o-REPLACE_WITH_ORG_ID/$ORG_ID/g" ./*.json
0.4 Record the starting pointManaged nodes, agent versions, instance profiles and IMDS settings before anything changes.
aws ssm describe-instance-information \ --query 'InstanceInformationList[].[InstanceId,PingStatus,AgentVersion,PlatformName,ResourceType]' --output table aws ec2 describe-instances --filters Name=instance-state-name,Values=running \ --query 'Reservations[].Instances[].[InstanceId,IamInstanceProfile.Arn,MetadataOptions.HttpTokens]' --output table
P1
Node identity: IMDSv2 and DHMC
0 / 0One default role for every instance in the Region, and no instance left on IMDSv1. Skip 1.3 to 1.5 if you keep instance profiles, but still do 1.6.
1.1 Make IMDSv2 the account default for new instances
aws ec2 modify-instance-metadata-defaults --http-tokens required --http-put-response-hop-limit 2 aws ec2 get-instance-metadata-defaults
1.2 Move running instances to IMDSv2Check the MetadataNoToken CloudWatch metric first: anything still above zero uses IMDSv1 and breaks.
for i in $(aws ec2 describe-instances \
--filters Name=metadata-options.http-tokens,Values=optional Name=instance-state-name,Values=running \
--query 'Reservations[].Instances[].InstanceId' --output text); do
aws ec2 modify-instance-metadata-options --instance-id "$i" --http-tokens required --http-endpoint enabled
done1.3 Create the DHMC role
trust-dhmc.json
{
"Version": "2012-10-17",
"Statement": [{"Effect": "Allow", "Principal": {"Service": "ssm.amazonaws.com"}, "Action": "sts:AssumeRole"}]
}aws iam create-role --role-name AWSSystemsManagerDefaultEC2InstanceManagementRole --path /service-role/ \ --assume-role-policy-document file://trust-dhmc.json aws iam attach-role-policy --role-name AWSSystemsManagerDefaultEC2InstanceManagementRole \ --policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedEC2InstanceDefaultPolicy
1.4 Turn on DHMC for the RegionNeeds agent 3.2.582.0 or later on the instances; up to 30 minutes before they use the role.
DHMC=arn:aws:ssm:$AWS_REGION:$ACCOUNT:servicesetting/ssm/managed-instance/default-ec2-instance-management-role aws ssm update-service-setting --setting-id $DHMC \ --setting-value service-role/AWSSystemsManagerDefaultEC2InstanceManagementRole aws ssm get-service-setting --setting-id $DHMC --query 'ServiceSetting.[SettingValue,Status]'
1.5 Replace the deprecated AmazonEC2RoleforSSMIt grants wide S3 access. Where a profile stays, use AmazonSSMManagedInstanceCore; where DHMC should take over, remove SSM rights from the profile entirely, because a profile that allows UpdateInstanceInformation wins.
OLD=arn:aws:iam::aws:policy/service-role/AmazonEC2RoleforSSM for r in $(aws iam list-entities-for-policy --policy-arn $OLD --query 'PolicyRoles[].RoleName' --output text); do aws iam attach-role-policy --role-name "$r" --policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore aws iam detach-role-policy --role-name "$r" --policy-arn $OLD echo "fixed $r" done
1.6 Confirm every running instance is managed and OnlineOn a node that is missing, run the diagnostics as root; it checks credentials, endpoints and DNS.
aws ssm describe-instance-information --query 'InstanceInformationList[].[InstanceId,PingStatus,AgentVersion]' --output table # on a missing node sudo ssm-cli get-diagnostics --output table
P2
Private endpoints
0 / 0Private subnets reach Systems Manager without NAT. The VPC needs DNS support and DNS hostnames turned on.
2.1 Endpoint security group: 443 from the VPC only
export EP_SG=$(aws ec2 create-security-group --group-name ssm-endpoints \ --description "Systems Manager interface endpoints" --vpc-id $VPC --query GroupId --output text) aws ec2 authorize-security-group-ingress --group-id $EP_SG --protocol tcp --port 443 --cidr $VPC_CIDR
2.2 Endpoint policy: principals of your organization only
endpoint-policy.json
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "OrgPrincipalsOnly", "Effect": "Allow", "Principal": "*", "Action": "*", "Resource": "*",
"Condition": {"StringEquals": {"aws:PrincipalOrgID": "o-REPLACE_WITH_ORG_ID"}}
}]
}2.3 Create the interface endpointsssm and ssmmessages always; kms and logs because phase 5 uses them. Add ec2messages only while agents older than 3.3.40.0 remain.
for svc in ssm ssmmessages kms logs; do
aws ec2 create-vpc-endpoint --vpc-id $VPC --vpc-endpoint-type Interface \
--service-name com.amazonaws.$AWS_REGION.$svc --subnet-ids $PRIV_SUBNETS \
--security-group-ids $EP_SG --private-dns-enabled --policy-document file://endpoint-policy.json \
--query 'VpcEndpoint.VpcEndpointId' --output text
done2.4 Add the S3 gateway endpointSession logs, command output, Amazon Linux repositories and agent updates. Leave its policy open to AWS-owned buckets or patching breaks.
aws ec2 create-vpc-endpoint --vpc-id $VPC --vpc-endpoint-type Gateway \ --service-name com.amazonaws.$AWS_REGION.s3 --route-table-ids $PRIV_RT
2.5 Check states and private DNSFrom a node, the ssm hostname must resolve to private addresses in your subnets.
aws ec2 describe-vpc-endpoints --filters Name=vpc-id,Values=$VPC \ --query 'VpcEndpoints[].[ServiceName,VpcEndpointType,State,PrivateDnsEnabled]' --output table # on a node getent hosts ssm.eu-central-1.amazonaws.com ssmmessages.eu-central-1.amazonaws.com
P3
Session key and log destinations
0 / 0One customer managed key encrypts session data, the log bucket and the log group. The key is also a kill switch: disable it and no new encrypted session can start.
3.1 Key policy: account administration plus CloudWatch Logs for one group
key-ssm-sessions.json
{
"Version": "2012-10-17",
"Statement": [
{"Sid": "AccountAdmin", "Effect": "Allow", "Principal": {"AWS": "arn:aws:iam::111122223333:root"},
"Action": "kms:*", "Resource": "*"},
{"Sid": "CloudWatchLogsSessionGroup", "Effect": "Allow",
"Principal": {"Service": "logs.eu-central-1.amazonaws.com"},
"Action": ["kms:Encrypt*", "kms:Decrypt*", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:Describe*"],
"Resource": "*",
"Condition": {"ArnLike": {"kms:EncryptionContext:aws:logs:arn": "arn:aws:logs:eu-central-1:111122223333:log-group:/ssm/sessions"}}}
]
}3.2 Create the key, alias and rotation; put the key ID into the JSON files
export KEY=$(aws kms create-key --description "SSM sessions and session logs" \ --policy file://key-ssm-sessions.json --query KeyMetadata.KeyId --output text) export KEY_ARN=$(aws kms describe-key --key-id $KEY --query KeyMetadata.Arn --output text) aws kms create-alias --alias-name alias/ssm-sessions --target-key-id $KEY aws kms enable-key-rotation --key-id $KEY --rotation-period-in-days 365 sed -i "s/REPLACE_WITH_SESSION_KEY_ID/$KEY/g" ./*.json
3.3 Create the session log bucketBlock Public Access, SSE-KMS with a bucket key, versioning. In us-east-1, drop the LocationConstraint.
aws s3api create-bucket --bucket $LOG_BUCKET --create-bucket-configuration LocationConstraint=$AWS_REGION
aws s3api put-public-access-block --bucket $LOG_BUCKET --public-access-block-configuration \
BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
aws s3api put-bucket-encryption --bucket $LOG_BUCKET --server-side-encryption-configuration \
"{\"Rules\":[{\"ApplyServerSideEncryptionByDefault\":{\"SSEAlgorithm\":\"aws:kms\",\"KMSMasterKeyID\":\"$KEY_ARN\"},\"BucketKeyEnabled\":true}]}"
aws s3api put-bucket-versioning --bucket $LOG_BUCKET --versioning-configuration Status=Enabled3.4 Bucket policy: TLS only
bucket-tls.json
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "TlsOnly", "Effect": "Deny", "Principal": "*", "Action": "s3:*",
"Resource": ["arn:aws:s3:::ssm-session-logs-111122223333-eu-central-1",
"arn:aws:s3:::ssm-session-logs-111122223333-eu-central-1/*"],
"Condition": {"Bool": {"aws:SecureTransport": "false"}}
}]
}aws s3api put-bucket-policy --bucket $LOG_BUCKET --policy file://bucket-tls.json
3.5 Create the encrypted log group with a retention period
aws logs create-log-group --log-group-name /ssm/sessions --kms-key-id $KEY_ARN aws logs put-retention-policy --log-group-name /ssm/sessions --retention-in-days 400
3.6 Give the node role logging and key rights, nothing moreThe DHMC default policy has no S3, Logs or KMS rights. Put the same inline policy on any instance profile role you kept, and on the hybrid role in phase 13.
node-logging.json
{
"Version": "2012-10-17",
"Statement": [
{"Sid": "SessionLogStreams", "Effect": "Allow",
"Action": ["logs:CreateLogStream", "logs:PutLogEvents", "logs:DescribeLogStreams"],
"Resource": "arn:aws:logs:eu-central-1:111122223333:log-group:/ssm/sessions:*"},
{"Sid": "FindLogGroups", "Effect": "Allow", "Action": "logs:DescribeLogGroups", "Resource": "*"},
{"Sid": "WriteLogObjects", "Effect": "Allow", "Action": "s3:PutObject",
"Resource": ["arn:aws:s3:::ssm-session-logs-111122223333-eu-central-1/sessions/*",
"arn:aws:s3:::ssm-session-logs-111122223333-eu-central-1/run-command/*",
"arn:aws:s3:::ssm-session-logs-111122223333-eu-central-1/patch/*"]},
{"Sid": "CheckBucketEncryption", "Effect": "Allow", "Action": "s3:GetEncryptionConfiguration",
"Resource": "arn:aws:s3:::ssm-session-logs-111122223333-eu-central-1"},
{"Sid": "SessionKey", "Effect": "Allow", "Action": ["kms:Decrypt", "kms:GenerateDataKey"],
"Resource": "arn:aws:kms:eu-central-1:111122223333:key/REPLACE_WITH_SESSION_KEY_ID"}
]
}aws iam put-role-policy --role-name AWSSystemsManagerDefaultEC2InstanceManagementRole \ --policy-name ssm-session-logging --policy-document file://node-logging.json
P4
OS baseline by association
0 / 0Before Run As is switched on, the Run As user must exist on every Linux node. ssm-user appears only at the first session, so the association runs daily and catches it later.
4.1 Linux: create the Run As user, strip ssm-user’s sudo, make sure screen is presentGive ssm-ops only the sudo rules the team needs, in its own sudoers file managed the same way.
os-baseline-linux.json
{
"commands": [
"id ssm-ops >/dev/null 2>&1 || useradd --create-home --shell /bin/bash ssm-ops",
"if [ -f /etc/sudoers.d/ssm-agent-users ]; then echo '#User rules for ssm-user' > /etc/sudoers.d/ssm-agent-users; fi",
"command -v screen >/dev/null || dnf -y install screen || yum -y install screen || apt-get -y install screen"
]
}aws ssm create-association --association-name os-baseline-linux --name AWS-RunShellScript \
--targets Key=tag:ssm-access,Values=ops --schedule-expression "rate(1 day)" \
--parameters file://os-baseline-linux.json --output-location \
"S3Location={OutputS3BucketName=$LOG_BUCKET,OutputS3KeyPrefix=run-command}"4.2 Windows: take ssm-user out of Administrators
os-baseline-windows.json
{
"commands": [
"if (Get-LocalGroupMember -Group Administrators -Member ssm-user -ErrorAction SilentlyContinue) { net localgroup Administrators ssm-user /delete }"
]
}aws ssm create-association --association-name os-baseline-windows --name AWS-RunPowerShellScript \ --targets Key=tag:ssm-access,Values=ops-windows --schedule-expression "rate(1 day)" \ --parameters file://os-baseline-windows.json
4.3 Check that both associations succeed
aws ssm list-associations --query 'Associations[].[AssociationName,Name,Overview.Status]' --output table aws ssm list-compliance-summaries --filters Key=ComplianceType,Values=Association,Type=EQUAL \ --query 'ComplianceSummaryItems[].[ComplianceType,CompliantSummary.CompliantCount,NonCompliantSummary.NonCompliantCount]'
P5
Session preferences
0 / 0One document per Region. With encryption required and the bucket or group not encrypted, sessions fail; phase 3 prevents that.
5.1 Write the hardened preferencesIdle 15 minutes, hard stop after 2 hours, streaming on, Run As ssm-ops on Linux and macOS. Windows sessions still run as ssm-user, now without Administrators.
SessionManagerRunShell.json
{
"schemaVersion": "1.0",
"description": "Session Manager preferences, hardened",
"sessionType": "Standard_Stream",
"inputs": {
"s3BucketName": "ssm-session-logs-111122223333-eu-central-1",
"s3KeyPrefix": "sessions",
"s3EncryptionEnabled": true,
"cloudWatchLogGroupName": "/ssm/sessions",
"cloudWatchEncryptionEnabled": true,
"cloudWatchStreamingEnabled": true,
"kmsKeyId": "REPLACE_WITH_SESSION_KEY_ID",
"runAsEnabled": true,
"runAsDefaultUser": "ssm-ops",
"idleSessionTimeout": "15",
"maxSessionDuration": "120",
"shellProfile": {
"linux": "umask 027; export TMOUT=900; export HISTTIMEFORMAT='%F %T '; echo 'Authorized use only. This session is recorded.'",
"windows": "Write-Host 'Authorized use only. This session is recorded.'"
}
}
}5.2 Update the document, or create it on a fresh Region
if aws ssm describe-document --name SSM-SessionManagerRunShell >/dev/null 2>&1; then
VER=$(aws ssm update-document --name SSM-SessionManagerRunShell --content file://SessionManagerRunShell.json \
--document-version '$LATEST' --query DocumentDescription.DocumentVersion --output text)
aws ssm update-document-default-version --name SSM-SessionManagerRunShell --document-version "$VER"
else
aws ssm create-document --name SSM-SessionManagerRunShell --content file://SessionManagerRunShell.json \
--document-type Session --document-format JSON
fi5.3 Read back what is in force
aws ssm get-document --name SSM-SessionManagerRunShell --query Content --output text | jq .inputs
P6
Operator access
0 / 0Shell for the ops team on tagged nodes, a fixed database tunnel for DBAs, and nothing implicit.
6.1 Tag the nodes each team may reach
aws ec2 create-tags --resources REPLACE_WITH_INSTANCE_IDS --tags Key=ssm-access,Value=ops
6.2 Write the shell policy
ssm-operator-shell.json
{
"Version": "2012-10-17",
"Statement": [
{"Sid": "ShellOnTaggedNodes", "Effect": "Allow", "Action": "ssm:StartSession",
"Resource": ["arn:aws:ec2:*:111122223333:instance/*", "arn:aws:ssm:*:111122223333:managed-instance/*"],
"Condition": {"StringEquals": {"ssm:resourceTag/ssm-access": "ops"},
"BoolIfExists": {"ssm:SessionDocumentAccessCheck": "true"}}},
{"Sid": "ShellDocumentOnly", "Effect": "Allow", "Action": "ssm:StartSession",
"Resource": "arn:aws:ssm:*:111122223333:document/SSM-SessionManagerRunShell",
"Condition": {"BoolIfExists": {"ssm:SessionDocumentAccessCheck": "true"}}},
{"Sid": "OwnSessionsOnly", "Effect": "Allow",
"Action": ["ssm:TerminateSession", "ssm:ResumeSession", "ssmmessages:OpenDataChannel"],
"Resource": "arn:aws:ssm:*:*:session/${aws:userid}-*"},
{"Sid": "SeeNodes", "Effect": "Allow",
"Action": ["ssm:DescribeSessions", "ssm:GetConnectionStatus", "ssm:DescribeInstanceProperties",
"ssm:DescribeInstanceInformation", "ec2:DescribeInstances"],
"Resource": "*"},
{"Sid": "SessionKey", "Effect": "Allow", "Action": "kms:GenerateDataKey",
"Resource": "arn:aws:kms:eu-central-1:111122223333:key/REPLACE_WITH_SESSION_KEY_ID"}
]
}6.3 Validate it with IAM Access Analyzer
aws accessanalyzer validate-policy --policy-type IDENTITY_POLICY --policy-document file://ssm-operator-shell.json \ --query 'findings[].[findingType,issueCode,findingDetails]' --output table
6.4 Attach it to the ops permission set and provision
export SSO_INSTANCE=$(aws sso-admin list-instances --query 'Instances[0].InstanceArn' --output text) export PS_OPS=REPLACE_WITH_OPS_PERMISSION_SET_ARN aws sso-admin put-inline-policy-to-permission-set --instance-arn $SSO_INSTANCE --permission-set-arn $PS_OPS \ --inline-policy file://ssm-operator-shell.json aws sso-admin provision-permission-set --instance-arn $SSO_INSTANCE --permission-set-arn $PS_OPS \ --target-type ALL_PROVISIONED_ACCOUNTS
6.5 Map operators to their own OS usersWith Run As on, the SSMSessionRunAs tag on the IAM role or user names the OS account; federated users can pass it as a session tag. Without a tag, sessions use ssm-ops. The OS user must exist.
aws iam tag-role --role-name REPLACE_WITH_BREAKGLASS_ROLE --tags Key=SSMSessionRunAs,Value=ops-breakglass
6.6 A database tunnel with the host fixedA copy of the remote-host forwarding document where only the local port is a parameter, so the node cannot be used to reach anything else.
pf-payments-db.json
{
"schemaVersion": "1.0",
"description": "Port forwarding to the payments database only",
"sessionType": "Port",
"parameters": {
"localPortNumber": {"type": "String", "description": "Local port", "allowedPattern": "^[0-9]{4,5}$", "default": "15432"}
},
"properties": {
"type": "LocalPortForwarding",
"host": "payments-db.cluster-REPLACE.eu-central-1.rds.amazonaws.com",
"portNumber": "5432",
"localPortNumber": "{{ localPortNumber }}"
}
}aws ssm create-document --name Ops-PortForward-PaymentsDb --document-type Session \ --document-format JSON --content file://pf-payments-db.json
6.7 DBA policy: that document, through tunnel nodes only
ssm-dba-tunnel.json
{
"Version": "2012-10-17",
"Statement": [
{"Sid": "TunnelNodes", "Effect": "Allow", "Action": "ssm:StartSession",
"Resource": "arn:aws:ec2:*:111122223333:instance/*",
"Condition": {"StringEquals": {"ssm:resourceTag/ssm-access": "db-tunnel"},
"BoolIfExists": {"ssm:SessionDocumentAccessCheck": "true"}}},
{"Sid": "PaymentsDbDocumentOnly", "Effect": "Allow", "Action": "ssm:StartSession",
"Resource": "arn:aws:ssm:eu-central-1:111122223333:document/Ops-PortForward-PaymentsDb",
"Condition": {"BoolIfExists": {"ssm:SessionDocumentAccessCheck": "true"}}},
{"Sid": "OwnSessionsOnly", "Effect": "Allow",
"Action": ["ssm:TerminateSession", "ssm:ResumeSession", "ssmmessages:OpenDataChannel"],
"Resource": "arn:aws:ssm:*:*:session/${aws:userid}-*"},
{"Sid": "SessionKey", "Effect": "Allow", "Action": "kms:GenerateDataKey",
"Resource": "arn:aws:kms:eu-central-1:111122223333:key/REPLACE_WITH_SESSION_KEY_ID"}
]
}aws ssm start-session --target REPLACE_WITH_TUNNEL_NODE --document-name Ops-PortForward-PaymentsDb \ --parameters localPortNumber=15432
P7
Just-in-time access for production
0 / 0Needs the unified console with a delegated administrator. Approval policies are created in the console under Manage node access, on the Approval policies tab. Pilot with a few users before removing StartSession from everyone.
7.1 Enable just-in-time node access for the production OUFrom the delegated administrator account. The trial starts now; per node-hour charges follow.
7.2 Deny-access policy: no auto-approval for production nodesOne per organization, in Cedar, from the delegated administrator; it overrides every auto-approval policy.
7.3 Manual approval policy for Environment=productionOne level, one approver from the platform group, 4 hours of access. Check that no node carries two tags that two manual policies target, or requests to it fail.
7.4 Auto-approval policy for development nodesAccess lasts 1 hour; one policy per account and Region.
permit (
principal in AWS::IdentityStore::Group::"REPLACE_WITH_DEV_GROUP_ID",
action == AWS::SSM::Action::"getTokenForInstanceAccess",
resource
)
when {
resource.hasTag("Environment") && resource.getTag("Environment") == "development"
};7.5 Notifications for approversEmail, and Slack or Microsoft Teams through Amazon Q Developer in chat applications. If Windows nodes are in scope, turn on RDP recording to the log bucket with the session key.
7.6 Give pilot users request rights and take StartSession away
ssm-jit-requester.json
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "RequestAndUseAccess", "Effect": "Allow",
"Action": ["ssm:StartAccessRequest", "ssm:GetAccessToken", "ssm:TerminateSession", "ssm:ResumeSession"],
"Resource": "*"
}]
}export PS_PROD=REPLACE_WITH_PROD_PERMISSION_SET_ARN aws sso-admin put-inline-policy-to-permission-set --instance-arn $SSO_INSTANCE --permission-set-arn $PS_PROD \ --inline-policy file://ssm-jit-requester.json aws sso-admin provision-permission-set --instance-arn $SSO_INSTANCE --permission-set-arn $PS_PROD \ --target-type ALL_PROVISIONED_ACCOUNTS
7.7 Walk one request end to endConnect to a production node from the console: a request with a reason is created, the approver gets the notice, approves, and the session opens and is logged like any other.
P8
Documents and commands
0 / 0No public documents, logged Automation scripts, and narrow documents in place of raw shell access.
8.1 Block public sharing in every RegionSecurity Hub SSM.7. The setting does not unshare documents that are already public; 8.2 does.
for r in $(aws ec2 describe-regions --query 'Regions[].RegionName' --output text); do
aws ssm update-service-setting --region "$r" \
--setting-id /ssm/documents/console/public-sharing-permission --setting-value Disable
done8.2 Find documents you share, and unshare any shared with All
for d in $(aws ssm list-documents --filters Key=Owner,Values=Self --query 'DocumentIdentifiers[].Name' --output text); do shared=$(aws ssm describe-document-permission --name "$d" --permission-type Share --query 'AccountIds' --output text) [ -n "$shared" ] && echo "$d -> $shared" done aws ssm modify-document-permission --name REPLACE_WITH_DOCUMENT --permission-type Share --account-ids-to-remove All
8.3 Send Automation script output to CloudWatch LogsSecurity Hub SSM.6. The default group is created on first use.
S=arn:aws:ssm:$AWS_REGION:$ACCOUNT:servicesetting/ssm/automation aws ssm update-service-setting --setting-id $S/customer-script-log-destination --setting-value CloudWatch aws ssm update-service-setting --setting-id $S/customer-script-log-group-name \ --setting-value /aws/ssm/automation/executeScript
8.4 Publish a narrow Command documentTeams that only need to restart services get this document, not AWS-RunShellScript. ENV_VAR needs agent 3.3.2746.0 or later.
restart-unit.json
{
"schemaVersion": "2.2",
"description": "Restart one allowed systemd unit",
"parameters": {
"unit": {
"type": "String",
"description": "Unit to restart",
"allowedValues": ["nginx", "php-fpm", "amazon-cloudwatch-agent"],
"interpolationType": "ENV_VAR"
}
},
"mainSteps": [{
"action": "aws:runShellScript",
"name": "restart",
"precondition": {"StringEquals": ["platformType", "Linux"]},
"inputs": {"runCommand": ["systemctl restart \"$SSM_unit\"", "systemctl is-active \"$SSM_unit\""]}
}]
}aws ssm create-document --name Ops-RestartUnit --document-type Command --document-format JSON \ --content file://restart-unit.json --tags Key=owner,Value=platform
8.5 Run it with rate controls and output to the log bucket
CMD=$(aws ssm send-command --document-name Ops-RestartUnit --parameters unit=nginx \ --targets Key=tag:Role,Values=web --max-concurrency 10% --max-errors 1 --timeout-seconds 600 \ --output-s3-bucket-name $LOG_BUCKET --output-s3-key-prefix run-command \ --comment "restart nginx, change 1234" --query Command.CommandId --output text) aws ssm list-command-invocations --command-id $CMD --query 'CommandInvocations[].[InstanceId,Status]' --output table
8.6 Tag the documents you ownSecurity Hub SSM.5, and tags let IAM scope who may use each document.
for d in Ops-RestartUnit Ops-PortForward-PaymentsDb; do aws ssm add-tags-to-resource --resource-type Document --resource-id $d --tags Key=owner,Value=platform done
P9
Patching, agent updates, inventory
0 / 0For an organization, create a patch policy in Quick Setup from the delegated administrator and skip 9.3 and 9.4. For one account, the associations and the window below do the same job.
9.1 Create your own baseline for Amazon Linux 2023
export BL=$(aws ssm create-patch-baseline --name al2023-security --operating-system AMAZON_LINUX_2023 \
--approval-rules "PatchRules=[{PatchFilterGroup={PatchFilters=[{Key=CLASSIFICATION,Values=[Security]},{Key=SEVERITY,Values=[Critical,Important]}]},ApproveAfterDays=7,ComplianceLevel=CRITICAL}]" \
--description "AL2023 security, critical and important, after 7 days" --query BaselineId --output text)
echo $BL9.2 Make it the default for that OS
aws ssm register-default-patch-baseline --baseline-id $BL aws ssm get-default-patch-baseline --operating-system AMAZON_LINUX_2023
9.3 Scan every node daily
aws ssm create-association --association-name patch-scan-daily --name AWS-RunPatchBaseline \
--targets Key=InstanceIds,Values='*' --parameters Operation=Scan --schedule-expression "rate(1 day)" \
--output-location "S3Location={OutputS3BucketName=$LOG_BUCKET,OutputS3KeyPrefix=patch}"9.4 Install weekly in a maintenance windowSunday 03:00 UTC, three hours, no new tasks in the last hour; dev nodes get an earlier window with their own tag.
MW=$(aws ssm create-maintenance-window --name patch-install-weekly --schedule "cron(0 3 ? * SUN *)" \
--duration 3 --cutoff 1 --no-allow-unassociated-targets --query WindowId --output text)
MWT=$(aws ssm register-target-with-maintenance-window --window-id $MW --resource-type INSTANCE \
--targets Key=tag:PatchWindow,Values=sun-0300 --query WindowTargetId --output text)
aws ssm register-task-with-maintenance-window --window-id $MW --task-type RUN_COMMAND \
--task-arn AWS-RunPatchBaseline --targets Key=WindowTargetIds,Values=$MWT \
--max-concurrency 20% --max-errors 1 --priority 1 \
--task-invocation-parameters '{"RunCommand":{"Parameters":{"Operation":["Install"],"RebootOption":["RebootIfNeeded"]},"TimeoutSeconds":600}}'9.5 Keep the agent current every 14 days
aws ssm create-association --association-name agent-update --name AWS-UpdateSSMAgent \ --targets Key=InstanceIds,Values='*' --schedule-expression "rate(14 days)"
9.6 Collect inventory every 12 hours
aws ssm create-association --association-name inventory --name AWS-GatherSoftwareInventory \ --targets Key=InstanceIds,Values='*' --schedule-expression "rate(12 hours)"
9.7 Read the compliance summary
aws ssm list-compliance-summaries \ --query 'ComplianceSummaryItems[].[ComplianceType,CompliantSummary.CompliantCount,NonCompliantSummary.NonCompliantCount]' \ --output table
P10
Parameter Store
0 / 0A key per environment, a path per application, and decryption bound to the parameter ARN.
10.1 Create the production parameter key
export PKEY=$(aws kms create-key --description "Parameter Store, production" --query KeyMetadata.KeyId --output text) aws kms create-alias --alias-name alias/ssm-params-prod --target-key-id $PKEY aws kms enable-key-rotation --key-id $PKEY sed -i "s/REPLACE_WITH_PARAM_KEY_ID/$PKEY/g" ./*.json
10.2 Store a secret without it touching shell history
read -rsp "db password: " DBPW; echo aws ssm put-parameter --name /prod/payments/db/password --type SecureString --key-id alias/ssm-params-prod \ --tier Standard --value "$DBPW" unset DBPW
10.3 Workload role: its own path, its own key use
param-read-payments.json
{
"Version": "2012-10-17",
"Statement": [
{"Sid": "ReadOwnPath", "Effect": "Allow",
"Action": ["ssm:GetParameter", "ssm:GetParameters", "ssm:GetParametersByPath"],
"Resource": ["arn:aws:ssm:eu-central-1:111122223333:parameter/prod/payments",
"arn:aws:ssm:eu-central-1:111122223333:parameter/prod/payments/*"]},
{"Sid": "DecryptOwnPathOnly", "Effect": "Allow", "Action": "kms:Decrypt",
"Resource": "arn:aws:kms:eu-central-1:111122223333:key/REPLACE_WITH_PARAM_KEY_ID",
"Condition": {"StringLike": {"kms:EncryptionContext:PARAMETER_ARN": "arn:aws:ssm:eu-central-1:111122223333:parameter/prod/payments/*"}}}
]
}aws iam put-role-policy --role-name REPLACE_WITH_PAYMENTS_ROLE --policy-name read-payments-params \ --policy-document file://param-read-payments.json
10.4 Expiry warning on a token that must be rotated by handAdvanced tier only, $0.05 per month, and it can never go back to standard. EventBridge receives the notification 14 days ahead.
read -rsp "partner token: " TOKEN; echo
aws ssm put-parameter --name /prod/payments/api/partner-token --type SecureString \
--key-id alias/ssm-params-prod --tier Advanced --value "$TOKEN" \
--policies '[{"Type":"Expiration","Version":"1.0","Attributes":{"Timestamp":"2027-01-31T00:00:00.000Z"}},{"Type":"ExpirationNotification","Version":"1.0","Attributes":{"Before":"14","Unit":"Days"}}]'
unset TOKEN10.5 Keep the default tier at Standard and throughput at the defaultHigher throughput costs money on every call; turn it on only when a workload is throttled.
aws ssm get-service-setting --setting-id /ssm/parameter-store/default-parameter-tier --query ServiceSetting.SettingValue aws ssm get-service-setting --setting-id /ssm/parameter-store/high-throughput-enabled --query ServiceSetting.SettingValue
P11
Organization guardrails
0 / 0From the management account. Create the PlatformSsmAdmin role in each account first, and list the Quick Setup role names your organization uses.
11.1 Write the SCP
scp-ssm-guardrails.json
{
"Version": "2012-10-17",
"Statement": [
{"Sid": "LockSessionPreferences", "Effect": "Deny",
"Action": ["ssm:CreateDocument", "ssm:UpdateDocument", "ssm:DeleteDocument", "ssm:UpdateDocumentDefaultVersion"],
"Resource": "arn:aws:ssm:*:*:document/SSM-SessionManagerRunShell",
"Condition": {"ArnNotLike": {"aws:PrincipalArn": ["arn:aws:iam::*:role/PlatformSsmAdmin"]}}},
{"Sid": "LockServiceSettings", "Effect": "Deny",
"Action": ["ssm:UpdateServiceSetting", "ssm:ResetServiceSetting"], "Resource": "*",
"Condition": {"ArnNotLike": {"aws:PrincipalArn": ["arn:aws:iam::*:role/PlatformSsmAdmin",
"arn:aws:iam::*:role/AWS-QuickSetup-*"]}}},
{"Sid": "NoSharingOut", "Effect": "Deny", "Action": "ssm:ModifyDocumentPermission", "Resource": "*",
"Condition": {"ArnNotLike": {"aws:PrincipalArn": ["arn:aws:iam::*:role/PlatformSsmAdmin"]}}},
{"Sid": "NoNewActivations", "Effect": "Deny", "Action": "ssm:CreateActivation", "Resource": "*",
"Condition": {"ArnNotLike": {"aws:PrincipalArn": ["arn:aws:iam::*:role/PlatformSsmAdmin"]}}},
{"Sid": "ProtectProdParameters", "Effect": "Deny",
"Action": ["ssm:PutParameter", "ssm:DeleteParameter", "ssm:DeleteParameters", "ssm:LabelParameterVersion"],
"Resource": "arn:aws:ssm:*:*:parameter/prod/*",
"Condition": {"ArnNotLike": {"aws:PrincipalArn": ["arn:aws:iam::*:role/PlatformSsmAdmin",
"arn:aws:iam::*:role/deploy-prod"]}}}
]
}11.2 Attach it to a sandbox OU, watch for a week, then to the workload OUsWatch for AccessDenied from Quick Setup associations and pipelines in CloudTrail before widening it.
SCP=$(aws organizations create-policy --type SERVICE_CONTROL_POLICY --name ssm-guardrails \ --description "Systems Manager control plane" --content file://scp-ssm-guardrails.json \ --query 'Policy.PolicySummary.Id' --output text) aws organizations attach-policy --policy-id $SCP --target-id REPLACE_WITH_SANDBOX_OU_ID
11.3 Config rules for Systems Manager, where Security Hub does not already deploy them
for pair in ec2-instance-managed-by-systems-manager:EC2_INSTANCE_MANAGED_BY_SSM \
ec2-managedinstance-patch-compliance-status-check:EC2_MANAGEDINSTANCE_PATCH_COMPLIANCE_STATUS_CHECK \
ec2-managedinstance-association-compliance-status-check:EC2_MANAGEDINSTANCE_ASSOCIATION_COMPLIANCE_STATUS_CHECK \
ssm-document-not-public:SSM_DOCUMENT_NOT_PUBLIC; do
aws configservice put-config-rule --config-rule \
"{\"ConfigRuleName\":\"${pair%%:*}\",\"Source\":{\"Owner\":\"AWS\",\"SourceIdentifier\":\"${pair##*:}\"}}"
done11.4 List failing Security Hub SSM controls
aws securityhub get-findings --filters '{
"ComplianceSecurityControlId":[{"Value":"SSM.","Comparison":"PREFIX"}],
"ComplianceStatus":[{"Value":"FAILED","Comparison":"EQUALS"}],
"RecordState":[{"Value":"ACTIVE","Comparison":"EQUALS"}]}' \
--query 'Findings[].[Compliance.SecurityControlId,Resources[0].Id]' --output tableP12
Detection
0 / 0An alert on anything that weakens the setup, one on unlogged tunnels, and a metric on risky commands inside sessions.
12.1 Confirm a trail records management events
aws cloudtrail describe-trails --query 'trailList[].[Name,IsOrganizationTrail,IsMultiRegionTrail,HomeRegion]' --output table
12.2 Alert topic that EventBridge may publish to
topic-policy.json
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "EventBridgePublish", "Effect": "Allow", "Principal": {"Service": "events.amazonaws.com"},
"Action": "sns:Publish", "Resource": "arn:aws:sns:eu-central-1:111122223333:ssm-alerts"
}]
}export TOPIC=$(aws sns create-topic --name ssm-alerts --query TopicArn --output text) aws sns set-topic-attributes --topic-arn $TOPIC --attribute-name Policy --attribute-value file://topic-policy.json aws sns subscribe --topic-arn $TOPIC --protocol email --notification-endpoint [email protected]
12.3 Rule: control-plane changes
ssm-control-plane.json
{
"source": ["aws.ssm"],
"detail-type": ["AWS API Call via CloudTrail"],
"detail": {
"eventSource": ["ssm.amazonaws.com"],
"eventName": ["UpdateDocument", "UpdateDocumentDefaultVersion", "DeleteDocument",
"UpdateServiceSetting", "ResetServiceSetting", "ModifyDocumentPermission",
"CreateActivation", "DeregisterManagedInstance"]
}
}aws events put-rule --name ssm-control-plane --event-pattern file://ssm-control-plane.json aws events put-targets --rule ssm-control-plane --targets Id=sns,Arn=$TOPIC
12.4 Rule: tunnels and SSH sessions, which leave no transcript
ssm-tunnels.json
{
"source": ["aws.ssm"],
"detail-type": ["AWS API Call via CloudTrail"],
"detail": {
"eventSource": ["ssm.amazonaws.com"],
"eventName": ["StartSession"],
"requestParameters": {"documentName": ["AWS-StartPortForwardingSession",
"AWS-StartPortForwardingSessionToRemoteHost", "AWS-StartSSHSession"]}
}
}aws events put-rule --name ssm-tunnels --event-pattern file://ssm-tunnels.json aws events put-targets --rule ssm-tunnels --targets Id=sns,Arn=$TOPIC
12.5 Metric and alarm on instance metadata calls inside sessionsStreamed transcripts take metric filters; extend the pattern list to your own red flags.
aws logs put-metric-filter --log-group-name /ssm/sessions --filter-name imds-in-session \ --filter-pattern '"169.254.169.254"' \ --metric-transformations metricName=ImdsInSession,metricNamespace=SSM/Sessions,metricValue=1 aws cloudwatch put-metric-alarm --alarm-name ssm-imds-in-session --namespace SSM/Sessions \ --metric-name ImdsInSession --statistic Sum --period 300 --evaluation-periods 1 --threshold 1 \ --comparison-operator GreaterThanOrEqualToThreshold --treat-missing-data notBreaching --alarm-actions $TOPIC
12.6 Review recent sessions weekly
aws ssm describe-sessions --state History --max-results 50 \ --query 'Sessions[].[StartDate,Owner,Target,DocumentName,Status]' --output table
P13
Hybrid nodes
0 / 0Only if you manage machines outside EC2. Run 13.1 and 13.2 as PlatformSsmAdmin, because the SCP blocks activations for everyone else. Since 30 September 2026 each session costs $0.05 and each command $0.002 on these nodes.
13.1 Service role that only Systems Manager in this account and Region can assume
trust-hybrid.json
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow", "Principal": {"Service": "ssm.amazonaws.com"}, "Action": "sts:AssumeRole",
"Condition": {"StringEquals": {"aws:SourceAccount": "111122223333"},
"ArnLike": {"aws:SourceArn": "arn:aws:ssm:eu-central-1:111122223333:*"}}
}]
}aws iam create-role --role-name SSMServiceRole-hybrid --assume-role-policy-document file://trust-hybrid.json aws iam attach-role-policy --role-name SSMServiceRole-hybrid \ --policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore aws iam put-role-policy --role-name SSMServiceRole-hybrid --policy-name ssm-session-logging \ --policy-document file://node-logging.json
13.2 Single-use activation that expires in two hoursTags on the activation land on the node. GNU date shown; on macOS use date -u -v+2H.
EXP=$(date -u -d '+2 hours' +%Y-%m-%dT%H:%M:%S) read ACT_ID ACT_CODE <<< "$(aws ssm create-activation --default-instance-name onprem-web01 \ --iam-role SSMServiceRole-hybrid --registration-limit 1 --expiration-date $EXP \ --tags Key=Environment,Value=production Key=ssm-access,Value=ops \ --query '[ActivationId,ActivationCode]' --output text)" echo $ACT_ID
13.3 Register the machine with ssm-setup-cliDownload it from the activation’s Region, check it with the AWS integrity script before running, and do not keep a copy for later use.
mkdir -p /tmp/ssm && cd /tmp/ssm curl -fsSo ssm-setup-cli https://amazon-ssm-eu-central-1.s3.eu-central-1.amazonaws.com/latest/linux_amd64/ssm-setup-cli chmod +x ssm-setup-cli sudo ./ssm-setup-cli -register -activation-code "REPLACE_WITH_ACT_CODE" -activation-id "REPLACE_WITH_ACT_ID" -region eu-central-1
13.4 Rotate the machine key every 30 daysAgent 3.0.1031.0 or later.
cd /etc/amazon/ssm [ -f amazon-ssm-agent.json ] || sudo cp amazon-ssm-agent.json.template amazon-ssm-agent.json sudo jq '.Profile.KeyAutoRotateDays = 30' amazon-ssm-agent.json | sudo tee amazon-ssm-agent.json.new >/dev/null sudo mv amazon-ssm-agent.json.new amazon-ssm-agent.json sudo systemctl restart amazon-ssm-agent
13.5 Delete the activation and check for leftovers
aws ssm delete-activation --activation-id $ACT_ID aws ssm describe-activations --query 'ActivationList[].[ActivationId,DefaultInstanceName,Expired,RegistrationsCount]' --output table aws ssm describe-instance-information --filters Key=ResourceType,Values=ManagedInstance \ --query 'InstanceInformationList[].[InstanceId,ComputerName,PingStatus,AgentVersion]' --output table
P14
Verify
0 / 0Prove each layer with a positive and a negative test, and keep the output as evidence.
14.1 A shell works, runs as ssm-ops, and has no unexpected sudoInside the session run whoami; sudo -n true: expect ssm-ops and a refusal.
export TEST_NODE=REPLACE_WITH_INSTANCE_ID aws ssm start-session --target $TEST_NODE --document-name SSM-SessionManagerRunShell
14.2 The transcript reached S3 and CloudWatch
aws s3 ls s3://$LOG_BUCKET/sessions/ | tail -3 aws logs describe-log-streams --log-group-name /ssm/sessions --order-by LastEventTime --descending \ --max-items 3 --query 'logStreams[].logStreamName'
14.3 The implicit shell is closedAs a DBA (phase 6.7), a session without the tunnel document must end in AccessDeniedException.
aws ssm start-session --target REPLACE_WITH_TUNNEL_NODE
14.4 Service settings report
for s in /ssm/managed-instance/default-ec2-instance-management-role \
/ssm/documents/console/public-sharing-permission \
/ssm/automation/customer-script-log-destination \
/ssm/parameter-store/high-throughput-enabled; do
aws ssm get-service-setting --setting-id "$s" --query 'ServiceSetting.[SettingId,SettingValue]' --output text
done14.5 A test change fires the alertAs PlatformSsmAdmin, rerun step 5.2 with the same file after a whitespace change; the email from 12.3 should arrive within minutes.
14.6 Save the evidence
mkdir -p evidence aws ssm get-document --name SSM-SessionManagerRunShell --query Content --output text > evidence/preferences.json aws ssm list-associations --output json > evidence/associations.json aws ssm list-compliance-summaries --output json > evidence/compliance.json aws ec2 describe-vpc-endpoints --filters Name=vpc-id,Values=$VPC --output json > evidence/endpoints.json
PB
Playbooks
For the day something goes wrong. Ticks here do not count toward the runbook progress.
PB1 Node missing or ConnectionLost
when a node drops out of Fleet Manager or start-session says the target is not connected
Run the diagnostics on the nodeReach it through the serial console or an EC2 Instance Connect Endpoint if needed.
sudo ssm-cli get-diagnostics --output table sudo systemctl status amazon-ssm-agent sudo tail -50 /var/log/amazon/ssm/errors.log
Check credentials in orderAn instance profile without SSM rights blocks DHMC only if it allows UpdateInstanceInformation; a profile with it wins. DHMC needs IMDSv2; containers on the node need hop limit 2.
Check the pathPrivate DNS on the endpoints, the endpoint security group allows 443 from the node, the endpoint policy allows your org, and ssmmessages is present.
PB2 Session refused or ends at once
when start-session fails or closes immediately
Read the error and match itAccessDenied: tag, document or SessionDocumentAccessCheck. KMS errors: user lacks GenerateDataKey or node lacks Decrypt. Immediate close with Run As: the OS user does not exist. Encryption check failures: bucket or log group not encrypted while the preferences require it.
Simulate the caller’s policy
aws iam simulate-principal-policy --policy-source-arn REPLACE_WITH_ROLE_ARN --action-names ssm:StartSession \ --resource-arns arn:aws:ssm:$AWS_REGION:$ACCOUNT:document/SSM-SessionManagerRunShell \ --query 'EvaluationResults[].[EvalActionName,EvalDecision]' --output table
PB3 Suspected misuse of sessions or commands
when an alert fires or a transcript shows something wrong
End the live sessions of that principal
aws ssm describe-sessions --state Active --query 'Sessions[].[SessionId,Owner,Target]' --output table aws ssm terminate-session --session-id REPLACE_WITH_SESSION_ID
Stop new sessions everywhere if you mustDisabling the session key blocks every new encrypted session in the Region until it is enabled again. Use it as the last resort.
aws kms disable-key --key-id $KEY
Collect what ran
aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventName,AttributeValue=SendCommand \ --max-results 50 --query 'Events[].[EventTime,Username]' --output table aws ssm list-commands --max-results 50 --query 'Commands[].[CommandId,DocumentName,RequestedDateTime,Status]' --output table
PB4 Activation code leaked
when an activation ID and code appear in a ticket, chat or repository
Delete the activation, then find what registered with it
aws ssm delete-activation --activation-id REPLACE_WITH_ACT_ID aws ssm describe-instance-information --filters Key=ActivationIds,Values=REPLACE_WITH_ACT_ID \ --query 'InstanceInformationList[].[InstanceId,ComputerName,IPAddress,RegistrationDate]' --output table
Deregister unknown nodes
aws ssm deregister-managed-instance --instance-id REPLACE_WITH_MI_ID
PB5 Session logging was switched off
when the control-plane alert names SSM-SessionManagerRunShell
Restore the known-good version as default
aws ssm list-document-versions --name SSM-SessionManagerRunShell \ --query 'DocumentVersions[].[DocumentVersion,CreatedDate,IsDefaultVersion]' --output table aws ssm update-document-default-version --name SSM-SessionManagerRunShell --document-version REPLACE_WITH_GOOD_VERSION
Find out who and howIf the SCP is attached, only PlatformSsmAdmin could have done it: review that role’s sessions and whether the SCP reached this account.
FIX
Troubleshooting
| Symptom | Cause | Fix | Step |
|---|---|---|---|
| Instances never appear after DHMC | IMDSv1, old agent, or a profile that allows UpdateInstanceInformation | require IMDSv2, update the agent, strip SSM rights from the profile | 1.2, 1.5 |
| Node drops off about an hour after a policy change | ssmmessages:OpenControlChannel removed from the node role | restore AmazonSSMManagedEC2InstanceDefaultPolicy or Core | 1.3 |
| Private node offline, NAT removed | missing ssmmessages endpoint or private DNS off | create it with private DNS; check the endpoint SG | 2.3 |
| Agent updates or patching fail in private subnets | S3 gateway endpoint missing or its policy too tight | add the gateway endpoint; allow the AWS-owned buckets | 2.4 |
| Session fails with an encryption error | bucket or log group unencrypted, or the node cannot use the key | encrypt both; Decrypt and GenerateDataKey for the node role | 3.3, 3.6 |
| Session closes at once on Linux | Run As user missing on that node | run the OS baseline association now | 4.1 |
| No CloudWatch transcript, S3 fine | node role cannot DescribeLogGroups or write to the group | node-logging.json on the role actually in use | 3.6 |
| Transcripts cut short | screen not installed on the image | install it, or let the association do it | 4.1 |
| A user meant for tunnels opens a shell | SessionDocumentAccessCheck missing | add the condition to both StartSession statements | 6.7 |
| Just-in-time request fails for one node | two manual approval policies match its tags, or no policy matches | one manual policy per node; a fallback for all nodes | 7.3 |
| Document parameter rejected | value outside allowedValues or allowedPattern | expected; extend the list through review | 8.4 |
| Nodes non-compliant right after patching | NoReboot left patches pending a restart | reboot in the window, or RebootIfNeeded | 9.4 |
| App gets AccessDenied on a SecureString | Decrypt condition does not match the parameter ARN, or wrong key | check the path in kms:EncryptionContext:PARAMETER_ARN | 10.3 |
| Quick Setup associations fail after the SCP | their role is not in the exceptions | add the role ARN pattern, reattach | 11.2 |
| No alert e-mail | topic policy missing, or subscription not confirmed | set the topic policy; confirm the e-mail | 12.2 |
| Hybrid registration fails | activation expired or used, wrong Region, clock skew | new single-use activation in the right Region; fix NTP | 13.2 |
AWS Systems Manager Runbook · REV 2026.10
© 2026 · BUB.IM · made in vim
0 comments