adrianhesketh.com

Secure your AWS CI/CD pipelines with a Permissions Boundary

The permissions required to deploy a serverless project are fairly broad. Lambda functions that access AWS resources such as S3 buckets, or DynamoDB require the creation of a new role that has the appropriate access, or the use of an existing role that has the access.

To support this, many teams give their CI user the AdministratorAccess role, or the PowerUserAccess role with iam:CreateRole and iam:PutRolePolicy permissions, or iam:* attached to enable the creation of roles during the deployment process.

Even if an attempt to create a minimal set of permissions has been given to the CI/CD user, if the CI/CD user can create roles or users, it opens up your AWS account to a privilege escalation attack via a number of routes if a permissions boundary is not applied.

How can this be exploited?

If your CI/CD user can create an IAM role, or IAM user, then malicious code that has been committed to the source code repository can be used to create a new user or role that has more permissions than the CI/CD user has.

For example, if you’ve limited the permissions of the CI/CD user, e.g. by not giving the user access to read from DynamoDB tables, or create EC2 instances, an attacker can gain permissions to do thesse things by executing code to create a new role that has AdministratorAccess and can do everything. The new role can be assumed from another AWS account to carry out additional attacks later.

In practical terms, there’s a few ways to exploit this. One way is to sneak your exploit code into the repository as a dependency (a supply chain attack) that is executed during unit test execution. A simple TypeScript microservice might have upwards of 50 explict dependencies and 900 transitive dependencies - sneaking a few lines of AWS SDK code in a dependency is likely to go unnoticed.

It’s also easy enough for an attacker to put exploit code in a specific version of an innocuous sounding NPM package, jest-mock-logger or mock-database-jest are names that might pass a code review from an unwary software engineer.

Another supply chain attack would be to exploit Docker images or Github Actions used by the CI/CD process.

The SSH keys and Github accounts of software engineers that are committing into the git repository also become serious attack vectors to the AWS environment. Many software engineers use SSH keys that are simply stored in their ~/.ssh directory as plaintext. If you have access to the filesystem of a developer’s machine, you may be able to steal their SSH key, and use that to commit code to the repository that creates a new role with the permissions you want. That’s why I use a hardware security token to store my SSH key.

There are lots of ways to steal keys - supply chain attacks on the developers, or temporary physical attacks such as using a USB Rubbber Ducky (a super-fast, programmable USB keyboard device) to type in and execute a script that exfiltrates data faster than a human could.

How can this be protected against?

One approach is to not give the CI/CD user permission to update or assign IAM roles, and route permission changes through a dedicated security team instead. This is an approach used by financial organisations I’ve worked with, but the overhead of requesting additional roles can result in the creation of fairly broad roles that are then applied to lots of Lambda functions, rather than exercising Pricinple of Least Privilege for IAM roles.

My preferred approach is to get the infrastructure / security team to define the maximum set of permissions that would be sensible for IAM roles created by the team to have, and then assign an IAM Permission Boundary to the team’s CI/CD user to configure the maximum set of permissions the CI/CD user can have.

The Permissions Boundary must prevent IAM entities from being created that don’t also have the Permissions Boundary attached. Without this protection, any IAM entities that are created can escape the Permissions Boundary, and have any set of permissions. [1]

With that in place, the development team’s CI/CD user can have broad permissions, because only changes that fall within the permission boundary will take effect.

How do I create a Permissions Boundary?

I’ve created examples in CDK and CloudFormation of how to create the boundaries, and users that have the boundary applied.

How do I use a Permissions Boundary?

CDK / Go

Apply the Permissions Boundary to the stack.

permissionsBoundary := awsiam.ManagedPolicy_FromManagedPolicyName(stack,
	jsii.String("permissionsBoundary"),
	jsii.String("ci-permissions-boundary"))
awsiam.PermissionsBoundary_Of(stack).Apply(permissionsBoundary)

Serverless Framework

Set the iam section’s permissionsBoundary property. [2]

Summary

To close up a privilege escalation attack vector in CI pipelines that need to create IAM roles, you can use a Permissions Boundary.

Permissions Boundaries set the maximum set of permissions that an IAM user or role can have, and can be made to apply to any IAM roles or users that it creates.

This is a good alternative to centralising the creation of IAM roles, which can be a blocker for teams.