I’ve gotten used to the luxury of CircleCI, but I thought I’d have a go at AWS CodeBuild to bring the build pipeline inside AWS to reduce the attack surface area.

AWS Code Pipeline supports triggering builds from Github source code repositories, so I was able to get the project’s source code from Github without issues. However, when you run go get to fetch dependencies, if any of them are private, then the build server needs to be able to authenticate against Github too as part of the build process.

At first, I created a “Deploy Key” for the project I wanted the build server to build, but Deploy Keys aren’t a good match for projects which require access to multiple private repositories, so I had to setup a Github user just for the build server and setup an SSH key for it. (https://developer.github.com/v3/guides/managing-deploy-keys/)

Next, I needed to run ssh-keygen to create a new key, then put that into the AWS SSM parameter store so that I could get it from within a CodeBuild build step and add it to the list of SSH identities.

aws ssm put-parameter --name build_ssh_key --type String --value "$(cat build_ssh_key.pem)"

Once I’d added the parameter, I then needed to give the AWS-CodeBuild-Role permission to read that key.

Once that was done, I could configure the buildspec.yml to get the SSM parameter, add it as an environment variable, then configure the build to be able to authenticate against Github. https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html

The setup for Go on CodeBuild requires a few extra stages https://www.contributing.md/2017/06/30/golang-with-aws-codebuild/). Most important was to copy the code from the CODEBUILD_SRC_DIR into the GOPATH and then to remember to change directory into the GOPATH copy, not where the source code started out. CodeBuild lacks the feature to checkout to a specific checkout directory.

Locally, I could replicate what I expected the build server to do by running Docker and then executing commands at the terminal.

$ docker run -v `pwd`:/codebuild/output/src223872387/src -e "CODEBUILD_SRC_DIR=/codebuild/output/src223872387/src" -e build_ssh_key="$build_ssh_key" --rm -it xxxxxxx/golang-build:latest /bin/bash

However, when trying to get it to work as part of CodeBuild, I got messages in the logs from unexpected directories and the GOPATH had an extra colon after it.

make[1]: Entering directory '/go:/codebuild/output/src823267486/src/github.com ...

Unlike CircleCI, there’s no way to SSH into a broken build and attempt to fix it, so the troubleshooting workflow was tortuous – basically putting echo statements into some YAML and running it again.

Given that I was unable to easily replicate the problem using the same Docker container locally, I gave up after wasting a few hours fiddling with it. I might give it another try in the future.