adrianhesketh.com

Go CDK: Building Go Lambda functions

Go Lambda functions are structured as an individual Linux executable that uses the github.com/aws/aws-lambda-go/lambda package’s Start function to run the Lambda runtime to process requests.

Requests are then processed by the handle function that is passed into lambda.Start.

The signature of the handle method is different depending on the event that’s triggering the invocation. For example, API Gateway expects a return value, but SQS or EventBridge don’t.

While it’s possible to trigger AWS Lambda functions with arbitrary JSON payloads using the AWS CLI, SDK or in the console, the vast majority of time, your Lambda function will be triggered by an AWS service, like API Gateway.

AWS provides structured types for Lambda triggers in the github.com/aws/aws-lambda-go/events package, so your Lambda handler will look something like this:

package main

import (
	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-lambda-go/lambda"
)


func main() {
	// lambda.Start(HandleDynamoDBStream)
	lambda.Start(HandleAPIGateway)
}

func HandleDynamoDBStream(ctx context.Context, event events.DynamoDBEvent) error {
	//TODO: Put your logic here.	
}

func HandleAPIGateway(ctx context.Context, req events.APIGatewayProxyRequest) (resp events.APIGatewayProxyResponse, err error) {
	//TODO: Put your logic here.	
}

Lambda in CDK

awslambdago

The best way to build Go Lambda function is to use the github.com/aws/aws-cdk-go/awscdk/awslambdago package. is a higher level construct than github.com/aws/aws-cdk-go/awscdk/awslambda and will automatically build your Lambda functions.

Even though I’ve used the Node.js equivalent, I didn’t spot that this package existed, and wasted a bit of time using the lower level awslambda package until my colleague Matthew Murray pointed me at this.

If you don’t have Go installed, it will try and build the functions in a Docker container.

The Go executables can be made smaller by stripping out the symbol table and debug information (-s) and omitting the DWARF symbol table (-w). This doesn’t make stack traces unreadable, so it’s quite appropriate for production use.

bundlingOptions := &awslambdago.BundlingOptions{
	GoBuildFlags: &[]*string{jsii.String(`-ldflags "-s -w"`)},
}

// notFound.
notFound := awslambdago.NewGoFunction(stack, jsii.String("notFoundHandler"), &awslambdago.GoFunctionProps{
	Runtime:  awslambda.Runtime_GO_1_X(),
	Entry:    jsii.String("../api/notfound"),
	Bundling: bundlingOptions,
})

awslambda

This package is only useful if you already have a process that has built the Lambda functions already.

In most cases, you’ll want to use the github.com/aws/aws-cdk-go/awscdk/awslambdago package instead which builds your code for you.

If you use the the awslambda package, you must set:

  • The Code field to be the directory containing the Go executable.
  • The Handler field to be the name of the Go executable.
notFound := awslambda.NewFunction(stack, jsii.String("notFoundHandler"), &awslambda.FunctionProps{
	Runtime: awslambda.Runtime_GO_1_X(),
	Code:    awslambda.Code_FromAsset(jsii.String("../api/notfound"), &awss3assets.AssetOptions{}),
	Handler: jsii.String("lambdaHandler"),
})

I use a Mac as my work machine, but at the time of writing Lambda requires the Go program to be compiled for Linux on the x64 processor architecture.

On my Mac, if I build a Go program with the go build command, it gets compiled for that operating system and processor architecture. To build it for Linux on x64, the GOOS (Go operating system) and GOARCH (Go architecture) environment variables need to be set:

GOOS=linux GOARCH=amd64 go build

That command produces a Go binary for a Linux x64 architecture that can run on Lambda, but the name of the executable will be the name of the current directory. To force the name to be lambdaHandler, the -o flag can be used.

GOOS=linux GOARCH=amd64 go build -o lambdaHandler .

Finally, the Go executable can be made smaller by stripping out the symbol table and debug information (-s) and omitting the DWARF symbol table (-w). This doesn’t make stack traces unreadable, so it’s quite appropriate for production use.

GOOS=linux GOARCH=amd64 go build -ldflags="-s -w" -o lambdaHandler .

The flags are documented at [0]

Now we have the command to run, it’s just a case of going into each directory (cd) and building the program with the command before we run cdk deploy.

My first pass at this was to use 4 lines of shell script. Unfortunately, it needs a lot of unpacking to understand…

find ./api -type f -name "*main.go" |
  xargs --no-run-if-empty dirname | 
  xargs readlink -f |
  awk '{print "cd "$1" && GOOS=linux GOARCH=amd64 go build -ldflags=\"-s -w\" -o lambdaHandler main.go\0"}' |
  sh -v

First, the find command finds all of the main.go files in the ./api path and the output is piped into xargs.

The first xargs calls dirname for each main.go file and strips /xxxx/main.go back to just the directory name /xxxx/.

Next, xargs is used again to run readlink to get the fully qualified path of each directory instead of the relative path.

awk is then used to construct a shell command print out the go build command for each line.

Finally, the output is passed into sh -v which shows both the output of the go build command, but also prints out the command that was executed.

Unfortunately, while the commands work great on Linux (and therefore Github Actions) they don’t work on MacOS, because xargs and readlink don’t have the same rich set of options that their Linux counterparts provide.

Go version

If your team is building Go programs, you might decide to run a build using Go instead. This has a couple of benefits:

  • Everyone understands Go - it doesn’t need deciphering.
  • It works across Linux and Mac platforms (I haven’t tried Windows).

But it has a downside of being a lot longer than 5 lines:

package main

import (
	"fmt"
	"io/fs"
	"os"
	"os/exec"
	"path"
	"path/filepath"
	"time"
)

func main() {
	start := time.Now()
	fmt.Println("Finding Go Lambda function code directories...")
	dirs, err := getDirectoriesContainingMainGoFiles("./api")
	if err != nil {
		fmt.Printf("Failed to get directories containing main.go files: %v\n", err)
		os.Exit(1)
	}
	fmt.Printf("%d Lambda entrypoints found...\n", len(dirs))
	for i := 0; i < len(dirs); i++ {
		fmt.Printf("Building Lambda %d of %d...\n", i+1, len(dirs))
		err = buildMainGoFile(dirs[i])
		if err != nil {
			fmt.Printf("Error: %v\n", err)
			os.Exit(1)
		}
	}
	fmt.Printf("Built %d Lambda functions in %v\n", len(dirs), time.Now().Sub(start))
}

func buildMainGoFile(path string) error {
	// GOOS=linux GOARCH=amd64 go build -ldflags="-s -w" -o lambdaHandler .
	cmd := exec.Command("go", "build", "-ldflags=-s -w", "-o", "lambdaHandler")
	cmd.Dir = path
	cmd.Env = append(os.Environ(),
		"GOOS=linux",
		"GOARCH=amd64")
	cmd.Stdout = os.Stdout
	cmd.Stderr = os.Stderr
	if err := cmd.Run(); err != nil {
		return fmt.Errorf("error running command: %w", err)
	}
	if exitCode := cmd.ProcessState.ExitCode(); exitCode != 0 {
		return fmt.Errorf("non-zero exit code: %v", exitCode)
	}
	return nil
}

func getDirectoriesContainingMainGoFiles(srcPath string) (paths []string, err error) {
	filepath.Walk(srcPath, func(currentPath string, info fs.FileInfo, err error) error {
		if info.IsDir() {
			// Continue.
			return nil
		}
		d, f := path.Split(currentPath)
		if f == "main.go" {
			paths = append(paths, d)
		}
		return nil
	})
	if err != nil {
		err = fmt.Errorf("failed to walk directory: %w", err)
		return
	}
	return
}

I just named the file build-lambda.go and run it with go run build-lambda.go in my Makefile before running deploy to make sure that all of the Lambda functions have been built.

build-lambda:
	go run build-lambda.go

deploy: build-lambda
	cd cdk && cdk deploy

Hopefully this will save you a few minutes on your next Go, Lambda and CDK project!