Getting Started with Serverless Go

On January 2018, AWS Lambda released official support for the Go language.

In this guide, you’ll learn how to get started with building Go applications on AWS Lambda with the Serverless framework. This brief guide consists of two parts: a brief section on the Go language and a hands-on section where you’ll build a Serverless Go CRUD API.

The final application is available on Github. Just hit deploy!

The Go Language

First, let’s setup Go on your machine and briefly look at the Go language.

Setup

Download Go and follow the installation instructions.

On OSX, you can download the go1.9.3.darwin-amd64.pkg package file, open it, and follow the prompts to install the Go tools. The package installs the Go distribution to /usr/local/go.

To test your Go installation, open a new terminal and enter:

$ go version
go version go1.9.2 darwin/amd64

Then, add the following to your ~/.bashrc to set your GOROOT and GOPATH environment variables:

export GOROOT=/usr/local/go
export GOPATH=/Users/<your.username>/gopath
export PATH=$PATH:$GOROOT/bin:$GOPATH/bin%

source ~/.bashrc

Basics

Next, try setting up a workspace: create a directory in $GOPATH/src/learn-go/ and in that directory create a file named hello.go.

$ mkdir learn-go
$ cd learn-go
$ touch hello.go

// hello.go

package main

import "fmt"

func main() {
   fmt.Printf("hello, world\n")
}

Run your code by calling go run hello.go. You can also go build Go programs into binaries, which lets us execute the built binary directly:

$ go build hello.go

The command above will build an executable named hello in the directory alongside your source code. Execute it to see the greeting:

$ ./hello
hello, world

If you see the “hello, world” message then your Go installation is working!

Package Management

dep is a dependency management tool for Go.

On MacOS you can install or upgrade to the latest released version with Homebrew:

$ brew install dep
$ brew upgrade dep

To get started, create a new directory learn-dep/ in your $GOPATH/src:

$ mkdir learn-dep
$ cd learn-dep

Initialize the project with dep init:

$ dep init
$ ls
Gopkg.lock Gopkg.toml vendor

dep init will create the following:

• Gopkg.lock is a record of the exact versions of all of the packages that you used for the project.
• Gopkg.toml is a list of packages your project depends on.
• vendor/ is the directory where your project’s dependencies are installed.

You can add new dependencies with the -add flag:

$ dep ensure -add github.com/pkg/errors

For detailed usage instructions, check out the official dep docs

AWS Lambda Go Programming Model

You write code for your Lambda function in one of the languages AWS Lambda supports. Regardless of the language you choose, there is a common pattern to writing code for a Lambda function that includes the following core concepts:

• Handler – Handler is the function AWS Lambda calls to start execution of your Lambda function. Your handler should process incoming event data and may invoke any other functions/methods in your code.

• The context object – AWS Lambda also passes a context object to the handler function, which lets you retrieve metadata such as the execution time remaining before AWS Lambda terminates your Lambda function.

• Logging – Your Lambda function can contain logging statements. AWS Lambda writes these logs to CloudWatch Logs.

• Exceptions – There are different ways to end a request successfully or to notify AWS Lambda an error occurred during execution. If you invoke the function synchronously, then AWS Lambda forwards the result back to the client.

Your Lambda function code must be written in a stateless style, and have no affinity with the underlying compute infrastructure. Your code should expect local file system access, child processes, and similar artifacts to be limited to the lifetime of the request. Persistent state should be stored in Amazon S3, Amazon DynamoDB, or another cloud storage service.

Go Lambda Function

Your Go programs are compiled into a statically-linked binary, bundled up into a Lambda deployment package, and uploaded to AWS Lambda.

You write your Go handler function code by including the github.com/aws/aws-lambda-go/lambda package and a main() function:

package main

import (
  "fmt"
  "context"
  "github.com/aws/aws-lambda-go/lambda"
)

type MyEvent struct {
  Name string `json:"name"`
}

func HandleRequest(ctx context.Context, name MyEvent) (string, error) {
  return fmt.Sprintf("Hello %s!", name.Name ), nil
}

func main() {
  lambda.Start(HandleRequest)
}

Note the following:

• package main: In Go, the package containing func main() must always be named main.

• import: Use this to include the libraries your Lambda function requires.

  • context: The Context Object.
  • fmt: The Go Formatting object used to format the return value of your function.
  • github.com/aws/aws-lambda-go/lambda: As mentioned previously, implements the Lambda programming model for Go.

• func HandleRequest(ctx context.Context, name string) (string, error): This is your Lambda handler signature and includes the code which will be executed. In addition, the parameters included denote the following:

  • ctx context.Context: Provides runtime information for your Lambda function invocation. ctx is the variable you declare to leverage the information available via the the Context Object.

  • name string: An input type with a variable name of name whose value will be returned in the return statement.

  • string error: Returns standard error information.

  • return fmt.Sprintf(“Hello %s!”, name), nil: Simply returns a formatted “Hello” greeting with the name you supplied in the handler signature. nil indicates there were no errors and the function executed successfully.

• func main(): The entry point that executes your Lambda function code. This is required. By adding lambda.Start(HandleRequest) between func main(){} code brackets, your Lambda function will be executed.

Each AWS event source (API Gateway, DynamoDB, etc.) has its own input/output structs. For example, lambda functions that is triggered by API Gateway events use the events.APIGatewayProxyRequest input struct and events.APIGatewayProxyResponse output struct:

package main

import (
	"context"
	"fmt"

	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-lambda-go/lambda"
)

func handleRequest(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	fmt.Printf("Body size = %d.\n", len(request.Body))
	fmt.Println("Headers:")
	for key, value := range request.Headers {
		fmt.Printf("    %s: %s\n", key, value)
	}

	return events.APIGatewayProxyResponse{Body: request.Body, StatusCode: 200}, nil
}

func main() {
	lambda.Start(handleRequest)
}

For more information on handling events from AWS event sources, see aws-lambda-go/events.

Building a Go CRUD API

In this section, you’ll create an HTTP CRUD API using Go, AWS Lambda, and the Serverless framework.

Prerequisites

Before we continue, make sure that you have:

• Go and serverless installed on your machine.
• Your AWS account set up.

New to Serverless? Get Going Serverless!

Design

For each endpoint in our backend’s HTTP API, you can create a Function that corresponds to an action. For example:

`GET /todos`          ->      `listTodos`

`POST /todos`         ->      `addTodo`

`PATCH /todos/{id}`   ->      `completeTodo`

`DELETE /todos/{id}`  ->      `deleteTodo`

The listTodos function returns all of our todos, addTodo adds a new row to our todos table, and so on. When designing Functions, keep the Single Responsibility Principle in mind.

Hands-On

The final serverless-crud-go sample application is available on Github as reference.

Start by cloning the serverless-go-boilerplate scaffold which offers a starting point for building a Serverless Go project.

Copy the entire project folder to your $GOPATH/src and rename the directory and to your own project name. Remember to update the project’s name in serverless.yml to your own project name!

The serverless-boilerplate-go project has this structure:

.
+-- scripts/
+-- src/
      +-- handlers/
+-- .gitignore
+-- README.md
+-- Gopkg.toml
+-- serverless.yml

Within this boilerplate, we have the following:

• scripts contains a build.sh script that you can use to compile binaries for the lambda deployment package.
• src/handlers/ is where your handler functions will live.
• Gokpkg.toml is used for Go dependency management with the dep tool.
• serverless.yml is a Serverless project configuration file.
• README.md contains step-by-step setup instructions.

In your terminal, navigate to your project’s root directory and install the dependencies defined in the boilerplate:

cd <your-project-name>
dep ensure

With that set up, let’s get started with building our CRUD API!

Step 1: Create the POST /todos endpoint

Event

First, define the addTodo Function’s HTTP Event trigger in serverless.yml:

// serverless.yml

package:
 individually: true
 exclude:
   - ./**

functions:
  addTodo:
    handler: bin/handlers/addTodo
    package:
      include:
        - ./bin/handlers/addTodo
    events:
      - http:
          path: todos
          method: post
          cors: true

In the above configuration, notice two things:

• Within the package block, we tell the Serverless framework to only package the compiled binaries in bin/handlers and exclude everything else.
• The addTodo function has an HTTP event trigger set to the POST /todos endpoint.

Function

Create a new file within the src/handlers/ directory called addTodo.go:

// src/handlers/addTodo.go

package main

import (
	"context"
	"fmt"
	"os"
	"time"
	"encoding/json"

	"github.com/aws/aws-lambda-go/lambda"
	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/dynamodb"
	"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"

	"github.com/satori/go.uuid"
)

type Todo struct {
	ID          string  `json:"id"`
	Description string 	`json:"description"`
	Done        bool   	`json:"done"`
	CreatedAt   string 	`json:"created_at"`
}

var ddb *dynamodb.DynamoDB
func init() {
	region := os.Getenv("AWS_REGION")
	if session, err := session.NewSession(&aws.Config{ // Use aws sdk to connect to dynamoDB
		Region: &region,
	}); err != nil {
		fmt.Println(fmt.Sprintf("Failed to connect to AWS: %s", err.Error()))
	} else {
		ddb = dynamodb.New(session) // Create DynamoDB client
	}
}

func AddTodo(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	fmt.Println("AddTodo")

	var (
		id = uuid.Must(uuid.NewV4(), nil).String()
		tableName = aws.String(os.Getenv("TODOS_TABLE_NAME"))
	)

	// Initialize todo
	todo := &Todo{
		ID:					id,
		Done:				false,
		CreatedAt:			time.Now().String(),
	}

	// Parse request body
	json.Unmarshal([]byte(request.Body), todo)

	// Write to DynamoDB
	item, _ := dynamodbattribute.MarshalMap(todo)
	input := &dynamodb.PutItemInput{
		Item: item,
		TableName: tableName,
	}
	if _, err := ddb.PutItem(input); err != nil {
		return events.APIGatewayProxyResponse{ // Error HTTP response
			Body: err.Error(),
			StatusCode: 500,
		}, nil
	} else {
		body, _ := json.Marshal(todo)
		return events.APIGatewayProxyResponse{ // Success HTTP response
			Body: string(body),
			StatusCode: 200,
		}, nil
	}
}

func main() {
	lambda.Start(AddTodo)
}

In the above handler function:

• In the init() function, we perform some initialization logic: making a database connection to DynamoDB. init() is automatically called before main().
• The addTodo handler function parses the request body for a string description.
• Then, it calls ddb.PutItem with an environment variable TODOS_TABLE_NAME to insert a new row to our DynamoDB table.
• Finally, it returns an HTTP success or error response back to the client.

Resource

Our handler function stores data in a DynamoDB table. Let’s define this table resource in the serverless.yml:

# serverless.yml

custom:
  todosTableName: ${self:service}-${self:provider.stage}-todos
  todosTableArn: # ARNs are addresses of deployed services in AWS space
    Fn::Join:
    - ":"
    - - arn
      - aws
      - dynamodb
      - Ref: AWS::Region
      - Ref: AWS::AccountId
      - table/${self:custom.todosTableName}

provider:
  ...
  environment:
    TODOS_TABLE_NAME: ${self:custom.todosTableName}
  iamRoleStatements: # Defines what other AWS services our lambda functions can access
    - Effect: Allow # Allow access to DynamoDB tables
      Action:
        - dynamodb:Scan
        - dynamodb:GetItem
        - dynamodb:PutItem
        - dynamodb:UpdateItem
        - dynamodb:DeleteItem
      Resource:
        - ${self:custom.todosTableArn}

resources:
  Resources: # Supporting AWS services
    TodosTable: # Define a new DynamoDB Table resource to store todo items
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: ${self:custom.todosTableName}
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1
        AttributeDefinitions:
          - AttributeName: id
            AttributeType: S
        KeySchema:
          - AttributeName: id
            KeyType: HASH

In the resources block, we define a new AWS::DynamoDB::Table resource using AWS CloudFormation.

We then make the provisioned table’s name available to our handler function by exposing it as an environment variable in the provider.environment block.

To give our functions access to AWS resources, we also define some IAM role statements that allow our functions to perform certain actions such as dynamodb:PutItem to our table resource.

Summary

Run ./scripts/build.sh and serverless deploy. If everything goes well, you will receive an HTTP endpoint url that you can use to trigger your Lambda function.

Verify your function by making an HTTP POST request to the URL with the following body:

{
  "description": "Hello world"
}

If everything goes well, you will receive a success 201 HTTP response and be able to see a new row in your AWS DynamoDB table via the AWS console.

Step 2: Create the GET /todos endpoint

Event

First, define the listTodos Function’s HTTP Event trigger in serverless.yml:

// serverless.yml

functions:
  listTodos:
    handler: bin/handlers/listTodos
    package:
     include:
       - ./bin/handlers/listTodos
    events:
      - http:
          path: todos
          method: get
          cors: true

Function

Create a new file within the src/handlers/ directory called listTodos.go:

// src/handlers/listTodos.go

package main

import (
	"context"
	"fmt"
	"encoding/json"
	"os"

	"github.com/aws/aws-lambda-go/lambda"
	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/dynamodb"
	"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
)

type Todo struct {
	ID          string  `json:"id"`
	Description string 	`json:"description"`
	Done        bool   	`json:"done"`
	CreatedAt   string 	`json:"created_at"`
}

type ListTodosResponse struct {
	Todos		[]Todo  `json:"todos"`
}

var ddb *dynamodb.DynamoDB
func init() {
	region := os.Getenv("AWS_REGION")
	if session, err := session.NewSession(&aws.Config{ // Use aws sdk to connect to dynamoDB
		Region: &region,
	}); err != nil {
		fmt.Println(fmt.Sprintf("Failed to connect to AWS: %s", err.Error()))
	} else {
		ddb = dynamodb.New(session) // Create DynamoDB client
	}
}

func ListTodos(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	fmt.Println("ListTodos")

	var (
		tableName = aws.String(os.Getenv("TODOS_TABLE_NAME"))
	)

	// Read from DynamoDB
	input := &dynamodb.ScanInput{
		TableName: tableName,
	}
	result, _ := ddb.Scan(input)

	// Construct todos from response
	var todos []Todo
	for _, i := range result.Items {
		todo := Todo{}
		if err := dynamodbattribute.UnmarshalMap(i, &todo); err != nil {
			fmt.Println("Failed to unmarshal")
			fmt.Println(err)
		}
		todos = append(todos, todo)
	}

	// Success HTTP response
	body, _ := json.Marshal(&ListTodosResponse{
		Todos: todos,
	})
	return events.APIGatewayProxyResponse{
		Body: string(body),
		StatusCode: 200,
	}, nil
}

func main() {
	lambda.Start(ListTodos)
}

In the above handler function:

• First, you retrieve the tableName from environment variables.
• Then, you call ddb.Scan to retrieve rows from the todos DB table.
• Finally, you return a success or error HTTP response depending on the outcome.

Summary

Run ./scripts/build.sh and serverless deploy. You will receive an HTTP endpoint url that you can use to trigger your Lambda function.

Verify your function by making an HTTP GET request to the URL. If everything goes well, you will receive a success 200 HTTP response and see a list of todo JSON objects:

> curl https://<hash>.execute-api.<region>.amazonaws.com/dev/todos
{
    "todos": [
        {
            "id": "d3e38e20-5e73-4e24-9390-2747cf5d19b5",
            "description": "buy fruits",
            "done": false,
            "created_at": "2018-01-23 08:48:21.211887436 +0000 UTC m=+0.045616262"
        },
        {
            "id": "1b580cc9-a5fa-4d29-b122-d20274537707",
            "description": "go for a run",
            "done": false,
            "created_at": "2018-01-23 10:30:25.230758674 +0000 UTC m=+0.050585237"
        }
    ]
}

Step 3: Create the PATCH /todos/{id} endpoint

Event

First, define the completeTodo Function’s HTTP Event trigger in serverless.yml:

// serverless.yml

functions:
  completeTodo:
    handler: bin/handlers/completeTodo
    package:
     include:
       - ./bin/handlers/completeTodo
    events:
      - http:
          path: todos
          method: patch
          cors: true

Function

Create a new file within the src/handlers/ directory called completeTodo.go:

package main

import (
	"fmt"
	"context"
	"os"
	"github.com/aws/aws-lambda-go/lambda"
	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/dynamodb"
	"github.com/aws/aws-sdk-go/aws"
)

var ddb *dynamodb.DynamoDB
func init() {
	region := os.Getenv("AWS_REGION")
	if session, err := session.NewSession(&aws.Config{ // Use aws sdk to connect to dynamoDB
		Region: &region,
	}); err != nil {
		fmt.Println(fmt.Sprintf("Failed to connect to AWS: %s", err.Error()))
	} else {
		ddb = dynamodb.New(session) // Create DynamoDB client
	}
}


func CompleteTodo(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	fmt.Println("CompleteTodo")

	// Parse id from request body
	var (
		id = request.PathParameters["id"]
		tableName = aws.String(os.Getenv("TODOS_TABLE_NAME"))
		done = "done"
	)

  // Update row
	input := &dynamodb.UpdateItemInput{
		Key: map[string]*dynamodb.AttributeValue{
			"id": {
				S: aws.String(id),
			},
		},
		UpdateExpression: aws.String("set# d = :d"),
		ExpressionAttributeNames: map[string]*string{
			# d": &done,
		},
		ExpressionAttributeValues: map[string]*dynamodb.AttributeValue{
			":d": {
				BOOL: aws.Bool(true),
			},
		},
		ReturnValues:     aws.String("UPDATED_NEW"),
		TableName: tableName,
	}
	_, err := ddb.UpdateItem(input)

	if err != nil {
		return events.APIGatewayProxyResponse{ // Error HTTP response
			Body: err.Error(),
			StatusCode: 500,
		}, nil
	} else {
		return events.APIGatewayProxyResponse{ // Success HTTP response
			Body: request.Body,
			StatusCode: 200,
		}, nil
	}
}

func main() {
	lambda.Start(CompleteTodo)
}

In the above handler function:

• First, you retrieve id from the request’s path parameters, and tableName from environment variables.
• Then, you call ddb.UpdateItem with both id, tableName, and UpdateExpression that sets the todo’s done column to true.
• Finally, you return a success or error HTTP response depending on the outcome.

Summary

Run ./scripts/build.sh and serverless deploy. You will receive an HTTP PATCH endpoint url that you can use to trigger the completeTodo Lambda function.

Verify your function by making an HTTP PATCH request to the /todos/{id} url, passing in a todo ID. You should see that the todo item’s done status is updated from false to true.

Step 4: Create the DELETE /todos/{id} endpoint

Event

First, define the deleteTodo Function’s HTTP Event trigger in serverless.yml:

// serverless.yml

functions:
  deleteTodo:
    handler: bin/handlers/deleteTodo
    package:
     include:
       - ./bin/handlers/deleteTodo
    events:
      - http:
          path: todos
          method: delete
          cors: true

Function

Create a new file within the src/handlers/ directory called deleteTodo.go:

package main

import (
	"fmt"
	"context"
	"os"
	"github.com/aws/aws-lambda-go/lambda"
	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/dynamodb"
	"github.com/aws/aws-sdk-go/aws"
)

var ddb *dynamodb.DynamoDB
func init() {
	region := os.Getenv("AWS_REGION")
	if session, err := session.NewSession(&aws.Config{ // Use aws sdk to connect to dynamoDB
		Region: &region,
	}); err != nil {
		fmt.Println(fmt.Sprintf("Failed to connect to AWS: %s", err.Error()))
	} else {
		ddb = dynamodb.New(session) // Create DynamoDB client
	}
}


func DeleteTodo(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	fmt.Println("DeleteTodo")

	// Parse id from request body
	var (
		id = request.PathParameters["id"]
		tableName = aws.String(os.Getenv("TODOS_TABLE_NAME"))
	)

	// Delete todo
	input := &dynamodb.DeleteItemInput{
		Key: map[string]*dynamodb.AttributeValue{
			"id": {
				S: aws.String(id),
			},
		},
		TableName: tableName,
	}
	_, err := ddb.DeleteItem(input)

	if err != nil {
		return events.APIGatewayProxyResponse{ // Error HTTP response
			Body: err.Error(),
			StatusCode: 500,
		}, nil
	} else {
		return events.APIGatewayProxyResponse{ // Success HTTP response
			StatusCode: 204,
		}, nil
	}
}

func main() {
	lambda.Start(DeleteTodo)
}

In the above handler function:

• First, you retrieve id from the request’s path parameters, and tableName from environment variables.
• Then, you call ddb.DeleteItem with both id and tableName.
• Finally, you return a success or error HTTP response depending on the outcome.

Summary

Run ./scripts/build.sh and serverless deploy. You will receive an HTTP DELETE endpoint url that you can use to trigger the completeTodo Lambda function.

Verify your function by making an HTTP DELETE request to the /todos/{id} url, passing in a todo ID. You should see that the todo item is deleted from your DB table.

In closing

Congratulations! You’ve gone serverless!

In this guide, you learned how to design and develop an API as a set of single-purpose functions, events, and resources. You also learned how to build a simple Go CRUD backend using AWS Lambda and the Serverless framework.

The final application is available on Github.

Thank you for reading!