Fra bloggen, 4. februar 2021

Running Go and React on AWS using Lambda

This tutorial shows you how to use AWS Lambda and S3 bucket to build and deploy a Go API with a React frontend.

Skrevet av

Marius Myrhaug
Jobbet tidligere som senior systemutvikler i Kantega

Part 1: Getting started

For this project I'm going with the super original name "myservice"(1).
(In the actual commits you'll see "someservice555" since myservice is taken)

Prerequisites:

Let's start off with creating the project folder and initializing a new Go project. We're also adding Gin, as it facilitates running the application both locally and in AWS Lambda.

text

mkdir myservice
cd myservice
go mod init example.com/myservice
go get -t github.com/gin-gonic/gin@v1.5.0

To keep gin and routing separated from our own logic, let's create a quick routing package to handle all the Gin logic:

package routing

import (
	"github.com/gin-gonic/gin"
)

// Exporting constants to avoid hardcoding these all over, and ending up with a uppercase "POST" bug in the future.
const (
	GET = "get"
	POST = "post"
)

func Build() *gin.Engine {
	engine := gin.New()
	engine.Use(gin.Logger())
	engine.Use(gin.Recovery())
	return engine
}

func setMethodHandler(method string, path string, fn gin.HandlerFunc, group *gin.RouterGroup) {
	switch method {
	case POST:
		group.POST(path, fn)
	case GET:
		group.GET(path, fn)
	}
}

// Add a new endpoint mapping
func AddRoute(engine *gin.Engine, path string, method string, fn gin.HandlerFunc) *gin.Engine {
	group := engine.Group("/")
	setMethodHandler(method, path, fn, group)
	return engine
}

This will allow us to easily add new endpoints to our application without duplicating a lot of code or worrying about gin.

Our endpoint will for simplicity's sake be a simple "hello world":

We'll make this in api/hello/hello.go

package hello

import (
	"example.com/myservice/internal/routing"
	"github.com/gin-gonic/gin"
	"net/http"
)

const Path = "/hello"
const Method = routing.GET

func ProcessRequest(c *gin.Context) {
	name := c.Query("name")
	c.JSON(http.StatusOK, gin.H{"msg": fmt.Sprintf("Hello %v!", name)})
}

We have 3 exports from our API:

The endpoint mapping
The request type
The code to execute.

We also need to add a mapping and start our server:

package main // Important: Packages with endpoints must be named 'main'
import (
	"example.com/myservice/api/hello"
	"example.com/myservice/internal/routing"
	"log"
)

func main() {
	engine := routing.Build()
	routing.AddRoute(engine, hello.Path, hello.Method, hello.ProcessRequest)

	if err := engine.Run(); err != nil {
		log.Printf("Error starting gin %v", err)
	}

	log.Printf("Application exiting.")
}

If we start this application and call http://localhost:8080/hello we can see it successfully returns the JSON response `{"msg":"Hello!"}`

curl -i http://localhost:8080/hello
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Fri, 29 Jan 2021 18:04:39 GMT
Content-Length: 17

{"msg":"Hello!"}

To add additional endpoints simply add new file under api\ and another `AddRoute` entry.

At the end of Step 1 you should have the following: See commit

Part 2: Adding the frontend

For our frontend we'll be using React with typescript (If you do not have npx installed, this is good time to do so)

We'll create a new project named "frontend", and add the cross-env1 and getenv2 npm packages.

npx create-react-app frontend --template typescript
cd frontend
npm install cross-env
npm install getenv
npm start

We now have an empty react application running on http://localhost:3000
Let's add the integration towards our Go API.

First off we need to add our API url to package.json, changing the "start" parameter to:ace the App.tsx file with our own, adding a simple input field and a button to test our

text

"start": "cross-env REACT_APP_API=http://localhost:8080 react-scripts start",

The environmental variable REACT_APP_API will allow us to easily set the API URL depending on the different environments.
We'll get back to this later when deploying to AWS.
(Curious why it has to start with "REACT_APP_"? Read more here.)

Next up we replace the App.tsx file with our own, adding a simple input field and a button to test our API.

tsx

import React from 'react';
import './App.css';
const getenv = require('getenv');

const url = getenv.string('REACT_APP_API'); // OBS: Important! This must start with REACT_APP_
class App extends React.Component {
  state = {message: '', name: ''};

  callApi() {
    fetch(`${url}/hello?name=${encodeURIComponent(this.state.name)}`, {
        method: 'GET',
        headers: {
          'Accept': 'application/json',
          'Content-Type': 'application/json'
        }
      }
    )
      .then((response) => response.json())
      .then((response) => {
        this.setState({message: response.msg})
      });
  }

  render() {
    return (
      <div className="App">
        <header className="App-header">
          <label htmlFor="name">Name:</label>
          <input type="input" placeholder="Your name (optional)" id="name" value={this.state.name} onChange={(e) => this.setState({name: e.target.value})} />
          <br/>
          <input type="button" onClick={() => this.callApi()} value="Call API"/>
          <p>
            {this.state.message}
          </p>
        </header>
      </div>
    );
  }
}

export default App;

Open http://localhost:3000 in your browser and click the button.

Skjermdump som viser feilmeldinger i Localhost:8000

It doesn't work. Upon closer inspection in the browser developer console, we can see it's lacking CORS headers.

By adding a middleware to the API we ensure that every request gets the correct headers set.

We configure the application to look for the environmental variable called "ALLOWED_ORIGIN", which we use to specify who is allowed to call our API. In AWS this will be our S3 bucket, for local development we set it to ALLOWED_ORIGIN=http://localhost:3000 (3)

text


func Build() *gin.Engine {
	engine := gin.New()
	engine.Use(gin.Logger())
	engine.Use(gin.Recovery())
	engine.Use(CORS())
	return engine
}


func CORS() gin.HandlerFunc {
	return func(c *gin.Context) {
		allowedOrigin := os.Getenv("ALLOWED_ORIGIN")
		c.Writer.Header().Set("Access-Control-Allow-Origin", allowedOrigin)
		c.Writer.Header().Set("Access-Control-Allow-Credentials", "true")
		c.Writer.Header().Set("Access-Control-Allow-Headers", "Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With")
		c.Writer.Header().Set("Access-Control-Allow-Methods", "GET, POST, OPTIONS")

		// Handle browser preflight requests, where it asks for allowed origin.
		if c.Request.Method == "OPTIONS" {
			c.AbortWithStatus(204)
			return
		}

		c.Next()
	}
}

Skjermdump fra localhost:8000 uten feilmeldinger

Success!

We now have a fully functional API with a frontend.
All that remains now is getting it running in AWS!

At the end of Step 2 you should have the following: See commit

Part 3: Deploying to Lambda

In order to simplify deployment and resource creation in AWS, we'll be using Serverless, which is a free command line tool that can both create, upload and deploy our API and frontend with a few simple commands.

At this point I assume you have the AWS CLI installed and have authenticated successfully.

We'll also be using serverless-finch to deploy our frontend to S3, so let's go ahead and install that as well.

Serverless uses a yaml config file to define deploys, so we start by adding serverless.yml

typescript

service: myservice # Name of our service

provider:
  name: aws
  runtime: go1.x
  stage: dev
  region: us-east-1
  memorySize: 128 # 1MB increments, minimum 128, affects pricing
  environment:
    ALLOWED_ORIGIN: ${ssm:/myservice/config/allowed_origin}

package:
  exclude:
    - ./**
  include:
    - ./bin/**

# Frontend deploy config
custom:
  stage: ${opt:stage, self:provider.stage}
  client:
    bucketName: myservice-frontend # Must be globally unique, if you're setting a custom domain, then use that. (ex: example.com)
    distributionFolder: frontend/build # If you named your frontend project something else, change this here.

# Endpoint config
functions:
  Hello:
    handler: bin/aws
    events:
      - http:
          path: /hello
          method: get
          cors: true # This allows OPTION requests to pass trough the AWS API Gateway


# Frontend deploy plugin config
plugins:
  - serverless-finch
resources:
  - ${file(resources/s3-bucket.yml)}

There's a few entries here which may need to be changed:

service is the name of your service, in my case myservice.
memorySize defines how much memory you need. AWS charges for usage, so let's start with the lowest of 128mb.
ALLOWED_ORIGIN this is the environmental variable we use to set origin. We're storing this value in SSM.
bucketName is the name of our S3 bucket where we will store the frontend. This will be created for us. If using a custom domain name, set your domain name here.
distributionFolder this is the folder where the frontend is stored.
functions contains the mapping for all of your endpoints.

SSM: The name we choose is unique per region, not per applicationSo it's good practise to namespace this. I've chosen `/myservice/config/allowed_origin` as the name to store in SSM.

We also need our serverless-finch configuration for S3, which we'll store in resources\s3-bucket.yml:

typescript

Resources:
  AttachmentsBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: ${self:custom.client.bucketName}
      CorsConfiguration:
        CorsRules:
          -
            AllowedOrigins:
              - '*'
            AllowedHeaders:
              - '*'
            AllowedMethods:
              - GET
              - PUT
              - POST
              - DELETE
              - HEAD
            MaxAge: 3000

# Print out the name of the bucket that is created
Outputs:
  AttachmentsBucketName:
    Value:
      Ref: AttachmentsBucket

This is pretty boilerplate, and we won't need to change anything in this file. It reads the bucket name from serverless.yml via `${self:custom.client.bucketName}`.

Let's go ahead and install our AWS dependencies for Go:

text

go get -t github.com/aws/aws-lambda-go
go get -t github.com/awslabs/aws-lambda-go-api-proxy

We'll also be adding a simple script to build our application:
Makefile (Linux) / Batch script (Windows)

Take note of the `-o bin/aws`, as it's the binary we're referencing in `serverless.yml`.
It's also important that it's compiled for target=linux regardless of your local environment, as it's what AWS Lambda expects.

Now all we need to do is add endpoints in a way that Lambda understands them:

package main

import (
	"example.com/someservice555/api/hello"
	"example.com/someservice555/internal/routing"
	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-lambda-go/lambda"
	"github.com/awslabs/aws-lambda-go-api-proxy/gin"
)

func main() {
	engine := routing.Build()

	// You may notice that this is identical to our localdev setup. 
	// As you add more endpoints, you may wish to store the endpoints in a separate list, used by both localdev and aws.
	routing.AddRoute(engine, hello.Path, hello.Method, hello.ProcessRequest)

	proxy := func(req events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
		adapter := ginadapter.New(engine)
		return adapter.Proxy(req)
	}

	lambda.Start(proxy)
}

That's it, we're ready to deploy this to AWS!

In the myservice folder, run: `sls deploy --verbose`

typescript

myservice>sls deploy

 Serverless Warning --------------------------------------

  A valid SSM parameter to satisfy the declaration 'ssm:/myservice/config/allowed_origin' could not be found.

Serverless: Deprecation warning: Starting with next major version, API Gateway naming will be changed from "{stage}-{service}" to "{service}-{stage}".
            Set "provider.apiGateway.shouldStartNameWithService" to "true" to adapt to the new behavior now.
            More Info: https://www.serverless.com/framework/docs/deprecations/#AWS_API_GATEWAY_NAME_STARTING_WITH_SERVICE
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Creating Stack...
Serverless: Checking Stack create progress...
........
Serverless: Stack create finished...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service myservice.zip file to S3 (4.9 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
....................................
Serverless: Stack update finished...
Service Information
service: myservice
stage: dev
region: us-east-1
stack: myservice-dev
resources: 13
api keys:
  None
endpoints:
  GET - https://xh517uj5a9.execute-api.us-east-1.amazonaws.com/dev/hello
functions:
  Hello: myservice-dev-Hello
layers:
  None

*****************************************************************************************************************************************************************************Serverless: Announcing an enhanced experience for Serverless Websites. Easy, cheap, scalable hosting for React, Vue.js apps and more: https://github.com/serverless-components/website
*****************************************************************************************************************************************************************************

As you can see our deploy finished successfully, and the url of our new API is "https://xh517uj5a9.execute-api.us-east-1.amazonaws.com/dev/hello"

You can also see a warning that 'ssm:/myservice/config/allowed_origin' could not be found, but we'll set that soon.

We can call the endpoint and verify that its running as expected:

curl -i https://xh517uj5a9.execute-api.us-east-1.amazonaws.com/dev/hello?name=Pete
HTTP/2 200
content-type: application/json; charset=utf-8
access-control-allow-origin:
access-control-allow-headers: Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With
access-control-allow-methods: GET, POST, OPTIONS
access-control-allow-credentials: true

And "access-control-allow-origin" is of course missing, since we've not yet set the SSM variable.

Part 4: Deploying the frontend

Now that we've deployed our backend to Lambda, and know the URL of our API, let's start with updating package.json:

text


  "scripts": {
    "start": "cross-env REACT_APP_API=http://localhost:8080 react-scripts start",
    "build:local": "cross-env REACT_APP_API=http://localhost:8080 react-scripts build",
    "build": "cross-env REACT_APP_API=https://xh517uj5a9.execute-api.us-east-1.amazonaws.com react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },

Running `npm run build` should now produce a deployable build which points towards our API running in AWS Lambda.

Let's go ahead and build it now with `npm run build` inside the frontend folder.
After the build has finished, return to the main project folder (myservice) and run `sls client deploy` which will deploy out frontend project. We already defined the folder in `serverless.yml`, so it knows to look in frontend\build.

typescript

> sls client deploy
Serverless: Looking for bucket...
Serverless: Bucket found...
Serverless: Deleting all objects from bucket...
Serverless: Configuring bucket...
Serverless: Configuring policy for bucket...
Serverless: Retaining existing tags...
Serverless: Configuring CORS for bucket...
Serverless: Uploading client files to bucket...
Serverless: Success! Your site should be available at http://myservice-frontend.s3-website-us-east-1.amazonaws.com/

As we can see the frontend was successfully deployed to http://myservice-frontend.s3-website-us-east-1.amazonaws.com/

If we visit our website, we can see that it loads correctly, and the button obviously fails, as we've not yet set our allowed origin.

Let's finish up and set our origin with:
`aws ssm put-parameter --name /myservice/config/allowed_origin --type String --value myservice-frontend.s3-website-us-east-1.amazonaws.com`

Finally redeploy our Go API with `sls deploy`.
The website is now up and running with a fully functional API!

At the end of Step 3 and 4 you should have the following: See commit

Going forwards

Modifying the API and deploying changes:

When modifying the Go API: Build it and redeploy with `sls deploy`.
When modifying the frontend, run `npm run build` in the frontend folder, followed by `sls client deploy` in the project folder.

Unsure how to add more endpoints? See an example!

Gotchas / Common problems

Your lambda function is running behind the Amazon API Gateway, and serverless is creating mappings for us in the background. If you accidentally mix up GET and POST, the API will not return "405 Method Not Allowed", instead it will return 403, usually with a JSON response referring to credentials or missing authentication tokens.
The API Gateway has a maximum response size of 10MB, exceeding this limit will return http 502/503.
Serverless does not by default enable gzip compression in the API Gateway. See the serverless-content-compression plugin for enabling gzip compression.
By default the API Gateway assumes the content type is text (json/html/xml/etc) and will mangle the response if you return binary data. This can be fixed by manually setting it as binary in the API Gateway or you can try a plugin such as serverless-apigw-binary.

Links/citations:

1) Naming convention: Ideally you should avoid uppercase letters, underscores and dashes in your project name, as you'll eventually run into a cloud service which does not allow it.
2) cross-env package allows us to easily run the application on both linux and windows without changing anything.
3) getenv allows us to easily fetch environmental variables, which we'll use to get the server URL
4) How you set the environmental variable locally depends on how you're running the project.