Build Scalable Applications with AWS Lambda and Go

The way we design, build and deploy web applications has changed a lot in the past few years. In the beginning, It all seemed like a simple task, our code lived all inside a big folder with a bunch of files in it, then we just uploaded the files using FTP and hoped everything would work out flawlessly. If something went wrong it was hard to keep the track of errors in an overwhelming number of lines of legacy code.

Deploying and managing our application was a time-consuming task that required us to be aware of the underlying infrastructure, servers that needed to be upgraded as our application and business growth, and all of this time that we spend on these tasks could have been spent on developing new features and improving our application instead.

Nowadays it has become easier to design and deploy applications that are able to scale, even automatically, as your business grows. With the rise of distributed systems and microservices, it is not necessary anymore to have all of your code and logic all in one place, we can rely on event-driven architecture to communicate between decoupled services.

We are going to take a brief overview on how AWS Lambda and the Serverless Framework work together to take advantage of all of these new ways of building scalable applications, all of this on top of the Go programming language, which brings us even more advantages while developing our application.

The Rise of Cloud Computing

If you think about it, everything has been "in the cloud" since the beginning, but the way we understand the term "Cloud Computing" has changed with the rise of new technologies. Back in the day, you had to purchase a server, no matter if it was virtual or physical, you had to guess how much resources you were going to need and pay everything up front, and later if needed more space, memory, or resources you had to purchase them and upgrade your server.

Today "Cloud Computing" is the way we deliver computing services (servers, databases, storage, software, resources, networking) over the internet to build applications that can offer faster innovation, flexible resources and that can scale you, this means that you will lower your operating cost because you only pay for the services you use, and you run your infrastructure more efficiently because it scales as your business needs to change.

Top benefits of Cloud Computing

Cost savings: you only pay for IT as you consume it
Elasticity: no need to over-provision resources, you can scale those resources instantly as your business needs change
Agility: you have access to a broad range of technologies, so you can innovate faster and build nearly anything you want

There are several Cloud Providers out there that offer great services, some of them by well-known companies, like Microsoft, Google, and Amazon. One of the most used is Amazon AWS, which has a lot of services to choose from, being AWS Lambda one of them.

Distributed Systems

It is very likely that you have been using distributed systems and you don't even know about it. On their most simple definition, a distributed system is a group of computers working together that share messages with each other to achieve a common goal. All of this will appear as it is a single computer to the end-user.

To better understand this, we can bring back the example where you upload all of your application code to a server, here your code lives on a single computer (virtual or physical), all of the business logic is inside of it and all of the components communicate within each other, if something fails on the infrastructure, all of the application goes down.

The distributed system allows us, in a certain way, to split our business logic into different components, they can be on different physical servers, virtual machines, or containers, and most likely located in different locations. So, by doing this, the system can maximize resources and information while preventing failures, so if one system fails, it won't affect the availability of the entire service.

Advantages of Distributed Systems

Horizontal Scaling: it is easy and inexpensive to add additional machines as necessary
Fault Tolerance: if one server or data center fails, others could still work without affecting the whole system
Performance: they are very efficient because workloads can be broken and sent to multiple machines

Most modern web applications are based upon distributed systems, building a scalable application heavily relies on this concept, even more, if it is built on top of any cloud provider services.

From Monolith to Microservices

Nowadays monolithic applications are still the backbone of most large organizations, but they are often tied to complex processes, which make them really fragile to changes on the codebase. They are built as a single unit where all of the components coexist within the same application, so any changes to the system involve building and deploying a new version of the entire application.

One of the modern and current software paradigms is to build smaller, independently deployable units of code, so you can break down the application into smaller services by applying the single responsibility principle. You design your system so each module is a standalone microservice responsible for a distinct business domain.

Designing your application as microservices is something that will give you great benefits if your plan is to build a scalable application that leverages the use of cloud computing and distributed systems.

FaaS (Function-as-a-Service)

FaaS or "Function as a Service" is a type of Cloud Computing service that allows you to execute code in response to events without thinking about the complexity of the underlying infrastructure typically associated when you are deploying microservices applications. Provisioning and managing the infrastructure is handled automatically by your Cloud Service provider, so you can focus only on the individual function in your code and it allows you to create new applications composed of tiny building blocks.

Benefits of FaaS

Focus more on the code: you can have to worry about infrastructure, this allows you to focus on the application code
Scale automatically: functions are scaled automatically, independently, and really fast
Pay only for the resources you use: you pay only when the function is executed, and usually, this is really cheap

AWS Lambda is the Function-as-a-Service option provided by Amazon AWS, it gives us all of these benefits plus the convenience of being on the AWS platform, which is very useful if you want to take advantage of all of the other services.

Event-Driven Architecture

AWS Lambda relies pretty much on the Event-Driven Architecture. This type of architecture allows your application to detect events whenever you want to trigger any business logic and do an action in response to that. This replaces the traditional "request/response" pattern where services would have to wait for a reply before they can move on to the next task.

An event can be anything, it can go from an item added to a cart, an order that was placed, an image uploaded to a server, a record updated on the database. This type of pattern uses events to trigger and communicate between services and it is very common in modern web applications that are built on top of microservices.

The future is Serverless

Serverless is a software development approach that eliminates the need to manage infrastructure by using Cloud Computing services (Functions-as-a-Service) to execute the code and external services and APIs to leverage the application.

We have to take into consideration that the term "Serverless" does not mean that servers are no longer being involved, it simply means that developers no longer need to be aware of the underlying infrastructure, computing resources are used as services, without managing physical capacities or limits.

The Serverless Framework

The Serverless Framework makes it easy to write event-driven functions for various providers, including AWS. For each provider, a series of events can be configured to invoke the function. All of this is done programmatically by editing a YAML configuration file and running some commands on the terminal. Also, you can integrate this with existing Continuous Integration and Continuous Delivery frameworks.

This really makes the task of deploying and updating the code of our function really easy and fast. Also, when you create your serverless project there are plenty of templates ready for you if you want to use them, so you don't have to start from scratch. Particularly on AWS you can accomplish this by using the AWS console and doing everything from there, but every time you want to update your code you will have to compress it on a zip file and upload it with the AWS console, also there are other things that you have to take into consideration, like IAM users, roles and policies.

The Serverless framework takes care of all of this, so you can focus just on the code.

AWS Lambda

AWS Lambda is an event-driven serverless computing platform. It lets you run your code without provisioning or managing servers and it runs your code only when needed and scales automatically, you pay only for the compute time that you consume, so they don't charge you when nothing is running.

This is how it works

Currently, AWS Lambda supports several programming languages through the use of runtimes, like node.js, Python, Ruby, Java, and Go. To use other languages in Lambda you can implement a custom runtime, so the possibilities are endless.

If you are using AWS Lambda, you can also take advantage of all of the other AWS Cloud Services. You can trigger events when a photo is uploaded to Amazon S3, listen to streams of data from Amazon Kinesis, trigger a function when data gets added to DynamoDB, or even use Amazon API Gateway to build a serverless REST API. The possibilities are endless.

Pricing

You only pay for what you use and charged based on the number of requests and duration of your function. We are not going to deep dive into this topic, but rest assured that it's ridiculously cheap to run a Lambda function, even with the "Free Tier" you can play around a lot and probably run a small service. FOr more information, you can take a look at the AWS Lambda Pricing page.

When should I use AWS Lambda?

First, you have to think in an Event-Driven way. If there's a feature on your application that needs to be run in response to an event, and that feature does not need to be running all of the time, then probably that is a good use case for AWS Lamba, here are some examples:

Rapid document conversion
Image resizing and processing
Handling uploaded AWS S3 objects
Operating serverless websites
Automated backups and everyday tasks
Real-time data processing

Why Go?

Go is without a doubt one of the fastest-growing programming languages today, it has been adopted by a large number of companies, but is has been really active in the Cloud Computing scenario. This is why AWS, one of the biggest Cloud Computing providers, has added support for Go in AWS Lambda since the beginning of 2018.

Some of the major advantages of using Go with AWS Lambda are:

Runtime versioning: the Go runtime for AWS Lambda supports any 1.x release, so whenever a new version of Go is released it is possible to use it right away, without having to wait for support on the updated runtime
Concurrency: concurrency was built into the core design of Go, this allows us to create more efficient functions with the great benefit of Goroutines
Pricing: the pricing model for AWS Lambda is per invocation, plus the duration of the invocation rounded up to the nearest 100ms. Go is not as memory hungry as other dynamic languages, and there's a direct relation between CPU performance and the allocated memory, so this makes it very efficient

Sample Application

To get a better idea of how AWS Lambda and the Serverless Framework work together we are going to build a simple function that just exposes an HTTP endpoint that receives a parameter on the URL and returns a JSON response. Nothing trivial, but I'm a true believer that with small and simple examples you understand a new concept better, also by doing this you will see how easy and quick is to build and deploy a Lambda function with the framework.

Prerequisites

Create an AWS Account
Create an IAM User on AWS Console (save the access key and secret key)
Install Go
Install Serverless (you will need to install node.js first)
```
$ npm install -g serverless
```

Function Code

The code is pretty straightforward, most of the Lambda functions are build following the same pattern, it also depends on which AWS services and integrations you are using, also the purpose of your function. The most important aspect here is that you are always going to need a Handler function, which is the one that takes care of all of the logic, and this is the one that you call on your main() function.

package main

import (
    "fmt"

    "github.com/aws/aws-lambda-go/events"
    "github.com/aws/aws-lambda-go/lambda"
)

// This function is the one that is going to be called
func Handler(request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {

    // Get the 'name' parameter from the URL
    name := request.PathParameters["name"]

    // Create the message that is going to be returned
    message := fmt.Sprintf(" { \"Message\" : \"AWS Lambda is Awesome! Right %s? \" } ", name)

    // Return the response 
    return events.APIGatewayProxyResponse{Body: message, StatusCode: 200}, nil
}

func main() {
  // Start the Handler
    lambda.Start(Handler)
}

Serverless Framework Setup

Configure Serverless: replace with your access key and secret key. serverless-admin is going to be the name of your profile, this can be whatever you want

$ serverless config credentials --provider aws --key <your_key> --secret <your_secret> --profile serverless-admin

Create the serverless.yml file (for full configuration options you can check the documentation)

yaml
   service: lambda-sample

   frameworkVersion: '2'

   provider:
     name: aws
     runtime: go1.x
     lambdaHashingVersion: 20201221
     profile: serverless-admin
     region: us-east-2

   package:
     exclude:
       - ./**
     include:
       - ./bin/**

   functions:
     hello:
       handler: bin/hello
       events:
         - http:
             path: hello/{name}
             method: get

Place the function code inside hello/main.go
Build the app

$ env GOOS=linux go build -ldflags="-s -w" -o bin/hello hello/main.go

Deploy the app

$ sls deploy -v
   Serverless: Packaging service...
   Serverless: Excluding development dependencies...
   Serverless: Creating Stack...
   Serverless: Checking Stack create progress...
   CloudFormation - CREATE_IN_PROGRESS - AWS::CloudFormation::Stack - hello-dev
   CloudFormation - CREATE_IN_PROGRESS - AWS::S3::Bucket - ServerlessDeploymentBucket
   CloudFormation - CREATE_IN_PROGRESS - AWS::S3::Bucket - ServerlessDeploymentBucket
   CloudFormation - CREATE_COMPLETE - AWS::S3::Bucket - ServerlessDeploymentBucket
   CloudFormation - CREATE_IN_PROGRESS - AWS::S3::BucketPolicy - ServerlessDeploymentBucketPolicy
   CloudFormation - CREATE_IN_PROGRESS - AWS::S3::BucketPolicy - ServerlessDeploymentBucketPolicy
   CloudFormation - CREATE_COMPLETE - AWS::S3::BucketPolicy - ServerlessDeploymentBucketPolicy
   CloudFormation - CREATE_COMPLETE - AWS::CloudFormation::Stack - hello-dev
   Serverless: Stack create finished...
   Serverless: Uploading CloudFormation file to S3...
   Serverless: Uploading artifacts...
   Serverless: Uploading service hello.zip file to S3 (2.76 MB)...
   Serverless: Validating template...
   Serverless: Updating Stack...
   Serverless: Checking Stack update progress...
   CloudFormation - UPDATE_IN_PROGRESS - AWS::CloudFormation::Stack - hello-dev
   CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
   CloudFormation - CREATE_IN_PROGRESS - AWS::Logs::LogGroup - HelloLogGroup
   CloudFormation - CREATE_IN_PROGRESS - AWS::ApiGateway::RestApi - ApiGatewayRestApi
   CloudFormation - CREATE_IN_PROGRESS - AWS::ApiGateway::RestApi - ApiGatewayRestApi
   CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
   CloudFormation - CREATE_COMPLETE - AWS::ApiGateway::RestApi - ApiGatewayRestApi
   CloudFormation - CREATE_IN_PROGRESS - AWS::Logs::LogGroup - HelloLogGroup
   CloudFormation - CREATE_COMPLETE - AWS::Logs::LogGroup - HelloLogGroup
   CloudFormation - CREATE_IN_PROGRESS - AWS::ApiGateway::Resource - ApiGatewayResourceHello
   CloudFormation - CREATE_IN_PROGRESS - AWS::ApiGateway::Resource - ApiGatewayResourceHello
   CloudFormation - CREATE_COMPLETE - AWS::ApiGateway::Resource - ApiGatewayResourceHello
   CloudFormation - CREATE_IN_PROGRESS - AWS::ApiGateway::Resource - ApiGatewayResourceHelloNameVar
   CloudFormation - CREATE_IN_PROGRESS - AWS::ApiGateway::Resource - ApiGatewayResourceHelloNameVar
   CloudFormation - CREATE_COMPLETE - AWS::ApiGateway::Resource - ApiGatewayResourceHelloNameVar
   CloudFormation - CREATE_COMPLETE - AWS::IAM::Role - IamRoleLambdaExecution
   CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Function - HelloLambdaFunction
   CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Function - HelloLambdaFunction
   CloudFormation - CREATE_COMPLETE - AWS::Lambda::Function - HelloLambdaFunction
   CloudFormation - CREATE_IN_PROGRESS - AWS::ApiGateway::Method - ApiGatewayMethodHelloNameVarGet
   CloudFormation - CREATE_IN_PROGRESS - AWS::ApiGateway::Method - ApiGatewayMethodHelloNameVarGet
   CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Permission - HelloLambdaPermissionApiGateway
   CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Version - HelloLambdaVersion0llM5cHtVxK5BhSODXsBnppzF2TM1roM6OfXw2qXINc
   CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Permission - HelloLambdaPermissionApiGateway
   CloudFormation - CREATE_COMPLETE - AWS::ApiGateway::Method - ApiGatewayMethodHelloNameVarGet
   CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Version - HelloLambdaVersion0llM5cHtVxK5BhSODXsBnppzF2TM1roM6OfXw2qXINc
   CloudFormation - CREATE_COMPLETE - AWS::Lambda::Version - HelloLambdaVersion0llM5cHtVxK5BhSODXsBnppzF2TM1roM6OfXw2qXINc
   CloudFormation - CREATE_IN_PROGRESS - AWS::ApiGateway::Deployment - ApiGatewayDeployment1611337803492
   CloudFormation - CREATE_IN_PROGRESS - AWS::ApiGateway::Deployment - ApiGatewayDeployment1611337803492
   CloudFormation - CREATE_COMPLETE - AWS::ApiGateway::Deployment - ApiGatewayDeployment1611337803492
   CloudFormation - CREATE_COMPLETE - AWS::Lambda::Permission - HelloLambdaPermissionApiGateway
   CloudFormation - UPDATE_COMPLETE_CLEANUP_IN_PROGRESS - AWS::CloudFormation::Stack - hello-dev
   CloudFormation - UPDATE_COMPLETE - AWS::CloudFormation::Stack - hello-dev
   Serverless: Stack update finished...
   Service Information
   service: hello
   stage: dev
   region: us-east-2
   stack: hello-dev
   resources: 12
   api keys:
     None
   endpoints:
     GET - https://upr8kq3qsc.execute-api.us-east-2.amazonaws.com/dev/hello/{name}
   functions:
     hello: hello-dev-hello
   layers:
     None

   Stack Outputs
   HelloLambdaFunctionQualifiedArn: arn:aws:lambda:us-east-2:132653656301:function:hello-dev-hello:1
   ServiceEndpoint: https://upr8kq3qsc.execute-api.us-east-2.amazonaws.com/dev
   ServerlessDeploymentBucketName: hello-dev-serverlessdeploymentbucket-rdd115w8vsw1

And that is it! As you can see the deploy output returns the URL where your endpoint is, in this case, https://upr8kq3qsc.execute-api.us-east-2.amazonaws.com/dev/hello/{name} (where you replace {name} with what you want to send to the function). If you visit that URL from your browser you will get:

{
     Message: "AWS Lambda is Awesome! Right, Ariel? "
}

Additional Notes

As you can see the only thing you needed was the serverless.yml configuration file to deploy your function. In this example everything was created from scratch, but I encourage you to try out any of the sample templates that are generated for you, they even come with some boilerplate code, so you can play around with them. You can generate an AWS Go template running this command:

$ sls create --template aws-go --path sample
  Serverless: Generating boilerplate...
  Serverless: Generating boilerplate in "/code/lambda/sample"
   _______                             __
  |   _   .-----.----.--.--.-----.----|  .-----.-----.-----.
  |   |___|  -__|   _|  |  |  -__|   _|  |  -__|__ --|__ --|
  |____   |_____|__|  \___/|_____|__| |__|_____|_____|_____|
  |   |   |             The Serverless Application Framework
  |       |                           serverless.com, v2.19.0
   -------'

  Serverless: Successfully generated boilerplate for template: "aws-go"

When you use a template, a Makefile is generated for you, so you can build your app with this command:

$ make build

You can also build and deploy at the same time

$ make deploy

Only deploy the function. It usually takes some time when you deploy your entire app, but you have to do it only once unless you are adding new dependencies. Remember to build with make build first

$  sls deploy function -f hello

View the logs

$ sls logs -f hello -t

And finally, remove the function

$ sls remove

AWS Lambda Opens Possibilities

AWS Lambda is powering a lot of enterprises around the world. From running cloud platforms to extending legacy applications. In this article, we only cover the top of the iceberg, but there are still a lot of features, integrations, and services that can be used with it. I encourage you to explore and read more about what you can do with AWS Lambda and how can you build awesome and scalable web applications.

Build Scalable Applications with AWS Lambda and Go

Comments

More from this blog

Best Practices for Writing Clean and Efficient Go Code

Testing in Golang

Optimizing Performance in Go Applications

Error Handling in Go: Best Practices and Patterns

Building Microservices in Go: A Practical Guide with gRPC

The Rise of Cloud Computing

Top benefits of Cloud Computing

Distributed Systems

Advantages of Distributed Systems

From Monolith to Microservices

FaaS (Function-as-a-Service)

Benefits of FaaS

Event-Driven Architecture

The future is Serverless

The Serverless Framework

AWS Lambda

Pricing

When should I use AWS Lambda?

Why Go?

Sample Application

Prerequisites

Function Code

Serverless Framework Setup

Additional Notes

AWS Lambda Opens Possibilities

Command Palette

Comments

More from this blog

The Rise of Cloud Computing

Top benefits of Cloud Computing

Distributed Systems

Advantages of Distributed Systems

From Monolith to Microservices

FaaS (Function-as-a-Service)

Benefits of FaaS

Event-Driven Architecture

The future is Serverless

The Serverless Framework

AWS Lambda

Pricing

When should I use AWS Lambda?

Why Go?

Sample Application

Prerequisites

Function Code

Serverless Framework Setup

Additional Notes

AWS Lambda Opens Possibilities