An ultimate guide developing lambda functions locally for AWS Lambda

About AWS Lambda

You can skip this if you are already aware of AWS Lambda and jump to the next section that elaborates on the local development.

In Serverless space AWS Lambda is one of the most important and well-known service that allows you to execute a piece of code based on a variety of events.

AWS supports a handfull of programming languages to write code called as a “lambda function” that can be invoked based on events. More details about the programming languages here.

AWS Lambda - FAQs

And here is the list of services / event-types that can be used to trigger a lambda-function

Using AWS Lambda with other services

Why AWS Lambda

So, the stage is set with sufficient evidence that lambda functions if done right are super-powerful . To put in context the extent of scalability and power of lambda functions , do checkout the video below on a Netflix case-study on AWS Lambda. I’m sure that’s sufficient to convince you how powerful lambda functions if they can support a service like Netflix that has millions of user watching on-demand videos.

Remember the catch of limited control I referred to. When you go serverless , there’s no physical server available for you to troubleshoot. Then how is your code executed ? That’s the magic achieved by AWS.

AWS essentially provisions a small custom built image that has all the necessary packages to run your code ( which is hosted on AWS Lambda ) and as soon as the code is executed , the custom image is destroyed . Thus all environment variables that you may have used within the execution environment are also lost. Much similar to a “docker”? Hang on to that thought.

And that’s why AWS Lambda funtions are also called “state-less” functions .

So how do we debug and test a Lambda function ?. As I said earlier, the execution environment for Lambda is a customized image focussed on secure code execution with very limited (call bare-minimum modules ).

LambCI

Docker Hub

A sandboxed local environment that replicates the live AWS Lambda environment almost identically — including installed software and libraries, file structure and permissions, environment variables, context objects and behaviors — even the user and running process are the same. Voila!! ( docker again )

You may ask why should I go through all this pain when I can simply build a docker image of my own specifications and deploy it ? Because, indeed AWS has recently announced support for lambda container images which is simply a docker image built for your own specifications being hosted on lambda and comply to all the invocation support for AWS Lambda.

Creating Lambda container images

A few challenges one might face with building custom docker images for AWS Lambda execution environment in my view are

1. Building and maintaining slim images to contain the runtime / compiler for your language
2. OS related security aspects in the underlying images
3. Size of the resultant image which is to be hosted on Lambda runtime ( remember the larger the size , the longer it takes to invoke ).

And each one of these are very important in a production environment when you want your functions to scale in subsecond latency.

So it makes a lot of sense to just host your code and let the execution be taken care by the custom Amazon built image .

Now let’s see how we can locally test a lambda function with lambci docker image.

Requirements for this tutorial :

a. Docker installed in your workstation

b. dockerhub ID setup . Just signup with dockerhub.

Note: I’m using Python as my environment. If you are using any other run-time you may have to adapt your script and docker execution commands .For more details please check .

GitHub - lambci/docker-lambda: Docker images and test runners that replicate the live AWS Lambda…

Steps to test function locally

1. Pull the image for Python 3.8 with the following command.

docker pull lambci/lambda:build-python3.8

Now you can see the image in your local machine with the following command

docker image ls

and you should see the image ( refer to the last image )

3. Let’s test a simple script ( below )to return a Hello world when executed.

Save the script above into the folder lambcitesting under the name “lambda_function.py”

md lambcitesting
cd lambcitesting

Now run the following docker command to invoke the function.

#if you are proficient in Linux you can simply use the pwd to refer to the current working directory
docker run --rm -v "$PWD":/var/task:ro,delegated lambci/lambda:python3.8 lambda_function.lambda_handler
#WINDOWS - Replace the <yourpath
docker run — rm -v “<yourpath>/lambcitesting”:/var/task:ro,delegated lambci/lambda:python3.8 lambda_function.lambda_handler

Observe the following syntax in the command. This specificies the filename and the function name .

Remember that is is not mandatory to keep your file name and function name the same . You can change it . But ensure that the command refers to the correct python file and the function name . This applies even in the AWS Lambda Runtime setup.

File Name : lambda_function.py

Function Name : lambda_handler

lambda_function.lambda_handler

Now when you execute the docker run command you should get the following output.

Observe something similiar ?. Yup , it mimics the lambda execution runtime .Because that’s what it is designed to do.

4. Now let’s a slightly complex example by passing a payload in json format

Now run the command with a minor change , a json payload being passed to the lambda

sudo docker run --rm -v "$PWD":/var/task:ro,delegated lambci/lambda:python3.8 lambda_function.lambda_handler '{"first_name":"Rajesh","last_name":"Rajamani"}'

And you get the message which processed the json payload you passed as request.

So far we saw how to setup the docker image and test a simple script and a complex script including a payload.

Enhancing the run-time environment

As I told earlier , the lambci and the original lambda environment contains a very limited selection of modules to run your scripts be it any language.

But that does not mean you cannot have external modules. For example the lambda environment for Python does not contain the “requests” module.

So if you attempt to import the requests module and execute the script you get an error

Observe the last line in the execution below

So we know that requests module is not present. Here we have a two options.Either install the requests module or use an alternate module that can suffice the functionalities of requests module such as http.

But that’s for another post stay tuned .

I’d like to thank the following Github user for putting together the code that allows you to enlist all the modules in the lambci python environment . So with this you know what packages to expect off the shelf.

gene1wood's gists