AWS Lambdas with layers and from containers

Credits Educative.io

AWS Lambda function is great to run something small and fast. In case where you need to use some external libraries, you can use layers or the whole container. Layers are zip files that contain libraries, custom runtimes, or other dependencies. You can upload and use your own layers with your function or use layers from AWS Lambda layers library. We’ll now try all three options.

1. Mix code and libraries

First we will directly incorporate the requests library into our lambda function:

Install virtual env and requests:

python3.11 -m venv virtual_env && source virtual_env/bin/activate && pip install requests && deactivate

Create zip file:

cd virtual_env/lib/python3.11/site-packages && zip -r requests_package.zip . && cp requests_package.zip /usercode/LambdaFolder && cd /usercode/LambdaFolder

Now add lambda function and HTML template to the requests_package.zip:

zip requests_package.zip lambda_function.py template.html

We can now deploy requests_package.zip, that again has mixed code and dependencies:

aws lambda update-function-code --function-name function_call_api --zip-file fileb://requests_package.zip --no-cli-pager

Lambda function:

import requests

def lambda_handler(event, context):
    api_endpoint = 'XYZ'
    api_key = 'XYZ'

    try:
        header_parameters = {
            'Content-Type': 'application/json',
            'x-api-key': api_key,
        }

        response = requests.get(api_endpoint, headers=header_parameters)
        response.raise_for_status()  # Raise an exception for HTTP errors

        data = response.json()[0]  # Reading the first item in the list
        
        # Read the HTML template from the file
        with open('template.html', 'r') as template_file:
            html_template = template_file.read()

        # Replace placeholders with actual quote data and author name
        html_response = html_template.replace('{{QUOTE}}', data['content'])
        html_response = html_response.replace('{{AUTHOR}}', data['author'])
        
        return {
            'statusCode': 200,
            'headers': {
                'Content-Type': 'text/html',
            },
            'body': html_response
        }
    except requests.exceptions.RequestException as error:
        print(error)
        raise Exception('An error occurred while fetching data')

2. Use layers

Now we are going to do the same but instead of mixing code and libraries, we are going to push libraries to a layer:

pip install requests -t ./requestsFolder/python
cd requestsFolder
zip -r requests_layer.zip .
aws lambda publish-layer-version --layer-name requests_layer --zip-file fileb://requests_layer.zip --compatible-runtimes python3.11 --no-cli-pager
# or by specifying region, one can do the same via aws configure
aws lambda publish-layer-version --layer-name requests_layer --zip-file fileb://requests_layer.zip --compatible-runtimes python3.11 --no-cli-pager --region us-east-1

Now, we attach the layer to the lambda function. That’s it. We now have pre-installed libraries (like requests) available to the lambda function.

Side note: to get some random quote:

curl "http://api.forismatic.com/api/1.0/?method=getQuote&lang=en&format=json"

3. Use container

Let’s create Amazon Elastic Container Registry (ECR), then we install Docker in Cloud9 and push our image to ECR.

sudo apt-get update
sudo apt-get -y install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl start docker
sudo systemctl enable docker
docker --version

We now create application.py and Dockerfile and any Lambda will automatically have installed libraries specified in a Dockerfile, and code from application.py.

Now we push, this is simple using view push commands in ECR console.

We can now create lambda function from container image.