Creating an API Gateway using the AWS CLI

Introduction

In a previous post, I explained how to make a Lambda function using only the CLI.

I always preferred using the CLI as companies like Amazon love updating their UX frequently and ruining my workflow, since then, I have moved on to using tools like the Vercel, Terraform and recently Flightcontrol instead of using the CLI but in my opinion, it is still worth understanding what operations are happening under the hood with these services as they all rely on the AWS CLI.

Another thing is that the AWS CLI documentation was always a pain and that’s why I feel like my old blog did well as the documentation was so poorly written that there was an audience for people like me to write blogs about doing simple tasks.

However, now that ChatGPT is here. Asking it to do things with the AWS CLI is a breeze and to explain individual commands is a breeze. However, it is not perfect and I have found that relying on humans to verify AI leads to much better results.

For example, in my series last year on using 1,000 bots in FPL to see how I did, I was consistently in the bottom 50%. However this year, using my Team Recommender’s artificial intelligence and my brain, I am currently in the top 0.001% (5k out of 11 million) and was even in the top 10 at Christmas! Check me out on YouTube if you want to follow my weekly journey.

I have peaked in FPL

So even though it was lovely talking to ChatGPT about this and I recommend using it for all things AWS CLI related, it still made the equivalent mistake of picking injured players now and then and I had to fill in the blanks.

It is a bit like reading the news, it all seems fine until you read something you know about and then you notice simple mistakes.

To start off, AWS offers a variety of services to build and deploy APIs quickly and easily. One of these services is Amazon API Gateway. It allows developers to create, publish, and manage APIs with ease, faster and cheaper than services like Django, Flask or FastAPI. In this tutorial, we will use the AWS CLI to create an API Gateway that triggers a simple Lambda function.

Before we begin, ensure that you have an AWS account and the AWS CLI installed on your system. If you haven't done so already, you can install the CLI here.

Step 1: Set the AWS region and profile

To begin, set the AWS region and profile that you will be using. In this example, we will use the "eu-west-1" region and "personal-key" profile. I like to use different profiles so I can’t do any serious damage when I am doing a simple personal project vs doing something serious.

#!/bin/bash

# Set the AWS region and profile
region="eu-west-1"
profile="personal-project"

Step 2: Make a simple Python script for the Lambda

Next, we need to create a simple Python script that will serve as our Lambda function. For the purposes of this example, we will create a function that retrieves the top scorer for a given game week of the Fantasy Premier League. Here is the code for the Python script:

import requests
from datetime import datetime, timedelta
import json

def lambda_handler(event, context):
    game_week = int(event['queryStringParameters']['gameweek'])
    url = f"https://fantasy.premierleague.com/api/event/{game_week}/live/"
    
    # Get the top scores for the specified game week
    response = requests.get(url)
    top_scores = sorted(response.json()["elements"], key=lambda x: x["stats"]["total_points"], reverse=True)[:1]
    
    # Get the player's name and total points
    player_name = top_scores[0]['id']
    total_points = top_scores[0]['stats']['total_points']
    
    # Create the API response
    response_body = {
        "player_name": player_name,
        "total_points": total_points
    }
    
    # Return the API response
    return {
        "statusCode": 200,
        "headers": {
            "Content-Type": "application/json"
        },
        "body": json.dumps(response_body)
    }

The FPL API is extremely robust and very handy for doing tutorials like this as there is no retrieving an API key or making a profile.

Step 3: Make a Requirements File for the Layer

Before we can create our Lambda function, we need to create a Lambda Layer that contains the required dependencies. A Lambda Layer defines the environment and lets us use the libraries we need to.

First, we need a requirements file, we will use the well known pip freeze command to generate our requirements.txt.

pip freeze > requirements.txt

Step 4: Create the Layer

Now that we have our dependencies listed in a requirements file, we can create the Lambda layer. We can do this using the following code:

layer_name="mylayer"
layer_zip="mylayer.zip"
pip install -r requirements.txt -t python/
cd python
zip -r ../$layer_zip .
cd ..
aws lambda publish-layer-version --layer-name $layer_name --description "Python dependencies" --zip-file fileb://$layer_zip --compatible-runtimes python3.8 --region $region --profile $profile
layer_arn=$(aws lambda list-layer-versions --layer-name $layer_name --query 'LayerVersions[0].LayerVersionArn' --output text --region $region --profile $profile)

aws lambda publish-layer-version is a command used to create a new version of a Lambda layer. This command uploads the contents of the layer archive file (in this case, mylayer.zip) to AWS, and creates a new layer version that can be referenced by AWS Lambda functions.

Here’s a breakdown of the flags used in the script:

• --layer-name: The name of the layer to create/update. In our case, we used the name mylayer.
• --description: A description of the layer. We used "Python dependencies" as the description for our layer.
• --zip-file: The path to the zip file containing the layer content. We used fileb://$layer_zip to specify the location of our mylayer.zip file.
• --compatible-runtimes: A list of AWS Lambda runtime environments compatible with the layer. We specified python3.8 as the runtime environment for our layer.
• --region: The AWS region in which to create/update the layer.
• --profile: The name of the AWS CLI profile to use.

Step 5: Publish the Lambda Function

Once the layer is created, the next step is to publish the Lambda function. Here’s the code to create the Lambda function using the AWS CLI:

# Create the Lambda function
lambda_name="my-lambda-function"
handler_name="lambda_function.lambda_handler"
zip_file="lambda_function.zip"
runtime="python3.8"
role_arn=$(aws iam get-role --role-name lambda_basic_execution --query 'Role.Arn' --output text --profile $profile)
aws lambda create-function --function-name $lambda_name --handler $handler_name --runtime $runtime --role $role_arn --zip-file fileb://$zip_file --layers $layer_arn --region $region --profile $profile

This code creates a Lambda function named my-lambda-function, using the python3.8 runtime. It sets the role for the function to lambda_basic_execution, and specifies the layer_arn we created earlier.

aws iam get-role is an AWS CLI command that retrieves information about an IAM role in the specified AWS account. IAM stands for Identity and Access Management, and it is a service that helps you manage users and their access to AWS resources.

Here are the flags used in the aws iam get-role command in the script:

• --role-name: specifies the name of the role that we want to retrieve information about.
• --query: specifies the JMESPath query to use in filtering the output. In this case, we are retrieving only the ARN of the role.
• --output: specifies the format of the output. In this case, we want the output to be in text format.
• --profile: specifies the name of the profile to use. The profile contains the AWS access key and secret access key that are used to authenticate and authorize the CLI to interact with the AWS account.

The aws lambda create-function command is arguably the most important step in our process, as it actually creates the Lambda function. This command is responsible for setting up the function's basic configuration, linking it to any required layers, and specifying the code that the function will execute. Here's what each flag in this command does:

• --function-name: specifies the name of the function to create.
• --handler: specifies the name of the file and function that will be executed when the function is triggered.
• --runtime: specifies the runtime environment that the function will use (e.g. Python 3.8).
• --role: specifies the IAM role that AWS Lambda assumes when it executes your function.
• --zip-file: specifies the location of the code that will be executed when the function is triggered. This can be a ZIP file containing the code, or the raw code itself.
• --layers: specifies any layers that the function should be linked to. Layers can contain shared code or libraries that the function can use, making it easier to manage and deploy functions with complex dependencies.
• --region: specifies the AWS region to create the function in.
• --profile: specifies the named profile to use when creating the function.

It’s worth noting that the --zip-file and --layers flags are particularly important. The --zip-file flag specifies the code that the function will execute, which should be in the format of a ZIP file containing the code. In our case, we had already created a ZIP file containing our code and dependencies and were referencing it with the $zip_file variable. The --layers flag specifies any layers that should be linked to the function, which can be useful for managing shared code or libraries across multiple functions. In our case, we were referencing the $layer_arn variable, which contained the ARN of the layer we had created in the previous step.

Once we have run this command, our function will be created and ready to be invoked. However, we still need to create an API Gateway to provide a public interface to our Lambda function, which we will cover in the next step.

Step 6: Create the API Gateway

Now that we have created the Lambda function, the next step is to create the API Gateway.

The first part of the script is responsible for creating a REST API and its resources. Here’s a breakdown of each command:

1. rest_api_name="my-api-gateway": This sets the name of the REST API to "my-api-gateway".
2. description="API for my Lambda function": This sets the description of the REST API to "API for my Lambda function".
3. rest_api_id=$(aws apigateway create-rest-api --name $rest_api_name --description "$description" --query 'id' --output text --region $region --profile $profile): This creates the REST API using the specified name and description, and then retrieves its ID using the --query and --output flags. The ID is stored in the rest_api_id variable.
4. resource_id=$(aws apigateway get-resources --rest-api-id $rest_api_id --query 'items[0].id' --output text --region $region --profile $profile): This retrieves the ID of the root resource of the REST API using the --rest-api-id, --query, and --output flags. The ID is stored in the resource_id variable.

# Create the API Gateway
rest_api_name="my-api-gateway"
description="API for my Lambda function"
rest_api_id=$(aws apigateway create-rest-api --name $rest_api_name --description "$description" --query 'id' --output text --region $region --profile $profile)
resource_id=$(aws apigateway get-resources --rest-api-id $rest_api_id --query 'items[0].id' --output text --region $region --profile $profile)

Step 7: Attach the Lambda

The second part of the script is responsible for creating a method and integration for the Lambda function. Here’s a breakdown of each command:

1. aws apigateway put-method --rest-api-id $rest_api_id --resource-id $resource_id --http-method GET --authorization-type NONE --region $region --profile $profile: This creates a method for the GET HTTP method on the root resource of the REST API using the --rest-api-id, --resource-id, --http-method, and --authorization-type flags.
2. uri="arn:aws:apigateway:$region:lambda:path/2015-03-31/functions/arn:aws:lambda:$region:$(aws sts get-caller-identity --query 'Account' --output text --profile $profile):function:$lambda_name/invocations": This constructs the URI for the Lambda function using the specified region, the caller's AWS account ID, and the name of the Lambda function.
3. aws apigateway put-integration --rest-api-id $rest_api_id --resource-id $resource_id --http-method GET --type AWS_PROXY --integration-http-method POST --uri $uri --region $region --profile $profile: This creates an integration for the GET method on the root resource of the REST API using the --rest-api-id, --resource-id, --http-method, --type, --integration-http-method, --uri, --region, and --profile flags. The integration type is set to AWS_PROXY and the integration HTTP method is set to POST.
4. aws apigateway create-deployment --rest-api-id $rest_api_id --stage-name prod --region $region --profile $profile: This creates a deployment of the REST API using the --rest-api-id, --stage-name, --region, and --profile flags. The stage name is set to "prod".
5. endpoint_url="https://$(aws apigateway get-rest-apis --query 'items[0].id' --output text --region $region --profile $profile).execute-api.$region.amazonaws.com/prod": This retrieves the endpoint URL for the deployed REST API using the --query, --output, --region, and --profile flagsHere’s the code to create the API Gateway using the AWS CLI:

# Attach the Lambda
aws apigateway put-method --rest-api-id $rest_api_id --resource-id $resource_id --http-method GET --authorization-type NONE --region $region --profile $profile
uri="arn:aws:apigateway:$region:lambda:path/2015-03-31/functions/arn:aws:lambda:$region:$(aws sts get-caller-identity --query 'Account' --output text --profile $profile):function:$lambda_name/invocations"
aws apigateway put-integration --rest-api-id $rest_api_id --resource-id $resource_id --http-method GET --type AWS_PROXY --integration-http-method POST --uri $uri --region $region --profile $profile
aws apigateway create-deployment --rest-api-id $rest_api_id --stage-name prod --region $region --profile $profile
endpoint_url="https://$(aws apigateway get-rest-apis --query 'items[0].id' --output text --region $region --profile $profile).execute-api.$region.amazonaws.com/prod"

This code creates an API Gateway named my-api-gateway, and sets up a GET method with no authorization required. It then sets up an integration between the API Gateway and our Lambda function, and creates a deployment with the name "prod". Finally, it generates an endpoint URL that we can use to test our API.

Once this has been made we can easily make changes and update our API Gateway

In conclusion, with the steps provided, you can create a simple AWS Lambda function, package it in a Lambda Layer, and create an API Gateway to invoke the Lambda function over HTTP.

Here is the complete final code:

#!/bin/bash

# Set the AWS region and profile
region="eu-west-1"
profile="personal-key"

# Create the layer
layer_name="mylayer"
layer_zip="mylayer.zip"
pip install -r requirements.txt -t python/
cd python
zip -r ../$layer_zip .
cd ..
aws lambda publish-layer-version --layer-name $layer_name --description "Python dependencies" --zip-file fileb://$layer_zip --compatible-runtimes python3.8 --region $region --profile $profile
layer_arn=$(aws lambda list-layer-versions --layer-name $layer_name --query 'LayerVersions[0].LayerVersionArn' --output text --region $region --profile $profile)

# Create the Lambda function
lambda_name="fpl-top-scorer"
handler_name="lambda_function.lambda_handler"
zip_file="lambda_function.zip"
runtime="python3.8"
role_arn=$(aws iam get-role --role-name lambda_basic_execution --query 'Role.Arn' --output text --profile $profile)
aws lambda create-function --function-name $lambda_name --handler $handler_name --runtime $runtime --role $role_arn --zip-file fileb://$zip_file --layers $layer_arn --region $region --profile $profile

# Create the API Gateway
rest_api_name="fpl-top-scorer-api"
description="API for retrieving the top scorer for a given game week of the Fantasy Premier League"
rest_api_id=$(aws apigateway create-rest-api --name $rest_api_name --description "$description" --query 'id' --output text --region $region --profile $profile)
resource_id=$(aws apigateway get-resources --rest-api-id $rest_api_id --query 'items[0].id' --output text --region $region --profile $profile)
aws apigateway put-method --rest-api-id $rest_api_id --resource-id $resource_id --http-method GET --authorization-type NONE --region $region --profile $profile
uri="arn:aws:apigateway:$region:lambda:path/2015-03-31/functions/arn:aws:lambda:$region:$(aws sts get-caller-identity --query 'Account' --output text --profile $profile):function:$lambda_name/invocations"
aws apigateway put-integration --rest-api-id $rest_api_id --resource-id $resource_id --http-method GET --type AWS_PROXY --integration-http-method POST --uri $uri --region $region --profile $profile
aws apigateway create-deployment --rest-api-id $rest_api_id --stage-name prod --region $region --profile $profile
endpoint_url="https://$(aws apigateway get-rest-apis --query 'items[0].id' --output text --region $region --profile $profile).execute-api.$region.amazonaws.com/prod"

# Update the Lambda function with the API Gateway trigger
aws lambda add-permission --function-name $lambda_name --statement-id apigateway-test-2 --action lambda:InvokeFunction --principal apigateway.amazonaws.com --source-arn "arn:aws:execute-api:$region:$(aws sts get-caller-identity --query 'Account' --output text --profile $profile):$rest_api_id/*/GET/" --region $region --profile $profile
aws lambda create-event-source-mapping --function-name $lambda_name --batch-size 1 --event-source-arn "arn:aws:execute-api:$region:$(aws sts get-caller-identity --query 'Account' --output text --profile $profile):$rest_api_id/*/GET/" --region $region --

Thank you for reading so far! In this blog I hope you have learned how to create and publish Lambda Layers, how to create a Lambda function, and how to expose the Lambda function as a RESTful API using AWS API Gateway all using the AWS CLI.

If you found this article useful feel free to connect with me on LinkedIn.