Recently, I encountered a situation where I needed to run a Python script from a Node.js runtime using AWS Lambda.
AWS supports several runtimes for Lambda including Node.js, Python, Java, .Net, Go, Ruby and also Custom Runtime.
In my case I was already using
nodejs16.x which of course, does not come with Python installed. After cosindering using Lambda layers or trying to compile the
.py script into some stand-alone binary (which failed due to missing Python dependencies on the Lambda anyway), I’ve decided to give a try to the Custom Runtime.
Initially, I was put off by the possible amout of work involved into configuring a custom image, I guessed it would require at least a Dockerfile, publish the image to a registry and then instruct my Lambda to use that image, configure IAM permissions, etc.. Luckily, with Serverless.com I figured the framework itself would do most of the heavy lifting so it turned out to be easier than expected. I presume doing the same with Terraform would require a little bit more of code.
I’ve decided to put together this blog post to help out folks doing something similar given that it wasn’t so easy to find docs about this.
Let’s go through the steps required:
- Create a
- Publish the image to ECR
- Instrument your Lambda to use the new custom runtime
Create a Dockerfile for a custom AWS Lambda runtime
This is simple as writing any Dockerfile, it’s helpful to know that you can use AWS built images as starting point for your image. This is my example:
FROM public.ecr.aws/lambda/nodejs:16 RUN yum install -y python3 && \ pip3 install --upgrade pip requests COPY . . CMD ["lib/functions/index.handler"]
To find other AWS Lambda base images, check out: Amazon ECR Public Gallery > Lambda.
You could give it a go and build the image manually, but it’s not required as the Serverless Framework will be doing that for us.
Publish the image to ECR
If you haven’t used ECR before, don’t worry. It’s a docker-compatible image registry which is available in your AWS account which can host public and private images.
All you have to do is add this snippet to your
provider: ecr: images: node16-python3-runtime: path: ./
node16-python3-runtimeis an arbitrary name I gave to my image, feel free to rename it to whatever is best in your use-case.
pathis the path to the directory containing the
Once you run
serverless deploy, the tool will build the image for you, and publish the image to AWS ECR. If successful, once you open ECR in your AWS console, it should look something like the following:
Instrument your Lambda to use the new custom runtime
This is the final part, but it requires attention because it’s very different from how you would usually define your Lambda and its handler in
First of all: make sure to remove the
handler: property that you usually have.
Second, add the property
image as follows:
functions: yourLambdaName: image: name: node16-python3-runtime command: - lib/functions/index.handler entryPoint: - '/lambda-entrypoint.sh'
Let me explain:
yourLambdaNameshould be renamed of course with the name of your own lambda
name: node16-python3-runtimethis should match the name of the image you created before, in
commandis the command executed when the docker container starts, it should point to the handler function
entryPointkeep is as it is, to allow the Lambda service to know what to do
Disclaimer: The cold start time will most likely increase using a custom runtime especially the one I’ve provided as example. In my case I’ve noticed
1.2 seconds which is much higher that what other Lambdas only using Node.js experience. However, because this is not user-facing or its latency matter, for me it’s not a big deal. Take it into account if you are building APIs for example.
That should be it, I hope this guide is helpful to someone!