Lambda with container

AWS Lambda is powerful, it lets you focus on business logic, rather than infrastructure. Coupled with Amazon API gateway, you can easily deploy your application as REST-APIs. It can also be used as an option to deploy machine learning applications.

However, serving machine learning prediction in Lambda is not without challenges, one important limitation is the maximum deployment package size must be under 250MB unzipped size (more service limit is here). However, machine learning model trained using deep learning technologies (in NLP, computer vision scenarios) is usually well-beyond 250MB, and machine learning application usually depends on big Python dependencies. In order to work around the limit, there are mainly 2 options:

1. Place machine learning model and Python dependencies on Amazon EFS, then mount EFS on Lambda.
2. Lambda supports running a custom container image, if your container image (including Python dependencies, machine learning model) is couple of GBs, you can consider this option

Within current container tooling, building container image is pretty simple. Here are some articles talking about Lambda with container image

Container Images for AWS Lambda with Python

AWS Lambda with custom docker images as runtime

Deploying AWS Lambda with Docker Containers: I Gave it a Try and Here’s My Review

However, in software world, after you learn the new concept, the next best thing is a repo with clear instruction (surprisingly, not a step-by-step guide). Here is a ready repo on how to deploy a BERT model to Lambda with SAM. With SAM, you can easily build, test locally and deploy to Lambda.

aws-lambda-docker-serverless-inference/hebert-sentiment-analysis-inference-docker-lambda at main ·…

The most important file in the repo is template.yaml, which defines Lambda infrastructure (e.g. which language it is using, container image it uses, Dockerfile to build the container image, etc.)

Here is the simple workflow to build container image for lambda and deploy. Certainly before that, you need to setup Elastic container registry.

Build

sam build

Local test

sam local invoke <your function name> --event <event data file>

Deploy

sam deploy --guided

Another option is EFS, however, syncing Python dependencies and machine learning models are still a bit complex than building container images in my mind.

Deploy multiple machine learning models for inference on AWS Lambda and Amazon EFS | Amazon Web…

Using Amazon EFS for AWS Lambda in your serverless applications | Amazon Web Services

Building deep learning inference with AWS Lambda and Amazon EFS | Amazon Web Services

Choosing between storage mechanisms for ML inferencing with AWS Lambda | Amazon Web Services

Hosting Hugging Face models on AWS Lambda for serverless inference | Amazon Web Services

New Serverless Bert with Huggingface, AWS Lambda, and AWS EFS

Setup —Serverless Machine Learning Inference with AWS Lambda + Amazon EFS

Also tried distributed tracing with X-Ray, which helps debug in a distributed/microservice environment.

https://github.com/marekq/aws-lambda-xray-node

If nothing is traced, X-Ray just shows “Get started” page. Otherwise, it will show “service map”.

Using AWS Lambda with AWS X-Ray

https://itnext.io/a-deep-dive-into-serverless-tracing-with-aws-x-ray-lambda-5ff1821c3c70

https://docs.aws.amazon.com/lambda/latest/dg/access-control-identity-based.html

AWSLambdaReadOnlyAccess is deprecated, becomes AWSLambda_ReadOnlyAccess