Running Python with Psycopg2 on AWS Lambda

I have outlined the overall “project” and here is my notes on the psycopg2 challenge:

Installing Psycopg and setting up logging

In general - you need to include all dependencies when deploying the Lambda. I decided on an approach that included a “vendored” directory and do sys.path mangling.

.
├── content
│   ├── common.py
│   ├── __init__.py
│   └── import.py
├── database -> ../database
├── handler.py
├── __init__.py
├── lambdahelper.py
├── serverless.yaml
└── vendored
    ├── psycopg2
....
    ├── contextlib2.py
....
    ├── dateutil
etc

In my git repo I added included the vendored library of psycopg2 and install the rest of the needed content of vendored with pip -t during deployment. The special case of psycopg turns out to be needed. I used the version from https://github.com/jkehler/awslambda-psycopg2. TLDR is:

And the same for any library depending on C extensions that isn’t already in the lambda AMI

Then, when running the lambda, to tell the Python code where I left it’s third party dependcies I do some sys.path wrangling:

Which means my handler.py must, before any other external dependencies, import _pathmangle

In order to get decent logging in cloudwatch I also include a “lambdahelper.py” that setups up the normal python logging with a Serverless Framework friendly format and silences the excessive boto logging.

Put together in a handler.py it looks like this:

Before running serverless deploy I then need to make sure my pip dependencies are installed and uptodate:

pip install -t vendored -r requirements.txt

Comments