I have outlined the overall “project” and here is my notes on the psycopg2 challenge:
Installing Psycopg and setting up logging
In general - you need to include all dependencies when deploying the Lambda. I decided on an approach that included a “vendored” directory and do sys.path mangling.
. ├── content │ ├── common.py │ ├── __init__.py │ └── import.py ├── database -> ../database ├── handler.py ├── __init__.py ├── lambdahelper.py ├── serverless.yaml └── vendored ├── psycopg2 .... ├── contextlib2.py .... ├── dateutil etc
In my git repo I added included the vendored library of psycopg2 and install
the rest of the needed content of vendored with
pip -t during deployment.
The special case of psycopg turns out to
needed. I used the version from
https://github.com/jkehler/awslambda-psycopg2. TLDR is:
- No PG libraries on the AMI that runs lambdas (http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html)
- You can spin up such an api up and build the C libraries statically linked
and include them along the psycogp2 code when zipping the lambda.
- Or checkout that repo and include that - if you trust that code.
And the same for any library depending on C extensions that isn’t already in the lambda AMI
Then, when running the lambda, to tell the Python code where I left it’s third party dependcies I do some sys.path wrangling:
Which means my handler.py must, before any other external dependencies,
In order to get decent logging in cloudwatch I also include a “lambdahelper.py” that setups up the normal python logging with a Serverless Framework friendly format and silences the excessive boto logging.
Put together in a handler.py it looks like this:
serverless deploy I then need to make sure my
dependencies are installed and uptodate:
pip install -t vendored -r requirements.txt