Introduction

Zappa is a  very powerful open source python project which lets you build, deploy and update your WSGI app hosted on AWS Lambda + API Gateway easily.This blog is a detailed step-by-step focusing on challenges faced while deploying Django application on AWS Lambda using Zappa as a deployment tool.

Building Your Application

If you do not have a Django application already you can build one by cloning this GitHub repository.

CODE: https://gist.github.com/velotiotech/ab76374bc689729d61f8d022f67ffa9c.js 

CODE: https://gist.github.com/velotiotech/5e25f8aa02ac63db225201838a6c7d5a.js

Once you have cloned the repository you will need a virtual environment which provides an isolated Python environment for your application. I prefer virtualenvwrapper to create one.

Command :

CODE: https://gist.github.com/velotiotech/f914eb0e86beb81b1120fe4e056408a3.js

CODE: https://gist.github.com/velotiotech/84b3a9eabec962a829c0f18669340ebb.js

Install dependencies from requirements.txt.

CODE: https://gist.github.com/velotiotech/45cb7456d2c98e37c8042dcaf9ac1375.js

CODE: https://gist.github.com/velotiotech/67365ee8d1d58400c2e9d1a1405e388b.js

Now if you run the server directly it will log a warning as the database is not set up yet.

CODE: https://gist.github.com/velotiotech/5ccdb2863d7df7b3af7fa0c93e656c1e.js  

CODE: https://gist.github.com/velotiotech/244489b224198d181a332f6be3f1daa1.js

Also trying to access admin page (http://localhost:8000/admin/) will throw an “OperationalError” exception with below log at server end.

CODE: https://gist.github.com/velotiotech/66ca4dd926c477acc0bdf56adda75911.js

In order to fix this you need to run the migration into your database so that essential tables like auth_user, sessions, etc are created before any request is made to the server.

CODE: https://gist.github.com/velotiotech/a564184b8eec31fa4b6dfc6d3fb0fd86.js

CODE: https://gist.github.com/velotiotech/1fccc974d19b2aa7fbc79604f166aa71.js

NOTE: Use DATABASES from project settings file to configure your database that you would want your Django application to use once hosted on AWS Lambda. By default, its configured to create a local SQLite database file as backend.

You can run the server again and it should now load the admin panel of your website.

Do verify if you have the zappa python package into your virtual environment before moving forward.

Configuring Zappa Settings

Deploying with Zappa is simple as it only needs a configuration file to run and rest will be managed by Zappa. To create this configuration file run from your project root directory -

CODE: https://gist.github.com/velotiotech/64b2fb2d76fd9cecb6fa0494893ab1a6.js 

CODE: https://gist.github.com/velotiotech/2119f47b95061bd78737771c7def11fe.js

You can verify zappa_settings.json generated at your project root directory.

TIP: The virtual environment name should not be the same as the Zappa project name, as this may cause errors.

Additionally, you could specify other settings in  zappa_settings.json file as per requirement using Advanced Settings.

Now, you're ready to deploy!

IAM Permissions

In order to deploy the Django Application to Lambda/Gateway, setup an IAM role (eg. ZappaLambdaExecutionRole) with the following permissions:

CODE: https://gist.github.com/velotiotech/e938baad91a45b341fae116a7b430d91.js

Deploying Django Application

Before deploying the application, ensure that the IAM role is set in the config JSON as follows:

CODE: https://gist.github.com/velotiotech/a949942b8a5684ddfbfd238ba42bad6d.js

Once your settings are configured, you can package and deploy your application to a stage called "dev" with a single command:

CODE: https://gist.github.com/velotiotech/226e082cb4a15860c41dc50c8eb10aee.js

CODE: https://gist.github.com/velotiotech/58086ab91ab3abc6efe6d8001b7fe59a.js

You should see that your Zappa deployment completed successfully with URL to API gateway created for your application.

Troubleshooting

1. If you are seeing the following error while deployment, it’s probably because you do not have sufficient privileges to run deployment on AWS Lambda. Ensure your IAM role has all the permissions as described above or set “manage_roles” to true so that Zappa can create and manage the IAM role for you.

CODE: https://gist.github.com/velotiotech/83a80278e5b9412274f08895390570c4.js

2. The below error will be caused as you have not listed “events.amazonaws.com” as Trusted Entity for your IAM Role. You can add the same or set “keep_warm” parameter to false in your Zappa settings file. Your Zappa deployment was partially deployed as it got terminated abnormally.

CODE: https://gist.github.com/velotiotech/c3bff02fe90ef2c1a2eff0320b013dab.js

3. Adding the parameter and running zappa update will cause above error. As you can see it says “Stack django-zappa-sa-dev does not exists” as the previous deployment was unsuccessful. To fix this, delete the Lambda function from console and rerun the deployment.

CODE: https://gist.github.com/velotiotech/454b47d56b78ed9fadf441ce49bf8d68.js

4.  If you run into any distribution error, please try down-grading your pip version to 9.0.1.

CODE: https://gist.github.com/velotiotech/772b30353ec89c3190468d091868366f.js

CODE: https://gist.github.com/velotiotech/55a2e822827a92f468c018ece747a774.js

or,

If you run into NotFoundException(Invalid REST API Identifier issue) please try undeploying the Zappa stage and retry again.

CODE: https://gist.github.com/velotiotech/8b4e99abcc58583ddca0e1887c7bc77c.js

TIP: To understand how your application works on serverless environment please visit this link.

Post Deployment Setup

Migrate database

At this point, you should have an empty database for your Django application to fill up with a schema.

CODE: https://gist.github.com/velotiotech/186cfb61d6f4a5f32b8ad8895415c089.js

Once you run above command the database migrations will be applied on the database as specified in your Django settings.

Creating Superuser of Django Application

You also might need to create a new superuser on the database. You could use the following command on your project directory.

CODE: https://gist.github.com/velotiotech/2180f1ff30565a4347698feebcf6d01f.js

Alternatively,

CODE: https://gist.github.com/velotiotech/bd9a931cd3c9f4c08e10d0c190c88ac3.js

Note that your application must be connected to the same database as this is run as standard Django administration command (not a Zappa command).

Managing static files

Your Django application will be having a dependency on static files, Django admin panel uses a combination of JS, CSS and image files.

NOTE: Zappa is for running your application code, not for serving static web assets. If you plan on serving custom static assets in your web application (CSS/JavaScript/images/etc.), you'll likely want to use a combination of AWS S3 and AWS CloudFront.

You will need to add following packages to your virtual environment required for management of files to and from S3 django-storages and boto.

CODE: https://gist.github.com/velotiotech/71d013e77a0ea0892a31f0a3c7dbcd51.js

Once you have setup the Django application to serve your static files from AWS S3, run following command to upload the static file from your project to S3.

CODE: https://gist.github.com/velotiotech/fc6953222abf0d748696891f9bdb7fcc.js

or

CODE: https://gist.github.com/velotiotech/450d04c22b2c9bbad612794bc7940b99.js

Check that at least 61 static files are moved to S3 bucket. Admin panel is built over  61 static files.

NOTE: STATICFILES_DIR must be configured properly to collect your files from the appropriate location.

Tip: You need to render static files in your templates by loading static path and using the same.  Example, {% static %}

Setting Up API Gateway

To connect to your Django application you also need to ensure you have API gateway setup for your AWS Lambda Function.  You need to have GET methods set up for all the URL resources used in your Django application. Alternatively, you can setup a proxy method to allow all subresources to be processed through one API method.

Go to AWS Lambda function console and add API Gateway from 'Add triggers'.

1. Configure API, Deployment Stage, and Security for API Gateway. Click Save once it is done.

2. Go to API Gateway console and,

a. Recreate ANY method for / resource.

i. Check `Use Lambda Proxy integration`

ii. Set `Lambda Region` and `Lambda Function` and `Save` it.

a. Recreate ANY method for /{proxy+} resource.

i. Select `Lambda Function Proxy`

ii. Set`Lambda Region` and `Lambda Function` and `Save` it.

3. Click on Action and select Deploy API. Set Deployment Stage and click Deploy

4. Ensure that GET and POST method for / and Proxy are set as Override for this method

Setting Up Custom SSL Endpoint

Optionally, you could also set up your own custom defined SSL endpoint with Zappa and install your certificate with your domain by running certify with Zappa. 

CODE: https://gist.github.com/velotiotech/0c7b94fd895ecbd3dee2cae91dd4c55d.js

Now you are ready to launch your Django Application hosted on AWS Lambda.

Additional Notes:

•  Once deployed, you must run “zappa update <stage-name>” for updating your already hosted AWS Lambda function.</stage-name>
• You can check server logs for investigation by running “zappa tail” command.
• To un-deploy your application, simply run: `zappa undeploy <stage-name>`</stage-name>

You’ve seen how to deploy Django application on AWS Lambda using Zappa. If you are creating your Django application for first time you might also want to read Edgar Roman’s Django Zappa Guide.

Start building your Django application and let us know in the comments if you need any help during your application deployment over AWS Lambda.