These days, we see that most software development is moving towards serverless architecture, and that’s no surprise. Almost all top cloud service providers have serverless services that follow a pay-as-you-go model. This way, consumers don't have to pay for any unused resources. Also, there’s no need to worry about procuring dedicated servers, network/hardware management, operating system security updates, etc.

Unfortunately, for cloud developers, serverless tools don't provide auto-deploy services for updating local environments. This is still a headache. The developer must deploy and test changes manually. Web app projects using Node or Django have a watcher on the development environment during app bundling on their respective server runs. Thus, when changes happen in the code directory, the server automatically restarts with these new changes, and the developer can check if the changes are working as expected.

In this blog, we will talk about automating serverless application deployment by changing the local codebase. We are using AWS as a cloud provider and primarily focusing on lambda to demonstrate the functionality.

Prerequisites:

• This article uses AWS, so command and programming access are necessary.
• This article is written with deployment to AWS in mind, so AWS credentials are needed to make changes in the Stack. In the case of other cloud providers, we would require that provider’s command-line access.
• We are using a serverless application framework for deployment. (This example will also work for other tools like Zappa.) So, some serverless context would be required.
  

Before development, let's divide the problem statement into sub-tasks and build them one step at a time.

Problem Statement

Create a codebase watcher service that would trigger either a stack update on AWS or run a local test. By doing this, developers would not have to worry about manual deployment on the cloud provider. This service needs to keep an eye on the code and generate events when an update/modify/copy/delete occurs in the given codebase.

Solution

First, to watch the codebase, we need logic that acts as a trigger and notifies when underlining files changes. For this, there are already packages present in different programming languages. In this example, we are using 'python watchdog.'

CODE: https://gist.github.com/velotiotech/e7b750e4b9fe9c9b39c2a436e6797e91.js

Here, the on_any_event() class method gets called on any updates in the mentioned directory, and we need to add deployment logic here. However, we can't just deploy once it receives a notification from the watcher because modern IDEs save files as soon as the user changes them. And if we add logic that deploys on every change, then most of the time, it will deploy half-complete services. 

To handle this, we must add some timeout before deploying the service.

Here, the program will wait for some time after the file is changed. And if it finds that, for some time, there have been no new changes in the codebase, it will deploy the service.

CODE: https://gist.github.com/velotiotech/ba1ee538083d4eaf85c39d805fc56205.js

The specified valid_events acts as a filter to deploy, and we are only considering these events and acting upon them.

Moreover, to add a delay after file changes and ensure that there are no new changes, we added interval_watcher_thread. This checks the difference between current and last directory update time, and if it's greater than the specified threshold, we deploy serverless resources.

CODE: https://gist.github.com/velotiotech/9200240786c9087cf7880d60a5c3885b.js

Here, the sleep time in deploy_service_on_change is important. It will prevent the program from consuming more CPU cycles to check whether the condition to deploy serverless is satisfied. Also, too much delay would cause more delay in the deployment than the specified value of DEPLOY_AFTER_CHANGE_THRESHOLD.

Note: With programming languages like Golang, and its features like goroutine and channels, we can build an even more efficient application—or even with Python with the help of thread signals.

Let’s build one lambda function that automatically deploys on a change. Let's also be a little lazy and develop a basic python lambda that takes a number as an input and returns its factorial value.

CODE: https://gist.github.com/velotiotech/07959fe2e857415e287b643a0e67055a.js

We are using a serverless application framework, so to deploy this lambda, we need a serverless.yml file that specifies stack details like execution environment, cloud provider, environment variables, etc. More parameters are listed in this guide. 

CODE: https://gist.github.com/velotiotech/ec416d9b78478b7e9a5945a6dbfaaed9.js

We need to keep both handler.py and serverless.yml in the same folder, or we need to update the function handler in serverless.yml.

We can deploy it manually using this serverless command: 

CODE: https://gist.github.com/velotiotech/c3f4dbd1bfbd27ff55cd80aa2950b776.js

Note: Before deploying, export AWS credentials.

The above command deployed a stack using cloud formation:

•  --stage is how to specify the environment where the stack should be deployed. Like any other software project, it can have stage names such as production, dev, test, etc.
• -v specifies verbose.
  

To auto-deploy changes from now on, we can use the watcher.

Start the watcher with this command: 

CODE: https://gist.github.com/velotiotech/bc1719f7850f8186986281d886709370.js

This will run continuously and keep an eye on the codebase directory, and if any changes are detected, it will deploy them. We can customize this to some extent, like post-deploy, so it can run test cases against a new stack.

If you are worried about network traffic when the stack has lots of dependencies, using an actual cloud provider for testing might increase billing. However, we can easily fix this by using serverless local development.

Here is a serverless blog that specifies local development and testing of a cloudformation stack. It emulates cloud behavior on the local setup, so there’s no need to worry about cloud service billing.

One great upgrade supports complex directory structure.

In the above example, we are assuming that only one single directory is present, so it's fine to deploy using the command: 

CODE: https://gist.github.com/velotiotech/6c65310bd9b4ed00ce1878a87e0726e5.js

But in some projects, one might have multiple stacks present in the codebase at different hierarchies. Consider the below example: We have three different lambdas, so updating in the `check-prime` directory requires updating only that lambda and not the others. 

CODE: https://gist.github.com/velotiotech/d828e96687d33c33763ba4c7871b4dc1.js 

The above can be achieved in on_any_event(). By using the variable event.src_path, we can learn the file path that received the event.

Now, deployment command changes to: 

CODE: https://gist.github.com/velotiotech/6e9974acb37a0afcf281d982a8492c09.js

This will deploy only an updated stack.

Conclusion

We learned that even if serverless deployment is a manual task, it can be automated with the help of Watchdog for better developer workflow.

With the help of serverless local development, we can test changes as we are making them without needing an explicit deployment to the cloud environment manually to test all the changes being made.

We hope this helps you improve your serverless development experience and close the loop faster.

Related Articles

1. To Go Serverless Or Not Is The Question

2. Building Your First AWS Serverless Application? Here’s Everything You Need to Know