Serverless Deploy Previews on GitHub Actions

Andrew Barba -

Five years ago Netlify changed front-end development in a big way - seamlessly deploying every commit to its own sandboxed enviornment, accessible with a unique url to that deployment. This made it dramatically easier to test new features, new pages, etc. by simply committing and pushing your code to GitHub, and receiving a url that you could share with your team to test the changes. Since then many hosting platforms have followed suit, but largely focused on front-end development.

At Barstool, all of our backend API's are deployed to AWS Lambda using the Serverless Framework. One challenge we continuously faced was testing new front-end features against a version of an API that could deliver the corresponding features. For example, we built a new feature that allowed logged in users to follow their favorite bloggers. When a user followed a blogger, we create an entry in a new MongoDB collection, and then use the entries in that collection to create a personalized feed of content based on those subscriptions. When we were building this feature, our front-end web developers would need to wait until the API was fully deployed to production before they could properly use it. This created a workflow where the backend team needed to be well ahead of the front-end team in order to give them enough time to build the features and properly test them before deploying. This process was clearly ineffecient, and perfectly solved by Netlify's deploy preview model. What if we could provide the same workflow on our backend API's and deploy every commit to it's own isolated stack?

Serverless Deploy

Deploying a serverless application can be as simple as running a one line command:

serverless deploy --stage prod

That one command will create the necessary resources in AWS and name them all according to the passed in stage parameter. This makes it easy to have multiple stages, such as dev and prod. Our approach to deploy previews involved taking advantage of this stage parameter and injecting the github branch name as the actual stage:

serverless deploy --stage feature-ch4625

This is relatively easy to run locally, but can be quite error prone. It becomes easy for developers to forget to deploy, or choose to only deploy on certain commits. To automate the deploy process, we wanted to build workflows on GitHub actions that would run the deploys automatically, and even deploy to product upon merging to main.

GitHub Actions

GitHub Actions is a fully managed CI/CD service that seamlessly hooks into your code branches, commits, tags, etc. We chose Actions due to its seamless integration with the rest of GitHub, and its extremely robust YML syntax for defing workflows in code. We had the following requirements:

  1. Build and test every commit, on every branch
  2. Deploy every commit, on every branch, to a sandboxed environment using the branch name as the --stage
  3. Deploy the production stage when merging to main branch
  4. Teardown the AWS resources from the deploy preview when a branch is deleted

Build & Test

To get Started we created a new folder .github/workflows and added the first workflow file build-test.yml. This is the simplest of the workflow files, simply checking out the code, installing dependencies, and running our tests:

name: Build & Test

on:
  push:
    branches-ignore:
      - main

jobs:
  build-test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: Use Node.js
        uses: actions/setup-node@v1
        with:
          node-version: 14.x

      - run: yarn install --frozen-lockfile

      - run: yarn lint

      - run: yarn test

Deploy Preview

Before we could start working on the deploy preview workflow we needed to find a simple way to inject the GitHub branch name into the workflow as an environment variable. You may think this is an easy, built in option on GitHub Actions but to our surprise it was not. Luckily the Actions community is extremely strong and there's an excellent package named github-slug-action to do just that. Using the action couldn't be simpler, define it as a step and use any of the injected environment variables in a subsequent step:

steps:      
      
      - name: GitHub Refs
        uses: rlespinasse/github-slug-action@v2.x
        
      - run: npx serverless deploy --stage $GITHUB_REF_SLUG_URL

The last remaining piece is correctly injecting the Serverless environment variables needed to deploy to AWS. This was easily accomplished by using the new project secrets on GitHub, which are then exposed to Actions workflows. The final deploy-preview.yml file looks like this:

name: Deploy Preview

on:
  push:
    branches-ignore:
      - main

jobs:
  deploy-preview:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: GitHub Refs
        uses: rlespinasse/github-slug-action@v2.x

      - name: Use Node.js
        uses: actions/setup-node@v1
        with:
          node-version: 14.x

      - run: yarn install --frozen-lockfile

      - run: yarn lint

      - name: Serverless Deploy
        id: serverless_deploy
        run: npx serverless deploy --stage $GITHUB_REF_SLUG_URL
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          SERVERLESS_ACCESS_KEY: ${{ secrets.SERVERLESS_ACCESS_KEY }}

It's important to note: We do not run this workflow on the main branch. This is because production deploys need some additional steps.

Deploy Teardown

Deploy previews are an incredible feature to boost developer productivity, but they can also dramtically increase your cloud resources depending on what you deploy. To combat this, we choose to completely teardown the deploy preview upon branch deletion. Add a new file called teardown.yml:

name: Teardown

on: delete

jobs:
  teardown:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: GitHub Refs
        uses: rlespinasse/github-slug-action@v2.x

      - name: Use Node.js
        uses: actions/setup-node@v1
        with:
          node-version: 14.x

      - run: yarn install --frozen-lockfile

      - name: Serverless Remove
        run: npx serverless remove --stage $GITHUB_EVENT_REF_SLUG_URL
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          SERVERLESS_ACCESS_KEY: ${{ secrets.SERVERLESS_ACCESS_KEY }}

Once again we define the github-slug-action step in order to get access to the same environment variable we used to name our stage. Then we pass that stage name to the serverless remove command which will fully teardown the preview stack.

Wrapping Up

Deploying Serverless applications on GitHub Actions has been a dramatic productivity boost for our team, but there's still more to do. Right now its necessary for an engineer to review the deploy logs and find the preview url created by the Serverless framework. In the future we'd like to automatically parse this url and post it as a comment on the pull request.