Finally, the CDK CI Pipeline that Serverless deserves

AWS CDK has built-in Pipelines - a high-level Construct provisioning CI/CD. However, it lacks features like feature-branch deployments and developer environments, which are crucial for serverless development. So, instead of repeating the same code to enrich Pipelines in every project, today, we share an open-source, high-level library for plug-and-play CI/CD in serverless projects.

Advantages of AWS CodePipeline

AWS CodePipeline is not the best CI/CD platform out there. But it's convenient - we can set up the project's CI on the client's AWS account, and usage will count into a single bill they will receive from the AWS. That's probably the major advantage of AWS Developer Tools (CodeCommit, CodeBuild, CodePipeline, etc.) - if you are on AWS, you already have them, with no additional contracts, invoices, setup, or maintenance.

Another benefit is security. You manage access with permissions for the IAM role assumed by the pipeline. No static credentials live on any third-party system, just waiting to be compromised.

Additionally, you pay only for the actual usage - the time your builds run. So after you finish your project's active development, CodePipeline can still wait at the ready without any costs for keeping it up.

Shortcomings of CDK Pipelines

In most new projects we do on AWS, we moved from Serverless Framework to AWS CDK. Among other CDK advantages, CDK makes it simple to build custom reusable Constructs that you can share across projects as a library.

CDK also comes with built-in Pipelines - high-level Construct for CI/CD on AWS using AWS CodePipeline to deploy CDK projects. The CDK Pipelines have several useful features, including:

• Deploying to multiple next stages/environments,
• Deploying to multiple AWS accounts,
• Self-mutating to automatically update the pipeline after code changes.

However, that's not enough for serverless projects. In addition to the standard "test", "staging", and "prod" environments that we deploy from the main Git branch, we also deploy:

• A separate environment for every feature branch to test the changes in the cloud, replicating the production as closely as possible,
• And a separate environment for every developer for their ongoing work.

Deploying multiple environments, some of them short lived, fits serverless well. New environment deployments take only a few minutes, and those environments do not cost a cent until we run any workloads.

But it's not the only thing we were missing in CDK Pipelines. Being built on AWS CodePipeline, Pipelines lack other features you would expect from a modern CI/CD, like commit build status notifications to the Git repository (to see if the build succeeded or failed directly in GitHub/Bitbucket) or deployment failure notifications to email or Slack.

Introducing Opinionated CI Pipeline

To fill all those gaps, we built a library incorporating CDK Pipelines and enriching them with several additional features. Today, we share our Opinionated CI Pipeline as an open-source package available for Node.js and Python.

To reduce the boilerplate to the minimum, you plug it in the CDK entrypoint script, creating an instance of the CDKApplication() instead of the regular cdk.App(). In its properties, you provide project configuration and stacks to create. In a minimal example, it can look like this:

Additionally, you set the project name and target environments configuration as context parameters in cdk.json:

The last bit is creating an access token allowing AWS to access the repository. You can find the instructions on how to do this for GitHub and Bitbucket in the project README.

Deploying multiple environments

Now, you can deploy the CI pipeline by running:

cdk deploy -c ci=true

This will create a CI stack responsible for further deployments from your Git repository. It will provision:

• A CodePipeline deploying from your main branch to defined stages (“test”, “staging”, and “prod” in this example),
• A CodeBuild project that deploys ephemeral feature branch environments,
• And another CodeBuild project that destroys those ephemeral environments on feature branch deletion.

Additionally, each developer can deploy their own environment from local CLI:

cdk deploy -c env=john --all

In all cases - deployments from the main Git branch, feature-branches, and local CI - all the same stacks, declared once in the CDKApplication parameters, are deployed.

Notifying on deployment status

Another feature of the Opinionated CI Pipeline are deployment notifications. In fact, there are two kinds of notifications.

The first are commit status notifications. The CI sends the status to the GitHub/Bitbucket repository for each build from the main branch and feature branches. Hence, looking at your repository, you see whether the build succeeded.

Maybe even more importantly, you see those statuses in the Pull Requests. Thanks to that, you won’t accidentally merge a branch that fails to build, pass tests, or deploy on the cloud.

The second kind of notifications are deployment failure notifications to a Slack channel. It’s important to know immediately if a deployment to production failed. If you provide Slack parameters in the configuration, the main pipeline will send failure notifications to the given channel, so your team can see and act upon it. Optionally, you can enable failure notifications from the feature branches too.

Additionally, since this setup already configures AWS Chatbot to connect to Slack, it also creates an SNS topic to which you can send alerts from the CloudWatch Alarms and have them delivered to Slack as well.

Opinionated but customizable

Though the goal was to have a fully-featured CI/CD in a project with minimal configuration, the library allows customization to make it suitable for any project.

The library uses default commands to install dependencies, synthesize, and deploy the project based on the provided package manager, like npm or pnpm. But you can override or extend them with additional build and test steps.

The environments deployed from the main pipeline are fully configurable and can be deployed to distinct AWS accounts. For example, you can have a single development account and separate accounts for staging and production environments:

Environment deployments can also go in parallel in "waves". Additionally, you can specify extra commands to run before and after particular deployments, which is perfect for running integration and end-to-end tests before proceeding with production deployment.

You can also override default CodeBuild and CodePipeline settings, adjusting parameters like build timeouts, compute size, etc.

You can review all available parameters in the library README or your IDE, checking out the documentation of CDKApplication properties.

Use it today

The Opinionated CI Pipeline is built specifically for serverless architectures, but you can use it with any CDK project. It provides an abstraction over CDK Pipelines, with ephemeral feature branch environments and the ability to deploy custom development environments from the CLI. It also gives observability into the CI/CD with commit status notifications to GitHub/Bitbucket and Slack notifications on deployment failures.

The library is available for both Node.js and Python:

• for Node.js: npm install -D opinionated-ci-pipeline
• for Python: pip install opinionated-ci-pipeline

It’s also listed in the CDK Constructs Hub.

For configuration instructions, check out the README.