DevOps allows software developers to release software in an automated, stable way. DevOps is more than one thing. It’s a mix of technology and culture that makes DevOps work.
This blog will focus on DevOps’ technology and tools. The core concept of Continuous Integration and Continuous Delivery is the basis of DevOps’ technical side. The idea behind CI/CD is to build an automated software delivery system that deploys new software releases continuously in an automated manner.
The developers commit the code changes to a repository. This triggers the delivery pipeline, also known as CI/CD pipeline.
To ensure faster software delivery and better security, we must incorporate security into our CI/CD pipelines. We often use open-source libraries and containers images to build our source code. It is essential to have security measures in the CI/CD pipeline. This will ensure that software we build and deploy is secure. It is equally important to decide what code or container image should be allowed to run on your target environment.
Everyone is responsible for security. Shifting left security is a DevOps technique that helps you address security issues early in the software development process. This policy can be implemented in a number of ways. These include vulnerability scanning container images and allowing trusted images to be uploaded to GKE.
What are we creating?
This blog post will demonstrate how to create a secure CI/CD pipeline with Google Cloud’s built in services. This tutorial will show you how to create a secure software delivery process that creates a Node.js sample application in a container image, and then deploys it on GKE clusters.
How do we build the CI/CD pipeline
To build the pipeline, we’ll use the following Google Cloud-integrated services:
- Cloud Build Cloud Build Cloud Build is a completely serverless CI/CD platform which allows you to automate build, test and deploy tasks.
- Artifact Registry The Artifact Registry is a secure way to store and manage your built artifacts.
- Cloud Delivery – Cloud deployment is a fully managed Continuous Delivery Service for Anthos and GKE.
- Binary Authorization The Binary Authorization provides security controls during deployment for Cloud Run and GKE deployments.
- GKE is a fully managed Kubernetes Platform.
- Google Pub/Sub- Pub/Sub is a non-serverless messaging platform.
- Cloud Functions Cloud Functions Cloud Functions is a serverless platform that allows you to run your code.
We use GitHub to store source code and Sendgrid APIs for email notifications.
Cloud Build triggers are set up in the CI/CD pipeline so that code pushed to a specific repository or branch in GitHub is detected and activated to start the build process.
Here’s how the CI/CD pipeline works without security policy enforcement.
- Developer reviews the code for a GitHub repo.
- Cloud Build triggers are configured to detect any new code pushed into this GitHub repo, and start the ‘build’ process. A successful build will result in a docker image.
- The Artifact Registry stores the container image.
- The Cloud Deploy deployment process begins with the Build process. It deploys the container image into three different GKE clusters. These GKE clusters are pre-configured to mimic the test, staging and production environments.
- Cloud Deploy must pass an approval stage before the image is deployed to the Production GKE cluster.
- A Cloud Function sends you an email notifying that the Cloud Deploy rollout is subject to your approval. The recipient can approve or deny the deployment to the production GKE Cluster. The Cloud Function code is available
We will be using a few of the built-in services and features in Google Cloud to secure this CI/CD process. We will first enable vulnerability scans on Artifact Registry. This is an unsupported feature. Finally, we will use the Binary Authorization service to create a security policy that allows only a particular image to be deployed on your GKE cluster.
Here is how we build and deploy containers with vulnerabilities.
- Developer reviews the code for a GitHub repo.
- Cloud Build triggers are configured to detect any new code pushed into this GitHub repo, and begin the ‘build’ process.
- The build fails due to an error message about vulnerabilities in the image.
Here is how it looks when we attempt to deploy a container picture to GKE. This violates the Binary Authorization policy.
- Developer reviews the code for a GitHub repo.
- Cloud Build triggers are configured to detect any new code pushed into this GitHub repo, and begin the ‘build’ process. A successful build will result in a docker image.
- Artifact Registry stores the container image.
- The Cloud Deploy deployment process begins with the Build process. It deploys the container image into three different GKE clusters. These GKE clusters are pre-configured to mimic the deployment pipeline for the production, staging and test environments.
- Cloud Deploy fails because the GKE clusters reject the image due to it violating the existing Binary Authorization Policy. An approval email is still sent before production deployment via Cloud Function. The email receiver will likely reject this release due to previous failures.
- Cloud Function will send an email to a pre-configured email address about a failed deployment if the Binary Authorization policy is violated. You can find the Cloud Function code.
Notification: If the timeout value for Cloud Deploy exceeds 10, it will fail. However, you can modify this value to suit your needs. See here.
Notification: The Cloud Function Code that is used to send the deployment failure notification and rollout approval email are located under the folder cloud functions in this repo. To receive email notifications, you will need to still create the cloud functions using this code in your Google Cloud Project.
Solution Architecture
Combining the aforementioned Google Cloud Services creates the CI/CD pipeline. Cloud Build is the core of automating this pipeline. It contains all the steps required to build and deploy our container images. Cloud Build executes each step in a YAML file. Cloud Build is flexible in how you want to design your ‘build’ or ‘deploy’ processes, and it ensures that they are executed correctly every time.
Conclusion and additional reading
This blog post explains how we created a secure CI/CD pipeline with Google Cloud’s built in services.
We discovered how to secure a CI/CD process using Google Cloud’s built in services such as binary authorization and vulnerability scanning of container images. There is only one way to control which images can be deployed to GKE clusters. Binary Authorization also offers Build verification. In this case, Binary Authorization uses attestations in order to confirm that an image was created by a specific build system.
Binary Authorization can also record all events in which a container image cannot be deployed due to security policies to audit logs. These log entries can be used to create alerts and notify the relevant team members about blocked deployment events.
Finally, the services that are used to secure and build the CI/CD pipelines can be spun up in a matter of minutes. This allows your teams to focus on creating and releasing software faster, more reliably, and at a lower cost.