Overview
In week three, we continued to practise building our own CI/CD pipeline but this time using Github Actions instead of Jenkins. The main goals of the week:
- to build a CI/CD pipeline using Github Actions;
- to deploy a containerised Node.js/Express application on AWS EC2 using our own CI/CD pipeline;
- to develop an understanding of IAM in AWS;
- to be a better DevOps Engineer than the day before;
- to have fun!
ℹ️ It was also emphasised to learn and understand the concepts rather than to rush the tasks to finish the project.
Using Docker to containerise our app
Similar to last week’s team project, we were given a Node.js web application to work with, enabling us to focus mainly on the pipeline and using AWS.
We kicked off the project by applying what we’d learnt about containers and Docker from week one to dockerise our application using a Dockerfile and push the new image to Docker Hub. It was great to see that we could do this part with more ease after only being introduced to Docker two weeks ago!
We also pushed our application with the Dockerfile to a Github repository.
Building a CI/CD pipeline with Github Actions
What’s Github Actions?
Like Jenkins, Github Actions is another popular CI/CD tool (but newer) that simplifies automation within the software development life cycle including running builds, executing tests, packaging an app and deploying. Github Actions is event-driven so the tasks will only trigger after a particular event takes place on the repository i.e. a pull request or a merge to the main branch.
To set up these tasks, a workflow file written in YAML needs to be created in the project repository inside the .github/workflows folder. A workflow has to contain one or more jobs consisting of steps that each run a command for a specific task. Each step can be defined using a shell command or an action — we can create our own action or use an action already created by the Github team or other companies in the Github Marketplace.
Our workflow steps:
- Setup a CI job to run the unit tests, Cypress test and Lint test after every push to the main branch
- Generate a new image of the application after each successful build
- Deploy the new image to Docker Hub
- Add a CD workflow to deploy the containerised app to an AWS S3 Bucket to be then deployed on an AWS EC2 instance via AWS CodeDeploy
In Github Actions, it’s possible to execute a workflow only after a separate workflow has completed. This is possible to set up by using workflow_run to configure the jobs (this StackOverflow page explains how this is done well).
We created three Github Actions workflows: test.yml, build.yml and deploy.yml.
- test.yml — this CI job executes first after every push to the main branch. All tests should pass before building a new image.
- build.yml — this next CI job executes after all the test in the previous build has successfully passed. It will build a new application image and upload it to Docker Hub.
- deploy.yml — this final CD workflow defines the deployment job to deploy the containerised app to AWS.
A cool thing with Github Actions is that it provides you real-time feedback into the progress of the steps in a sequential order with clear error messages if any, and green ticks for each passed step.
Learning new AWS Services — IAM and CodeDeploy
Once we completed building our CI workflows, it was time to move on to setting up several AWS services to complete the CD workflow and get our containerised app deployed to EC2.
So far in the Bootcamp, the AWS services we’ve covered are S3 Buckets, EC2, Elastic Beanstalk, Lambda and Amazon API Gateway. It was time to add AWS IAM and AWS CodeDeploy into the mix as part of the last two sections of the project architecture. A summary of what both services do:
AWS IAM
- IAM stands for Identity and Access Management
- It’s a feature that allows you to manage access to AWS services and resources securely
- You can create users and groups, and set permissions to allow or deny their access to specific AWS services
AWS CodeDeploy
- A deployment service that automates software application deployments to various compute services including EC2 instances
- It can deploy applications that are stored in S3 buckets or Github repositories
- It eliminates the need for error-prone manual operations
The remaining steps to do on AWS to automate the deployment job were:
- Create an EC2 instance with the correct IAM roles assigned to it so it can communicate with CodeDeploy and the S3 bucket
- Setup CodeDeploy to deploy the application from the S3 bucket to EC2
- Retrieve our team’s IAM Group’s Access Key and Secret Key with permission to read and write to S3 to enable us to add them to our Github Action deployment workflow so it can push the containerised app to S3 accordingly
- Configure AWS Credentials Action for Github Actions
Reflection
Learning Github Actions and understanding the YAML syntax to create workflows took some time meaning we didn’t have enough time to complete the last step of getting our CD workflow to deploy the containerised app to the EC2 instance.
Nonetheless, I have a better understanding of how different AWS services can connect together with CI/CD tools such as Github Actions to create an automated software development life cycle.
It was also a good week of consolidating what we’ve learnt so far over the last three weeks where I now feel more confident in navigating the AWS management console (which is by the way a beast!), using Docker and transferring my knowledge of CI/CD into a new tool— Github Actions. The ability to automate software builds, tests and deployments right from Github makes it, in my opinion, more appealing than Jenkins.
We’re moving away from AWS in week four to learn about another cloud computing service — Azure!
