Makers DevOps Bootcamp: Week 4 — Implementing Infrastructure as Code using Terraform to deploy an application on Azure

(My reflection of week three is here.)

We’re over the halfway mark of the DevOps Bootcamp and I’m liking this DevOps thing so far!

Overview

In week four, the focus was to understand the concept of Infrastructure as Code (IaC) using Terraform and to deploy an application to Azure. The main goals to achieve by the end of the week were:

• to be able to explain what Infrastructure as Code (IaC) is;
• to create a Github Actions automatic workflow to build and manage the project infrastructure using Terraform;
• to set up a Github Actions CI/CD workflow to deploy an application to the infrastructure on Azure we created using Terraform;
• to be a better DevOps engineer than the day before;
• to have fun!

Infrastructure as Code

What is Infrastructure as Code?

The name intimidated me at first but it’s exactly what the name suggests. IaC in the world of DevOps is writing multiple simple lines of code to provision and manage infrastructure i.e. servers, network interface, data storage, databases, etc., and automating the infrastructure deployment process. It’s essentially a programmable IT environment.

There are many benefits to implementing IaC especially when you compare it to the old manual process of how IT infrastructure is provisioned traditionally. Before the world of cloud services, humans would have to physically stack servers, manually configure the hardware to the requirements of the operating system and the application being hosted, and then finally deploy the application to the hardware.

Photo by İsmail Enes Ayhan on Unsplash

Thanks to IaC, these tasks can now be automated in one or more code files with the option to version control the infrastructure code.

Terraform

To grasp the concept of IaC, we got to learn one of IaC’s popular tools, Terraform, and build our own infrastructure configuration files. Terraform is an infrastructure automation tool created by HashiCorp. It can be used to define the initial setup of the infrastructure on any cloud provider (AWS, Azure, Google Cloud, etc), maintain the infrastructure and also the initial setup of the application.

I think of Terraform’s job as a tool that deploys a server, but not what’s on it.

Building our first Infrastructure as Code on Azure using Terraform

Azure is another popular cloud service provider by Microsoft so it was a good opportunity to see how it compares with AWS.

Setting up Azure resources:

• Azure Resource Group — Before diving into using Terraform to set up an infrastructure for our application, we first needed to create an Azure Resource Group. An Azure Resource Group is a container that will hold all of the different Azure resources that we’ll later create as part of the Azure solution for our application.
• Azure Storage Account for Terraform State — We created an Azure Storage Account with a Blob container as the first resource to go under the resource group. The container will hold the Terraform state as a Blob, which is Microsoft’s object storage solution for the cloud. A Terraform state contains information about what infrastructure and related configuration are managed by Terraform. Every time the infrastructure is updated, Terraform will compare the current code (the current state) with its last known state (the file that’ll be saved in the Blob storage container).
• Azure Service Principal — The next Azure service we created was an Azure Service Principal. This creates an identity to enable Terraform to run on Github Actions and link it with Azure resources.

Getting started with Terraform

Before Terraform can do anything useful, we first needed to create an initial Terraform configuration file in a separate project directory. This file specifies that we’d like to use the Azure Provider, which will give us access to the Microsoft Azure’s resources using the Azure Resource Manager’s (ARM) API. ARM is an Azure service that manages and deploys resources using IaC.

Our first Infrastructure as Code configuration file

A great thing about Terraform is that it’s written in declarative code, meaning all you do is simply define what infrastructure you want and Terraform will figure out how to get to that end state for you.

To initialise the Terraform configuration, we ran the ‘terraform init’ command in the terminal. After initialising, we were then able to run ‘terraform plan’ which creates an execution plan to tell Terraform what the desired state is, and lastly ‘terraform apply’ to carry out the current execution plan.

And voila — after applying the changes, we could see that the Blob had been updated with the new Terraform state in our Blob container within the Blob storage account on Azure.

Time for automation

We wouldn’t be following DevOps best practices if we didn’t automate the manual process of creating our Terraform configuration! Applying what we learnt about Github Actions last week, we created two workflows that would automate our IaC.

• terraform-plan.yml — this workflow triggers after every pull request to the main branch. The main jobs it will carry out are to check the formatting of the Terraform configuration file, followed by the initialisation and creation of the execution plan (just like before).
• terraform-apply.yml — this workflow will only run after the successful completion of the first workflow. This will apply the new Terraform changes to reach the desired state when merging into the main branch.

A snippet of our team’s first workflow

The output that we got when creating our Azure Service Principal earlier came in handy here. Setting the ARM Client ID, ARM Client Secret, ARM Subscription ID and ARM Tenant ID as environment variables enabled our Terraform workflows to link with Azure.

Adding more Azure resources to our infrastructure

Application architecture

Like in previous weeks, our coach provided an application for our team to use. Our main task was to deploy that application to the infrastructure we just created on Azure using Terraform.

But first, as the above diagram shows, we first needed to create the additional Azure resources under our existing Azure Resource Group so that our application can deploy to Azure successfully. Continuing with Terraform, we could set this up by creating additional configuration files to our existing Terraform project directory.

The Azure resources to create were:

• Azure Container Registry (ACR) — this will contain the dockerised application image
• Azure Web App Service — this is an HTTP-based hosting service where we will deploy the application
• Azure PostgreSQL server — a server for our PostgreSQL database
• Azure PostgreSQL database — an Azure Database for PostgreSQL

We separated the Azure resources into separate Terraform configuration files in addition to the initial infrastructure setup file ‘main.tf’.

Terraform IaC configuration files

The Terraform syntax to define new resources is pretty straight forward to understand. To create an Azure Container Registry, it is represented like this:

Code snippet of acr-create.tf

• resource — this keyword tells Terraform that we are about to define a resource.
• azurerm_container_registry — this defines the resource type that correlates to the infrastructure element to create on the cloud service provider. In this case, we created an Azure Container Registry.
• acr — this is the resource name that Terraform will use.
• name, resource_group_name, location and SKU — everything inside the curly braces are arguments we pass to let Terraform know what we want to create concerning that resource.

We created two more configuration files ‘pg-create.tf’ and ‘web-app-create.tf’ to create our PostgreSQL and Azure Web App Service respectively.

Once we merged our updated IaC working directory to apply the changes, we could see that all of the new Azure resources under our Azure Resource Group on the Azure portal. 🎉

Deploying the application to Azure

Now that we have our infrastructure set up on Azure thanks to our Terraform configuration files, we were ready to deploy the application onto the infrastructure. We first needed to update the CI-CD pipeline in the application itself with our Azure credentials so that after every pull request and push made on the main branch, the application is containerised and deployed to Docker and our container registry on Azure.

After hours of debugging with further configurations on our Azure resources and the application code itself, we could finally see our application live on Azure, which was a task listing app! 🎉

Reflections

I wanted to go into week 4 being more kinder to myself by putting less pressure to understand new concepts immediately and set realistic learning goals for myself. It paid off because I had fun and developed a good high-level understanding of what IaC and Terraform are, which were my two main focus of the week. I’m excited to continue learning more about configuration management and Terraform specifically in the future.

It was also a good week of consolidating the method of CI/CD and automating manual processes using Github Actions. Being able to learn Azure and seeing how it compares with AWS was also interesting. To my surprise, its capabilities are similar and navigating on the Azure portal was relatively straight forward.

As a team, we went through many debugging sessions throughout the week.

These sessions helped improve my confidence in reading error logs, get comfortable with shell scripting and researching for the right documentation to debug the errors.

We’re now on the final stretch of the DevOps Bootcamp with our final two-week project left to go. We’ll be working on solving common Site Reliability Engineering issues in a live application!

Thanks for reading 😁