T3CON23 Recap—Infrastructure as Code

Fast-forward 10 years, to 2013. At a conference, Martin presented his state-of-the-art deployment process that included version control, code review, CI/CD systems, and an early, pre-1.0 version of Surf (a package still in use today). Inspired by the Ruby tool Capistrano, Surf implements atomic deployment, ensuring that an application cannot be deployed in a partially updated state. In a nutshell, the tool uploads a new version of an application to a new directory and then updates symbolic links to point to the directories and files of the new version.

In a multi-environment scenario, deployments must be consistent across environments. This is an entirely new situation, Martin pointed out. Software developers can no longer focus solely on an application itself. They have to ensure that the application behaves exactly the same way regardless of the environment it is deployed to.

How to ensure this consistency? The easiest way is to deploy environments just like applications.

“I think my main point for today is that we shouldn't just talk about deployment and infrastructure, but we should talk about deploying your infrastructure.”
– Martin Helmich

The best practice for software deployment is that it is automated and repeatable. Martin describes what can happen if you don’t automate: “Imagine you're trying to roll out a TYPO3 application manually. You might connect to a server via SSH, do a Git pull and a Composer update, and then run scripts to do your database migrations.”

“If you do this in a production environment and forget one of those steps, then — oh, no! — you have an internal server error. Boom. The customer calls you, and they’re not happy.” 

This is why modern software development uses automated deployments. Setting up infrastructure manually is as prone to errors as deploying software manually. That’s why Martin is passionate about Infrastructure as Code — infrastructure that can be deployed like software

The infrastructure landscape is wide and varied. You can choose to run servers on-premise or buy infrastructure from vendors. “And there are different offerings that range from Platform-as-a-Service [PaaS] to Infrastructure-as-a-Service [IaaS] offerings, where you have a gradually greater degree of control over your infrastructure”, Martin explained. The best option for you usually means prioritizing control or convenience.

When you choose a vendor, you normally use their tooling by default. However, there are several vendor-independent tools available for infrastructure management. Terraform and Docker are two examples of tools that turn your infrastructure requirements into textual descriptions for easy automation.

Terraform: Deploy your infrastructure with a single command

Rooted in the cloud-native ecosystem, Terraform (or its open-source fork OpenTofu) orchestrates infrastructure based on a description language called HCL. Martin described how it works: “You have a machine-readable text file. [...] You describe the desired state of your infrastructure; for example: I want to have an AWS instance using the following image with the following amount of resources. And then [...] you can say

terraform apply,

and TerraForm will run the API on all your vendors and provision these infrastructure items as required.” If the description changes, Terraform updates the infrastructure accordingly.

To address the diversity of today’s infrastructure landscape, Terraform has a plugin framework. Vendors who want to make their infrastructure available to Terraform can write custom plugins, and if you need a custom plugin that does not exist yet, writing one is straightforward. 

Martin wrote a Terraform provider for his company, Mittwald, as a proof of concept. “Turns out it's not that hard”, he concludes. With this plugin, a Terraform user can include resources from the Mittwald cloud platform into their infrastructure.

With Terraform and similar tools, infrastructure can be deployed and managed just like applications. But with this model, you are still deploying the infrastructure first and the application second. Ideally, you could deploy an application with all the required infrastructure in a single step. Enter: containers. 

Docker: Give your app a replicable environment

The container technology lets you package an application and all of its requirements into a deployable image, including operating-system-specific shared libraries or auxiliary binaries. 

Compared to fully-fledged virtual machines, which are typically the basis of a PaaS offering, containers “give you a higher level of control over your infrastructure than traditional Platform-as-a-Service offerings, but you do not have that many problems you have to worry about, like with regular infrastructure deployments,” Martin pointed out. 

The most prominent container system is Docker. Here is how it works:

In a text file called Dockerfile, you specify how your environment shall be built, and Docker turns this into a container image. Then, tools like Docker Compose let you describe, in a purely declarative way, how your containers shall be started and configured. The Dockerfile and the docker-compose file are text files that allow integrating the infrastructure deployment process into CI/CD pipelines and managing it like a software deployment.

The big advantage of containers is that all the application’s requirements are placed inside replicable container images. The target platforms only need to know how to run containers. 

“So you basically split your infrastructure into multiple parts, and you take the parts that are of concern for you, like the required environments that your application needs to run. That's your problem. That's part of your deployment artifact when working with containers, and the rest of your infrastructure is not your problem anymore. All you need is an infrastructure that is able to run these containers”, Martin explained.

There are, however, types of resources that cannot be put into a container, such as managed DNS or mail services. Tools like Terraform are still useful for integrating resources like these. 

Containers allow us to integrate operational concerns into a classic setup. For example, security patch management can be made part of the standard deployment process. And now you can automate security management, for example, by adding a security scanner to the build pipeline.

Infrastructure can, and should, be treated like code

Automating deployment can help de-silo the development and operation teams, reduce friction, and shorten development cycles — getting your product to market sooner. As Martin pointed out, “If we're doing Agile, we're iterating quickly. And this also needs to apply to our infrastructure.” 

The Infrastructure as Code approach is suitable for any kind of organization. The actual implementation depends on factors like your service level or how much of your infrastructure you manage yourself. Whenever you have a continuous delivery process for your software, you can easily integrate your infrastructure setup into the same process and manage applications and infrastructure closely together.

A range of tools are available for managing infrastructure as code. “There is no silver bullet”, Martin warned. “There’s no one right answer.” Choose the tools that fit your needs and use cases best. Terraform and Docker are two examples of how to turn infrastructure requirements into textual descriptions for easy automation. Treat your infrastructure as software and reap all the benefits you’ve come to rely on from modern software deployment: automation, replicability, and flexibility. As Martin suggested, “Try to make it as easily repeatable as you can.”