How to Do Post-Deployment Configuration Steps when Using Terraform

USE CASE

The purpose of this page 📝 is to describe four options for performing additional configuration sometimes required after deployment that one would do with Terraform. This could be for example:

• Loading an app onto a Virtual Machine (VM)
• Configuring a Database (DB) cluster
• Generating files on a Network File System (NFS) share based on resources created

Option 1: Terraform-native options utilizing resources and providers

• Local files resource
• MySQL provider
• K8s/help provider

Option 2: Passing data via APIs offered by cloud providers, e.g. using user data on AWS

• Data can be passed as a startup script to the server operating system
• All the major cloud providers have options to pass a script
• The name of the argument varies
• For AWS provider, use user_data argument of aws_instance to pass a startup script
• for 🠉, see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance#user_data
• The downside is that Terraform does not have a way to track if the script is performing as intended
• As long as it exits with code 0, Terraform is satisfied
• Because Terraform does not know the desired state of the machine, it will try to recreate the machine if you change the startup script at all, even if the current instance is not affected by the change

resource "aws_instance" "nginx2" {
  ami                    = nonsensitive(data.aws_ssm_parameter.amzn2_linux.value)
  instance_type          = "t2.micro"
  ...

  user_data = <<EOF
#! /bin/bash
sudo amazon-linux-extras install -y nginx1
sudo service nginx start
sudo rm /usr/share/nginx/html/index.html
sudo cp /home/ec2-user/index.html /usr/share/nginx/html/index.html
EOF
}

Option 3: Using Configuration managers such as Ansible, Puppet or Chef

• Many solutions that Terraform can hand off to for post-deploy configuration, e.g.
• Ansible
• Puppet
• Chef
• Usually, the configuration manager is baked into the base image for the machine and Terraform uses that image
• A cool post describing the combination of Terraform + Ansible was published by Flavius on the Spacelift blog, claiming:

A common pattern is to use Terraform to set up base infrastructure, including networking, VM instances, and other foundational resources. Once that’s done, Ansible can be invoked (either manually or via Terraform) to configure those instances, set up necessary software, and deploy applications.

— https://spacelift.io/blog/using-terraform-and-ansible-together

Option 4: Provisioners — which should be a last resort, but it is still possible

• Usually a bad idea!
• They are defined in the Terraform resource
• Executed during resource creation OR destruction
• A single resource can have multiple provisioners
• If > 1, provisioners are executed in the order of their explicit definition (exceptional for Terraform to do this)
• A provisioner without a resource can be run with null_resource or with the built-in terraform_data resource (since version 1.4)
• Failure action can be defined (break, continue)
• This is considered a last resort — there is no API that Terraform can use and manage
• Terraform is unable to ensure the essentials — no error checking, no idempotency, no consistency
• There are three provisioner types

1. File — create files and directories on the remote system
2. Local exec — run a script on the local machine executing the Terraform run. This is probably used the most.
3. Remote exec — run a script on a remote system.

• Most of the time, File + Remove provisioners can be avoided by a start-up script
• In the past, there were more provisioners, but they were deprecated as this is deemed unsafe
• Provisioner syntax is a nested block inside of a resource block

file provisioner: copies a file from source to destination

provisioner "file" {
    connection {
        type = "ssh"

        user = "root"
        private_key = var.private_key
        host = self.public_ip
    }
    source = "/local/path/to/file.txt"
    destination = "/path/to/file.txt"
}

• For more, see * https://developer.hashicorp.com/terraform/language/resources/provisioners/file

local-exec provisioner: calls a local executable to run after a resource is created

resource "aws_instance" "web" {
  # ...

  provisioner "local-exec" {
    command = "echo ${self.private_ip} >> private_ips.txt"
  }
}

remote-exec provisioner: calls a script on a remote machine after a resource is created

• It can execute an inline script, a list in a file, or a list of paths to local scripts

resource "aws_instance" "web" {
  # ...

  # Establishes connection to be used by all
  # generic remote provisioners (i.e. file/remote-exec)
  connection {
    type     = "ssh"
    user     = "root"
    password = var.root_password
    host     = self.public_ip
  }

  provisioner "remote-exec" {
    inline = [
      "puppet apply",
      "consul join ${aws_instance.web.private_ip}",
    ]
  }
}

LINKS

• https://spacelift.io/blog/using-terraform-and-ansible-together
• https://developer.hashicorp.com/terraform/language/resources/provisioners/syntax
• https://developer.hashicorp.com/terraform/language/resources/provisioners/file
• https://developer.hashicorp.com/terraform/language/resources/provisioners/remote-exec