Using Ansible alongside Terraform for Effective Configuration Management

The default location of the hosts’ file on the Ansible controller host is /etc/ansible/hosts. However, if this file is not present or if hosts files are found in other locations, they can also be used.
The Ansible Hosts File is an introduction to the files used in Ansible for storing information about remote nodes that need to be managed.

As part of the Write for Donations program, the author chose to donate to the Free and Open Source Fund.

Introduction

Ansible is a
Configuration Management
utility that runs playbooks, which consist of customizable actions written in YAML and executed on designated target servers. It is capable of performing various bootstrapping operations such as software installation and updates, user creation and removal, and system service configuration. Consequently, it is well-suited for provisioning servers deployed through Terraform, as they are initially created empty.

Ansible and Terraform serve distinct purposes in infrastructure and software deployment, making them non-competitive solutions. While Terraform enables the creation of system infrastructure, encompassing the hardware for application running, Ansible focuses on software configuration and deployment through playbook execution on server instances. By running Ansible on the resources provisioned by Terraform, you can expedite resource usability for your specific needs, simplify maintenance, and enhance troubleshooting capabilities, as all deployed servers will share the same applied actions.

This tutorial will guide you through deploying Droplets with Terraform. Once the Droplets are created, you will bootstrap them using Ansible. Ansible will be invoked directly from Terraform during the resource deployment to prevent race conditions. By utilizing Terraform’s

remote-exec

and

local-exec

provisioners in the configuration, the Droplet deployment will be fully completed before proceeding with additional setup.

Prerequisites

  • To obtain a DigitalOcean Personal Access Token, you can follow the instructions provided in the <a class=”text-blue-600″ href=”https://in4any.com/building-for-production-web-applications-deploying” title=”Building for Production: Web Applications — Deploying”>DigitalOcean product</a> documents titled “How to Create a Personal Access Token” within the DigitalOcean Control Panel.

  • Ensure that you have Terraform installed on your local machine and have set up a project using the DigitalOcean provider. Follow the instructions in Step 1 and Step 2 of the how to use terraform with digitalocean tutorial, making sure to name the project folder as <code>
    terraform-ansible
    </code> instead of <code>
    loadbalance
    </code>.

  • Ensure that Ansible is installed on your system. For <a class=”text-blue-600″ href=”https://in4any.com/install-older-ansible-on-ubuntu-20-04″ title=”Install Older Ansible on Ubuntu 20.04″>Ubuntu 20.04</a>, follow the initial step outlined in the <a class=”text-blue-600″ href=”https://in4any.com/how-to-install-and-configure-ansible-on-ubuntu-20-04″ title=”How To Install and Configure Ansible on Ubuntu 20.04″>How To Install and Configure Ansible on Ubuntu 20.04</a> tutorial. To gain a better understanding of Ansible, refer to the article titled “introduction to configuration management with ansible“.

Please take note that this tutorial has been specifically tested with Terraform

1.0.2

.

Step 1 — Defining Droplets

During this step, you will specify the Droplets that will be used to run an
Ansible playbook
, which is responsible for configuring the Apache web server.

If you are currently located in the directory

terraform-ansible

that was created in the prerequisites, you will proceed to define a Droplet resource. By specifying

count

, you will create three copies of the Droplet and retrieve their respective IP addresses. These definitions will be stored in a file called

droplets.tf

. To begin, create the file and open it for editing by executing the following command:

  1. tf nano droplets.

Add the following lines:

~/terraform-ansible/droplets.tf
resource "digitalocean_droplet" "web" {
  count  = 3
  image  = "ubuntu-18-04-x64"
  name   = "web-${count.index}"
  region = "fra1"
  size   = "s-1vcpu-1gb"
  ssh_keys = [
      data.digitalocean_ssh_key.terraform.id
  ]
}
output "droplet_ip_addresses" {
  value = {
    for droplet in digitalocean_droplet.web:
    droplet.name => droplet.ipv4_address
  }
}

In this section, you can configure a Droplet resource that runs on a CPU core with 1GB RAM in the specified region (

fra1

). Terraform will automatically retrieve the prerequisites (
SSH key
) from your account and add them to the provisioned Droplet using the provided unique ID list element (

ssh_keys

). If the parameter (

count

) is set, Terraform will deploy the Droplet three times. The output block following the parameter will display the IP addresses of the three Droplets. The loop will iterate through the list of Droplets, pairing each instance’s name with its corresponding IP address and appending it to the resulting map.

Once you have finished, save and close the file.

Once the Droplets that Terraform will deploy are defined, you will proceed to write an
Ansible Playbook
that will be executed on each of the three deployed Droplets to deploy the Apache web server. Later on, you will integrate Ansible into the Terraform code.

Step 2 — Writing an Ansible Playbook

To begin with, an Ansible playbook will be created to execute the necessary tasks for the initial server setup, including the creation of a new user and the upgrading of installed packages. The instructions for Ansible will be written in the form of tasks, which represent actions performed on the target hosts. These tasks can utilize built-in functions or specify custom commands to be executed. In addition to the initial setup tasks, the playbook will also include the installation of the Apache web server and the activation of its

mod_rewrite

module.

Prior to writing the playbook, make sure that your public and private SSH keys, which match the ones in your DigitalOcean account, are present and reachable on the machine where you are executing Terraform and Ansible. On Linux, a common location for keeping them would be

~/.ssh

(although alternative storage locations are possible).

Please ensure that the private key file on Linux has the necessary permissions. You can set the permissions by executing the following command:

  1. Set the file permissions of your private key to 600 by using the "chmod" command and specifying the location of the key.

Since you already have a variable defined for the private key, you just need to create another variable for the location of the public key.

Launch the editing interface for

provider.tf

by executing:
“””.

  1. The provider.tf file is dedicated to handling nanoscale configurations.

Add the following line:

~/terraform-ansible/provider.tf
terraform {
  required_providers {
    digitalocean = {
      source = "digitalocean/digitalocean"
      version = "~> 2.0"
    }
  }
}
variable "do_token" {}
variable "pvt_key" {}
variable "pub_key" {}
provider "digitalocean" {
  token = var.do_token
}
data "digitalocean_ssh_key" "terraform" {
  name = "terraform"
}

After completing the task, make sure to save the file and then close it.

Once you have defined the variable

pub_key

, you can begin writing the Ansible playbook. Save it in a file named

apache-install.yml

and open it for editing.

  1. Execute the nano command using the apache-install.yml file.

The playbook will be built in a step-by-step manner. Firstly, you will specify the hosts on which the playbook will run, its name, and whether the tasks should be executed as the root user. Please include the following lines:

~/terraform-ansible/apache-install.yml
- become: yes
  hosts: all
  name: apache-install

By assigning the value of

become

to

yes

, you direct Ansible to execute commands with superuser privileges. Additionally, by specifying

all

as the value for

hosts

, you grant Ansible the ability to perform tasks on any server, including those provided through the command line, similar to Terraform.

To avoid repetition, add the following task definition to your playbook as the first task, which will create a new, non-root user.

~/terraform-ansible/apache-install.yml
  tasks:
    - name: Add the user 'sammy' and add it to 'sudo'
      user:
        name: sammy
        group: sudo

Start by creating a list of tasks and then include a task in it. This will involve generating a user called “sammy” and providing them with superuser privileges through the use of

sudo

by adding them to the relevant group.

The upcoming task involves adding your public SSH key to the user account, enabling you to connect to it at a later time.

~/terraform-ansible/apache-install.yml
    - name: Add SSH key to 'sammy'
      authorized_key:
        user: sammy
        state: present
        key: "{{ lookup('file', pub_key) }}"

In the upcoming step, you will provide the value for the variable

pub_key

. This task guarantees that the public SSH key, obtained from a local file, is

present

on the target.

By appending the following tasks, you have the option to order the installation of Apache and the

mod_rewrite

module.

~/terraform-ansible/apache-install.yml
    - name: Wait for apt to unlock
      become: yes
      shell:  while sudo fuser /var/lib/dpkg/lock >/dev/null 2>&1; do sleep 5; done;
      
    - name: Install apache2
      apt:        
        name: apache2
        update_cache: yes
        state: latest
        
    - name: Enable mod_rewrite
      apache2_module:
        name: rewrite
        state: present
      notify:
        - Restart apache2
  handlers:
    - name: Restart apache2
      service:
      name: apache2
      state: restarted

The initial task will wait until any previous package installation using the package manager referred to as

apt

is finished. Subsequently, the second task will execute

apt

in order to install Apache. Following that, the third task will verify that the

mod_rewrite

module is properly

present

. Once it is activated, you must make sure to restart Apache, as this cannot be configured within the task. To address this, a handler is called upon to initiate the restart.

By now, your playbook will resemble the following:

~/terraform-ansible/apache-install.yml
- become: yes
  hosts: all
  name: apache-install
  tasks:
    - name: Add the user 'sammy' and add it to 'sudo'
      user:
        name: sammy
        group: sudo
        
    - name: Add SSH key to 'sammy'
      authorized_key:
        user: sammy
        state: present
        key: "{{ lookup('file', pub_key) }}"
        
    - name: Wait for apt to unlock
      become: yes
      shell:  while sudo fuser /var/lib/dpkg/lock >/dev/null 2>&1; do sleep 5; done;
      
    - name: Install apache2
      apt:
        name: apache2
        update_cache: yes
        state: latest
      
    - name: Enable mod_rewrite
      apache2_module:
        name: rewrite 
        state: present
      notify:
        - Restart apache2
  handlers:
    - name: Restart apache2
      service:
        name: apache2
        state: restarted

After completing the task, ensure that the indentations of all YAML elements are accurate and consistent with the ones displayed above. This is the only requirement for defining the Ansible side, so save and close the playbook. Next, you will make changes to the Droplet deployment code to run this playbook once the Droplets have finished provisioning.

Step 3 — Running Ansible on Deployed Droplets

After defining the actions Ansible will perform on the target servers, the Terraform configuration will be modified to execute it during Droplet creation.

Terraform provides two provisioners that can execute commands: one that runs on the target machine (

local-exec

) and another that runs remotely (

remote-exec

). The former (

local-exec

) requires connection data, such as type and access keys, while the latter (

remote-exec

) does not require any connection information. It is important to note that the provisioner (

local-exec

) runs immediately after the resource finishes provisioning, without waiting for the resource to boot up. Instead, it runs after the cloud platform acknowledges the resource’s presence in the system.

To execute Ansible after deployment, include
provisioner definition
s in your Droplet. Open

droplets.tf

for editing.

  1. tf nano droplets.

Add the highlighted lines:

~/terraform-ansible/droplets.tf
resource "digitalocean_droplet" "web" {
  count  = 3
  image  = "ubuntu-18-04-x64"
  name   = "web-${count.index}"
  region = "fra1"
  size   = "s-1vcpu-1gb"
  ssh_keys = [
      data.digitalocean_ssh_key.terraform.id
  ]
  provisioner "remote-exec" {
    inline = ["sudo apt update", "sudo apt install python3 -y", "echo Done!"]
    connection {
      host        = self.ipv4_address
      type        = "ssh"
      user        = "root"
      private_key = file(var.pvt_key)
    }
  }
  provisioner "local-exec" {
    command = "ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u root -i '${self.ipv4_address},' --private-key ${var.pvt_key} -e 'pub_key=${var.pub_key}' apache-install.yml"
  }
}
output "droplet_ip_addresses" {
  value = {
    for droplet in digitalocean_droplet.web:
    droplet.name => droplet.ipv4_address
  }
}

Similar to Terraform, Ansible operates locally and establishes an SSH connection with the target servers. To initiate it, you specify a provisioner in the Droplet definition that executes a particular command. This provisioner includes the username (root), the IP address of the current Droplet (obtained using a specific method), the SSH public and private keys, and the playbook file to be executed. By configuring the environment variable to a specific value, you can bypass the need to verify if the server was previously connected.

To avoid the Droplet becoming available after the execution of the playbook, the provisioner

local-exec

is designed to run independently. To ensure the execution of

remote-exec

, the provisioner

remote-exec

is defined with commands for the target server. Since

remote-exec

is executed before

local-exec

, the server will be fully initialized by the time Ansible is invoked. It is worth mentioning that

python3

is preinstalled on Ubuntu 18.04, allowing the option to comment out or remove the command as needed.

After finishing the modifications, save the file and then close it.

To deploy the Droplets, execute the given command, ensuring that you substitute


private_key_location


and


public_key_location


with the appropriate locations for your private and public keys.

  1. Apply terraform with the specified variables: do_token, pvt_key, and pub_key, where do_token is set to the value of DO_PAT, pvt_key refers to the location of the private key, and pub_key refers to the location of the public key.

The Droplets will provision and establish a connection, followed by the execution and installation of the provisioner. The output will be lengthy.

Output
... digitalocean_droplet.web[1] (remote-exec): Connecting to remote host via SSH... digitalocean_droplet.web[1] (remote-exec): Host: ... digitalocean_droplet.web[1] (remote-exec): User: root digitalocean_droplet.web[1] (remote-exec): Password: false digitalocean_droplet.web[1] (remote-exec): Private key: true digitalocean_droplet.web[1] (remote-exec): Certificate: false digitalocean_droplet.web[1] (remote-exec): SSH Agent: false digitalocean_droplet.web[1] (remote-exec): Checking Host Key: false digitalocean_droplet.web[1] (remote-exec): Connected! ...

Once the Droplets are provisioned, Terraform will execute Ansible through the

local-exec

provisioner. The output below illustrates this process for one of the Droplets.

Output
... digitalocean_droplet.web[2] (local-exec): Executing: ["/bin/sh" "-c" "ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u root -i 'ip_address,' --private-key private_key_location -e 'pub_key=public_key_location' apache-install.yml"] digitalocean_droplet.web[2] (local-exec): PLAY [apache-install] ********************************************************** digitalocean_droplet.web[2] (local-exec): TASK [Gathering Facts] ********************************************************* digitalocean_droplet.web[2] (local-exec): ok: [ip_address] digitalocean_droplet.web[2] (local-exec): TASK [Add the user 'sammy' and add it to 'sudo'] ******************************* digitalocean_droplet.web[2] (local-exec): changed: [ip_address] digitalocean_droplet.web[2] (local-exec): TASK [Add SSH key to 'sammy''] ******************************* digitalocean_droplet.web[2] (local-exec): changed: [ip_address] digitalocean_droplet.web[2] (local-exec): TASK [Update all packages] ***************************************************** digitalocean_droplet.web[2] (local-exec): changed: [ip_address] digitalocean_droplet.web[2] (local-exec): TASK [Install apache2] ********************************************************* digitalocean_droplet.web[2] (local-exec): changed: [ip_address] digitalocean_droplet.web[2] (local-exec): TASK [Enable mod_rewrite] ****************************************************** digitalocean_droplet.web[2] (local-exec): changed: [ip_address] digitalocean_droplet.web[2] (local-exec): RUNNING HANDLER [Restart apache2] ********************************************** digitalocean_droplet.web[2] (local-exec): changed: [ip_address] digitalocean_droplet.web[2] (local-exec): PLAY RECAP ********************************************************************* digitalocean_droplet.web[2] (local-exec): [ip_address] : ok=7 changed=6 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 ...

Upon completion, a list containing the names and corresponding IP addresses of three Droplets will be provided in the output.

Output
droplet_ip_addresses
= { "web-0" = "..." "web-1" = "..." "web-2" = "..." }

To access the default Apache welcome page and confirm the successful installation of the web server, you can navigate to any of the IP addresses in your browser.

Apache Welcome Page

Your servers have been provisioned by Terraform and the execution of your Ansible playbook on them was successful.

In order to verify the successful addition of the SSH key to the provisioned Droplets assigned to ‘sammy’, establish a connection to any of them using the provided command.

  1. Using the private key located at private_key_location, establish an SSH connection to droplet_ip_address with the username sammy.

Ensure that you include the private key location and IP address of a provisioned Droplet from your Terraform output.

The resulting output will appear akin to the following.

Output
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-121-generic x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage System information as of ... System load: 0.0 Processes: 88 Usage of /: 6.4% of 24.06GB Users logged in: 0 Memory usage: 20% IP address for eth0: ip_address Swap usage: 0% IP address for eth1: ip_address 0 packages can be updated. 0 updates are security updates. New release '20.04.1 LTS' available. Run 'do-release-upgrade' to upgrade to it. *** System restart required *** Last login: ... ...

After establishing a successful connection to the target, the shell access for the user “sammy” has been obtained. This confirms that the SSH key was properly configured for the aforementioned user.

To terminate the deployed Droplets, simply execute the provided command and enter

yes

when prompted.

  1. Execute the 'terraform destroy' command with the variables "do_token=${DO_PAT}", "pvt_key=private_key_location", and "pub_key=public_key_location".

During this step, you have incorporated Ansible playbook execution as a provisioner (

local-exec

) into your Droplet definition. To guarantee server accessibility for connections, you have included the provisioner (

remote-exec

) that can be used to install the necessary prerequisite (

python3

) before Ansible is executed.

Conclusion

The combination of Terraform and Ansible provides a versatile workflow for quickly setting up servers with the necessary software and hardware configurations. By incorporating Ansible directly into the Terraform deployment process, you can expedite the server provisioning and ensure that all dependencies for your development work and applications are readily available.

This tutorial is a part of a series called how to manage infrastructure with terraform. The series encompasses various Terraform topics, starting from the initial installation of Terraform to the management of intricate projects.

On our Ansible topic page, you can discover supplementary resources related to Ansible.

Greetings.
Just to inform you, I have limited experience with Terraform and Ansible, so please forgive any lack of comprehension displayed in my question.
Is this method still effective for updating the Ansible playbook configuration? For example, if I initially run Terraform to provision resources and execute the Ansible playbook, and then modify the playbook and rerun Terraform, will it recognize the changes in Ansible and implement them?

Frequently Asked Questions