This post has been created for people who plan to start using Terraform on a project, in the hope it may help to save some time by sharing some of my lessons learned. And yes, the title is true – I wish I had known most of these lessons before starting to work with Terraform. I have split 11 lessons across two posts – here is part 2.

5. DEFINING OUTPUTS

Outputs show the information needed after Terraform templates are deployed. They are also used within modules to export information.

output "instance_id" {
value = "${aws_instance.instance_ws.id}"
}

When used within modules, two outputs must be defined, one in the module and a similar one in the configuration files. These outputs need to be explicitly defined. Output information is stored in a Terraform state file and can be queried by other terraform templates.

It's recommended to define Outputs for resources even if you are not using them at the time. Check the resource and the outputs provided by the resource and choose wisely which information will be useful for your infrastructure when you are using this Terraform resource. By doing so, you will decrease the need to go back and edit your module and your resource because an output is required by a new resource that you are defining.

Also, as you may want to organize your files, you can save the outputs files in a specific file called outputs.tf.

6. DEFINE EVEN THE SMALLEST COMPONENTS

When you use Terraform there is a tendency to start by focusing on the larger components, meaning that sometimes you miss smaller components that can cause frustration and technical debt. During the creation of components, Terraform will use the default options of the provider that you are using, if these are not predefined. It's important to acknowledge the default components in use and define them in Terraform, as it's possible that you need them in the future and default options may be modified by your chosen providers with no notice, resulting in two different component sets or changes in the properties of the components.

Examples of these include the Route Tables, which are sometimes not a focus area at the beginning of a project, or Elastic Container Repositories, which are easy to define but not always top of mind.

resource "aws_ecr_repository" "repository" {
name = "name_of_repo"
}

7. TERRAFORM INTERPOLATIONS

The interpolation syntax is a powerful feature which allows you to reference variables, resource attributes, call functions, etc.

String variables -> ${var.foo}
Map variables -> ${var.amis["us-east-1"]}
List variables -> ${var.subnets[idx]}

When you need to retrieve data from a module output or from the state of particular resources, you can use the module. or the data. syntax to call the desired attributes.

# Getting information from a module
output "my_module_bar_value_from_module" {
value = ${module.my_module.bar}
}

# Getting information from a data source
resource "aws_instance" "web" {
ami = "${data.aws_ami.my_amy.id}"
instance_type = "t3.micro"
}

You can also use some arithmetic or logical operations with interpolation. In this snip of code, if the evaluation of var.something is true (1, true) the VPN resource will be included.:

resource "aws_instance" "vpn" {
count = "${var.something ? 1 : 0}"
}

You can find more information about the supported Interpolations in the Terraform documentation

8. ENVIRONMENT MANAGEMENT

Workspaces in Terraform

Initially, these were known in Terraform version 0.9 as Environments, but since version 0.10 onwards, Terraform has renamed this feature to Workspaces.

It is possible to define new workspaces, change workspaces or delete workspaces using the terraform workspace command.

$ terraform workspace -h
Usage: terraform workspace

Create, change and delete Terraform workspaces.

Subcommands:
delete Delete a workspace
list List Workspaces
new Create a new workspace
select Select a workspace
show Show the name of the current workspace

There are a couple of advantages to using Workspaces:

1. They are defined by Hashicorp, so it's possible that improved features could be developed in the future
2. They reduce the usage of code

However, Workspaces also present a couple of challenges:

1. They are still an early implementation
2. They are not yet supported by all backends
3. It is not clear at the time of deployment (terraform apply) which workspace will be used (terraform workspace show)

Folders structure

One simple and useful option is to define components inside folders by environment.

project-01
├── dev
│ ├── clusters
│ │ └── ecs_cluster
│ │ ├── service01
│ │ │ ├── main.tf
│ │ │ ├── outputs.tf
│ │ │ ├── variables.tf
│ │ ├── service02
│ │ ├── service03
│ │ ├── service04
│ ├── database
│ │ ├── main.tf
│ │ ├── outputs.tf
│ │ └── variables.tf
│ ├── elasticsearch
│ │ ├── main.tf
│ │ ├── outputs.tf
│ │ └── variables.tf
│ └── vpc
│ ├── main.tf
│ ├── outputs.tf
│ └── variables.tf
├── global
│ │ └── web_login
│ │ ├── main.tf
│ │ ├── outputs.tf
│ │ └── variables.tf
│ └── terraform_state
│ ├── output.tf
│ ├── variables.tf
│ └── main.tf
├── prod
│ ├── clusters
│ ├── …
├── qa
│ ├── clusters
│ ├── …
└── README.md

Advantages:

1. Clear definition of the environment being deployed (in the folder path)
2. Most commonly used and fail-proof option for public deployments
3. Terraform States can be defined for each environment folder with no issues
4. Specify the name of the outputs for each environment

Issues:

1. Duplicated code
2. An overwhelming number of folders for larger projects
3. Copying of code and replacing of core values is always needed

So, which one should you choose? The folder structure by environments is easier and simple to use and it's a current recommended way to split the different components of your infrastructure.

9. RECOMMENDED WORKFLOW FOR COMMANDS

The terraform command has multiple options, so I wanted to share this as a recommended workflow at the moment of deployment:

1. Download the modules and force the update

The command terraform init initialises the workspace, downloading the providers, modules and setting up the terraform state backend. But if a module is already downloaded, Terraform won't recognise that a new version of a module is available. With terraform get it is possible to download the modules, but it's recommended to use the -update option to force an update.

terraform get -update

2. Once that you have the latest modules, is necessary to initialise the Terraform workspace to start downloading the providers and modules (already completed through the first command) and initialise the terraform state backend

terraform init
It's possible to also use the -upgrade option to force the update of providers, plugins and modules.

3. Prior deployment, Terraform is able to define a plan for deployment (what will be created, modified or destroyed). This plan is useful to use in a Pipeline to check the changes before initialising the real deployment.

It's important to note that the plan option is not always 1:1 with the deployment. If a component is already deployed, or if there are not enough permissions provided, the plan will pass but the deployment could fail.

terraform plan

To simplify things, it's possible to run all in one bash line: terraform get -update && terraform init && terraform plan

4. When you reach the final deployment step, Terraform will create a plan and provide the option to respond with yes or no to deploy the desired architecture.

terraform apply

Terraform can't rollback after deployment. So, if an error appears in the deployment, the issue should be solved in that moment Also, is possible to destroy the deployment (terraform destroy), but it will destroy everything and not rollback the changes.

It's possible to specify the application of a specific change or to destroy a specific resource with the option -target.

10. CALLING EXTERNAL DATA

The use of data sources allows a Terraform configuration to build on information defined outside of Terraform, or defined by another separate Terraform configuration, including:

■ Data from a remote state, this is useful to call states from another terraform deployments>
data "terraform_remote_state" "vpc_state" {
backend = "s3"
config {
bucket = "ci-cd-terraform-state"
key = "vpc/terraform.tfstate"
region = "us-east-1"
}
}

■ Data from AWS or external systems>
data "aws_ami" "linux" {
most_recent = true

filter {
name = "name"
values = ["amzn2-ami-hvm-2.0.20180810-x86_64-gp2*"]
}

filter {
name = "virtualization-type"
values = ["hvm"]
}

owners = ["137112412989"]
}

11. NEW AND INTERESTING INFORMATION

Terratest

To test your code, you can use a great tool called Terratest. This tool uses Go's unit testing framework.

Terraform version 0.12

The next Terraform version was announced a few months ago and will improve some interpolation and bring some changes to the HCL language.