<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=4958233&amp;fmt=gif">
RSS Feed

Architecture | Julian Alarcon |
25 June 2019

I created this post for people who plan to start using Terraform on a project, in the hope it may help to save some time by sharing some of my lessons learned. And yes, the title is true – I wish I had known most of these lessons before starting to work with Terraform. I have split 11 lessons across two posts - here is part 1.

1. Always use Terraform

Terraform is a tool for building, changing and versioning infrastructure safely and efficiently, helping you to define your Infrastructure as Code (IaC). Using Terraform and then making changes with other tools besides Terraform (eg Web Consoles, CLI Tools, or SDK) will create inconsistencies and affect the stability and confidence of the infrastructure.

Terraform will try to maintain the previously defined state, and any of the manual changes won't be on the defined VCS, so if a redeployment is required, those changes will be lost.

Exceptions can be necessary, but these are only for specific needs like security restrictions (Key Pairs) or the specific debugging of issues (security group rules). But keep in mind that these changes should affect controlled components.

2. Use modules to avoid repetitive work

How do you avoid having to copy and paste the code for the same app deployed in multiple environments, such as stage/services/frontend-app and prod/services/frontend-app?

Modules in Terraform allow you to reuse predefined resource structures. Using modules will decrease the snowflake effect and provide a great way to reuse existing infrastructure code.

Modules have some variables as inputs, which are located in different places (eg. A different folder, or even a different repository). They define elements from a provider and can define multiple resources in themselves:

# my_module.tf:

resource "aws_launch_configuration" "launch_configuration" {

name = "${var.environment}-launch-configuration-instance"

image_id = "ami-04681a1dbd79675a5"

instance_type = "t3.micro"


resource "aws_autoscaling_group" "autoscaling_group" {

launch_configuration = "${aws_launch_configuration.launch_configuration.id}"

availability_zones = ["us-east-1a"]

min_size = "${var.min_size}"

max_size = "${var.max_size}"


Modules are called using the module block in our Terraform configuration file, variables are defined according to the desired requirement. In the example above, we call the module twice but with different values for each different environment.

This uses the module my_module and creates an AutoScaling Group with a minimum instance size of 1 and maximum of 2 and a Launch Configuration. Both resources are defined with a specific prefix name, in this case dev:

# my_dev.tf

module "develpment_frontend" {
source = "./modules/my_module"
min_size = 1
max_size = 2
environment = "dev"

Afterwards, we can reuse the module. In our production environment, we call the same module my_module, and create the ASG with a minimum size of instances of 2 and maximum of 4, and the Launch Configuration, both with the specified prod prefix.

# my_prod.tf

module "development_frontend" {
source = "./modules/my_module"
min_size = 2
max_size = 4
environment = "prod"

Terraform config files and modules
Figure 1. Relation Terraform config files and modules


It’s recommended to define and use different versions of a specific module which allows us to work using a version control system.

If we store our modules in a VCS, for example git, we can use tags or branches names to call a specific version using the ?ref= option:

module "my-db-module" {
source = "git::ssh://git@mygitserver.com/my-modules.git//modules/my_module?ref=feature-branch-001"
allocated_storage = "200"
instance_class = "db.t2.micro"
engine = "postgres"

3. Manage Terraform State

The Terraform state file is important for Terraform because all the current states of our Infrastructure are stored here. It's a .json file normally located in the hidden folder .terraform inside your Terraform configuration files (.terraform/terraform.tfstate) and is autogenerated when you execute the command terraform apply. The direct file editing of the state file is not recommended.

This is an example of a terraform.tfstate file:

"version": 3,
"terraform_version": "0.11.8",
"lineage": "35a9fcf6-c658-3697-9d74-480408535ce6",
"modules": [
"path": ["root"],
"depends_on": []

As we work on our infrastructure, other collaborators might need to modify the infrastructure and apply their changes, changing the Terraform state file, which is why we recommend that this file be stored in a shared storage. Terraform supports multiple Backends to store this file, like etcd, azurem, S3 or Consul.

This is an example of how to define the path of the Terraform State file using the S3 provider. A DynamoDB is also used to control the lock access to the file, needed in case someone else is editing the Infrastructure at the same time. This will lock the write access to just one user at a time.

terraform {
required_version = ">= 0.11.7"
backend "s3" {
encrypt = true
bucket = "bucket-with-terraform-state"
dynamodb_table = "terraform-state-lock"
region = "us-east-1"
key = "locking_states/terraform.tfstate"

As the Terraform state file could store delicate information, it's recommended to encrypt the Storage using the options provided by the Backends.

Also, as your Infrastructure grows and you need to define multiple environments, you might need to split your Terraform state by environments and by components inside each environment. This way you will be able to work on different environments at the same time and multiple collaborators could work on different components of the same Infrastructure without being locked (one user modifying Databases and another modifying Load Balancers).

This can be achieved using the specific key component in the Backend definition:

# my_infra/prod/database/main.tf:
key = "prod/database/terraform.tfstate"

# my_infra/dev/database/main.tf:
key = "dev/database/terraform.tfstate"

# my_infra/dev/loadbalancer/main.tf:
key = "dev/loadbalancer/terraform.tfstate"

Terraform backend files
Figure 2 - Terraform backend files

4. Split everything

I mentioned earlier, splitting the Terraform state by Environments and by Components will help you build all the different components of you infrastructure in isolation from each other. What kind of division should you manage? That depends on the size of the project, its complexity and the size of your team.

For example, some components options can be defined inside themselves as inline blocks. But sometimes it's recommended to define these structures in a different resource. In this example an AWS Route Table has the routes definition inline:

resource "aws_route_table" "route_table" {
vpc_id = "${aws_vpc.vpc.id}"
route {
cidr_block = ""
gateway_id = "${aws_internet_gateway.example.id}"

Alternatively, you can create the exact same route as a separate AWS Route resource:

resource "aws_route_table" "route_table" {
vpc_id = "${aws_vpc.vpc.id}"

resource "aws_route" "route_1" {
route_table_id = "${aws_route_table.route_table.id}"
destination_cidr_block = ""
gateway_id = "${aws_internet_gateway.example.id}"

This allows you to be more flexible in the definition of your Infrastructure as it increases in complexity. Just keep in mind that it's easier to group components once they are defined, rather than splitting them after you have already deployed your Infrastructure.

As the level of complexity increases, you can deploy all your infrastructure with one command, using Bash scripts or tools like Ansible or Terragrunt.

I hope you found this helpful! Stay tuned to read the second part of the lessons I’ve learned through working with Terraform and in the meantime you can consult the Terraform documentation to help you get started with the basics.

Julian Alarcon

DevOps Engineer

Julian is a DevOps engineer who loves open source software culture, sharing his knowledge and working on weird and wonderful projects that get him to think outside the box. He also enjoys learning from new cultures and tries to experience one new thing every day. With a technical background spanning almost 10 years, Julian and his team help bring big ideas to life. He is also a coffee lover from a coffee country, an amateur photographer and a great conversationalist, especially when beer is involved.


From This Author

  • 23 July 2019

    11 Things I wish I knew before working with Terraform – part 2



  • 13 November 2023

    Delving Deeper Into Generative AI: Unlocking Benefits and Opportunities

  • 07 November 2023

    Retrieval Augmented Generation: Combining LLMs, Task-chaining and Vector Databases

  • 19 September 2023

    The Rise of Vector Databases

  • 27 July 2023

    Large Language Models Automating the Enterprise – Part 2

  • 20 July 2023

    Large Language Models Automating the Enterprise – Part 1

  • 11 July 2023

    Boost Your Game’s Success with Tools – Part 2

  • 04 July 2023

    Boost Your Game’s Success with Tools – Part 1

  • 01 June 2023

    Challenges for Adopting AI Systems in Software Development

  • 07 March 2023

    Will AI Transform Even The Most Creative Professions?

  • 14 February 2023

    Generative AI: Technology of Tomorrow, Today

  • 25 January 2023

    The Joy and Challenge of being a Video Game Tester

  • 14 November 2022

    Can Software Really Be Green

  • 26 July 2022

    Is Data Mesh Going to Replace Centralised Repositories?

  • 09 June 2022

    A Spatial Analysis of the Covid-19 Infection and Its Determinants

  • 17 May 2022

    An R&D Project on AI in 3D Asset Creation for Games

  • 07 February 2022

    Using Two Cloud Vendors Side by Side – a Survey of Cost and Effort

  • 25 January 2022

    Scalable Microservices Architecture with .NET Made Easy – a Tutorial

  • 04 January 2022

    Create Production-Ready, Automated Deliverables Using a Build Pipeline for Games – Part 2

  • 23 November 2021

    How User Experience Design is Increasing ROI

  • 16 November 2021

    Create Production-Ready, Automated Deliverables Using a Build Pipeline for Games – Part 1

  • 19 October 2021

    A Basic Setup for Mass-Testing a Multiplayer Online Board Game

  • 24 August 2021

    EHR to HL7 FHIR Integration: The Software Developer’s Guide – Part 3

  • 20 July 2021

    EHR to HL7 FHIR Integration: The Software Developer’s Guide – Part 2

  • 29 June 2021

    EHR to HL7 FHIR Integration: The Software Developer’s Guide – Part 1

  • 08 June 2021

    Elasticsearch and Apache Lucene: Fundamentals Behind the Relevance Score

  • 27 May 2021

    Endava at NASA’s 2020 Space Apps Challenge

  • 27 January 2021

    Following the Patterns – The Rise of Neo4j and Graph Databases

  • 12 January 2021

    Data is Everything

  • 05 January 2021

    Distributed Agile – Closing the Gap Between the Product Owner and the Team – Part 3

  • 02 December 2020

    8 Tips for Sharing Technical Knowledge – Part 2

  • 12 November 2020

    8 Tips for Sharing Technical Knowledge – Part 1

  • 30 October 2020

    API Management

  • 22 September 2020

    Distributed Agile – Closing the Gap Between the Product Owner and the Team – Part 2

  • 25 August 2020

    Cloud Maturity Level: IaaS vs PaaS and SaaS – Part 2

  • 18 August 2020

    Cloud Maturity Level: IaaS vs PaaS and SaaS – Part 1

  • 08 July 2020

    A Virtual Hackathon Together with Microsoft

  • 30 June 2020

    Distributed safe PI planning

  • 09 June 2020

    The Twisted Concept of Securing Kubernetes Clusters – Part 2

  • 15 May 2020

    Performance and security testing shifting left

  • 30 April 2020

    AR & ML deployment in the wild – a story about friendly animals

  • 16 April 2020

    Cucumber: Automation Framework or Collaboration Tool?

  • 25 February 2020

    Challenges in creating relevant test data without using personally identifiable information

  • 04 January 2020

    Service Meshes – from Kubernetes service management to universal compute fabric

  • 10 December 2019

    AWS Serverless with Terraform – Best Practices

  • 05 November 2019

    The Twisted Concept of Securing Kubernetes Clusters

  • 01 October 2019

    Cognitive Computing Using Cloud-Based Resources II

  • 17 September 2019

    Cognitive Computing Using Cloud-Based Resources

  • 03 September 2019

    Creating A Visual Culture

  • 20 August 2019

    Extracting Data from Images in Presentations

  • 06 August 2019

    Evaluating the current testing trends

  • 23 July 2019

    11 Things I wish I knew before working with Terraform – part 2

  • 12 July 2019

    The Rising Cost of Poor Software Security

  • 09 July 2019

    Developing your Product Owner mindset

  • 25 June 2019

    11 Things I wish I knew before working with Terraform – part 1

  • 30 May 2019

    Microservices and Serverless Computing

  • 14 May 2019

    Edge Services

  • 30 April 2019

    Kubernetes Design Principles Part 1

  • 09 April 2019

    Keeping Up With The Norm In An Era Of Software Defined Everything

  • 25 February 2019

    Infrastructure as Code with Terraform

  • 11 February 2019

    Distributed Agile – Closing the Gap Between the Product Owner and the Team

  • 28 January 2019

    Internet Scale Architecture