Terraform

HashiCorp Certified Terraform Associate 2023

Advantages Terraform

Advantages IaC

Consistency: Ensures that environments are set up in a repeatable and consistent manner, reducing human error.

Version Control: Infrastructure changes can be tracked and managed using version control systems like Git, enabling rollback and historical tracking.

Automation: Allows for automated provisioning and management of infrastructure, reducing manual intervention.

Scalability: Easily scales infrastructure up or down based on demand, using predefined templates.

Documentation: Infrastructure code serves as documentation, making it easier to understand and replicate setups.

Testability: Enables testing of infrastructure before deployment, reducing the risk of deployment issues.

Terraform vs. AWS Cloudformation

Cloud-Agnostic: Supports multiple cloud providers (AWS, Azure, GCP, etc.), making it easier to manage hybrid and multi-cloud environments.

State Management: Tracks the state of infrastructure, ensuring that changes are applied correctly and resources are not unintentionally modified.

Modularization: Allows the use of modules to break down infrastructure configurations into reusable components, improving maintainability.

Declarative Syntax: Uses a declarative language to define infrastructure, which makes it easier to understand and manage.

Plan and Apply: Terraform’s plan command allows users to preview changes before applying them, minimizing unexpected outcomes.

Extensibility: Supports custom providers and plugins, allowing for integration with various third-party tools and services.

Commands

Here are some useful Terraform commands along with brief explanations:

terraform init: a. Initialized local backend. b. Downloaded the provider plugins (initialized plugins) c. Review the folder structure .terraform folder. d. Create .terraform.lock.hcl

terraform plan: Generates an execution plan, outlining actions Terraform will take.

terraform apply: Applies the changes described in the Terraform configuration.

terraform destroy: Destroys all resources described in the Terraform configuration.

terraform validate: Checks the syntax and validity of Terraform configuration files.

terraform refresh: Updates the state file against real resources in the provider.

terraform output: Displays the output values from the Terraform state.

terraform state list: Lists resources within the Terraform state.

terraform show: Displays a readable output of the current state or a specific resource's state.

terraform import: Imports existing infrastructure into Terraform state.

terraform fmt: Rewrites Terraform configuration files to a canonical format.

terraform graph: Generates a visual representation of the Terraform dependency graph.

terraform providers: Prints a tree of the providers used in the configuration.

terraform workspace list: Lists available workspaces.

terraform workspace select: Switches to another existing workspace.

terraform workspace new: Creates a new workspace.

terraform workspace delete: Deletes an existing workspace.

terraform output: Retrieves output values from a module.

terraform state mv: Moves an item in the state.

terraform state pull: Pulls the state from a remote backend.

terraform state push: Pushes the state to a remote backend.

terraform state rm: Removes items from the state.

terraform taint: Manually marks a resource for recreation.

terraform untaint: Removes the 'tainted' state from a resource.

terraform login: Saves credentials for Terraform Cloud.

terraform logout: Removes credentials for Terraform Cloud.

terraform force-unlock: Releases a locked state.

terraform import: Imports existing infrastructure into your Terraform state.

terraform plan -out: Saves the generated plan to a file.

terraform apply -auto-approve: Automatically applies changes without requiring approval.

terraform apply -target-resource: Applies changes only to a specific resource.

terraform destroy -target-resource: Destroys a specific resource.

terraform apply -var="key=value": Sets a variable's value directly in the command line.

terraform apply -var-file=filename.tfvars: Specifies a file containing variable definitions.

terraform apply -var-file=filename.auto.tfvars: Automatically loads variables from a file.

Terraform Installation Commands


sudo apt-get update && sudo apt-get install -y gnupg software-properties-common

wget -O- https://apt.releases.hashicorp.com/gpg | \
gpg --dearmor | \
sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg > /dev/null

gpg --no-default-keyring \
--keyring /usr/share/keyrings/hashicorp-archive-keyring.gpg \
--fingerprint

echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] \
https://apt.releases.hashicorp.com $(lsb_release -cs) main" | \
sudo tee /etc/apt/sources.list.d/hashicorp.list

sudo apt update

sudo apt-get install terraform


# Terraform Initialize
terraform init
Observation:
1) Initialized Local Backend
2) Downloaded the provider plugins (initialized plugins)
3) Review the folder structure ".terraform folder"
4) Create .terraform.lock.hcl

# Terraform Validate
terraform validate
Observation:
1) If any changes to files, those will come as printed in stdout

# Terraform Plan
terraform plan
Observation:
1) No changes - Just prints the execution plan

# Terraform Apply
terraform apply
[or]
terraform apply -auto-approve
Observations:
1) Create resources on cloud
2) Created terraform.tfstate file when you run the terraform apply command

Terraform Block

This is a block type which is used to configure the behaviours of Terraform itself.

Required Terraform Version

List Required Providers

Terraform Backend


terraform {
  required_version = "~> 0.14" # Allow = 0.15 Deny = 1.14
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

Provider Block

A provider in Terraform is a plugin that enables interaction with an API.

As a best practice, we should lock providers to a specific version. This ensures that updates to the provider don't unexpectedly break your code.


provider "aws" {
    region = "us-west-2" 
    access_key = "ExAmpLEKey21fjwljsfjkrf"
    secret_key = "EXamPLeSECreTKEySDAHhhhsJJn23Ksda"
}

Data sources

Data sources allow data to be fetched for use elsewhere in the Terraform configuration.

Provisioners

Provisioners are used to execute scripts on a local or remote machine.

It can be run either run locally (on the same system where Terraform commands are being issued from), or remote resources created through terraform

For example: We can code our config file that when we create a new EC2 instance, Terraform should also execute a script which installs Nginx web-server.

There are two main types of provisioners: local-exec and remote-exec.

1. Local Exec:

Local Exec Provisioner allows Terraform to execute scripts or commands locally on the machine where Terraform is running.

It is ideal for tasks such as initializing configurations or setting up dependencies.


resource "null_resource" "mk" {
  provisioner "local-exec" {
    command = "echo '0' > status.txt"
  }
  provisioner "local-exec" {
    when    = destroy
    command = "echo '1' > status.txt"
  }

2. Remote Exec

It allows Terraform to execute scripts or commands on the provisioned resources themselves. It’s useful for tasks like software installation or configuration management.


resource "aws_instance" "example" {
  # Resource configuration
  
  provisioner "remote-exec" {
    inline = [
      "sudo apt update",
      "sudo apt install nginx -y",
    ]
  }

Note: Provisioners are recommended for use only when we want to invoke actions that not covered by terraform declarative model.

State

Terraform stores the state of the infrastructure that is cretaed in the Terraform State File.

It is critical to Terraforms functionality.

Itis stored in flat files, by default named “terraform.tfstate”

terraform state list: Lists all resources tracked by the Terraform state file. terraform state rm: Deletes a resource from the Terraform state file. terraform state show: Displays details of a resource tracked in the Terraform state file.

Note: Terraform leaves behind a terraform.tfstate.backup file in case you need to recover to the last deployed Terraform state.

Remote Backend

It is basically saving the state file to a remote data source, such as AWS S3 or Google Storage,

Which means that everyone in a team can work with the same state file.

Below is an example to establish an s3 remote backend for your Terraform state files.

State Locking

Whenever you are performing write operation, terraform will lock the state file. This is very important because whenever we are doing terraform apply, if others also try for the same, it can corrupt your state file.

If state locking fails, Terraform will not continue.

State locking happens automatically on all operations that could write state. You won't see any message that it is happening.

You can create an s3 bucket in a terraform config like so:


# create an S3 bucket to store the state file in
resource "aws_s3_bucket" "terraform-state-storage-s3" {
    bucket = "terraform-remote-state-storage-s3"
    versioning {
      enabled = true
    }
    lifecycle {
      prevent_destroy = true
    }
    tags {
      Name = "S3 Remote Terraform State Store"
    }      
}
terraform {
  backend "s3" {
    bucket = "mybucket"
    key    = "path/to/my/key" #path within the S3 bucket for the state file.
    region = "us-east-1"
  }
}

Then create the s3 backend resource like so:


terraform {
 backend “s3” {
 encrypt = true
 bucket = "terraform-remote-state-storage-s3"
 region = us-west-2
 key = path/to/state/file
 }
}

Create the dynamoDB table like this:


# example.tf
# create a dynamodb table for locking the state file
resource "aws_dynamodb_table" "dynamodb-terraform-state-lock" {
  name = "terraform-state-lock-dynamo"
  hash_key = "LockID"
  read_capacity = 20
  write_capacity = 20
 
  attribute {
    name = "LockID"
    type = "S"
  }
 
  tags {
    Name = "DynamoDB Terraform State Lock Table"
  }
}

You will need to modify the Terraform S3 backend resource and add in the dynamoDB table:


# terraform.tf
terraform {
 backend “s3” {
 encrypt = true
 bucket = "terraform-remote-state-storage-s3"
 dynamodb_table = "terraform-state-lock-dynamo"
 region = us-west-2
 key = path/to/state/file
 }
}

Force Unlocking State

Terraform has a force-unlock command to manually unlock the state if unlocking failed. If you unlock the state when someone else is holding the lock it could cause multiple writers. Force unlock should only be used to unlock your own lock in the situation where automatic unlocking failed.

Variables

Variables let us customize the code without altering the main source code.

Code


# Input Variables
# AWS Region
variable "aws_region" {
  description = "Region in which AWS Resources to be created"
  type = string
  default = "us-east-1"
}

# AWS EC2 Instance Type
variable "instance_type" {
  description = "EC2 Instnace Type"
  type = string
  default = "t3.micro"
}

# AWS EC2 Instance Key Pair
variable "instance_keypair" {
  description = "AWS EC2 Key Pair that need to be associated with EC2 Instance"
  type = string
  default = "terraform-key"
}

Outputs

After running terraform apply, output variable values are displayed in the terminal.

They work like return values, helping us track important information after a successful terraform apply.

We can use the optional -out=FILE option to save the generated plan to a file on disk, which you can later execute by passing the file to terraform apply as an extra argument.

Code


# Terraform Output Values
output "instance_publicip" {
  description = "EC2 Instance Public IP"
  value = aws_instance.myec2vm.public_ip
}

output "instance_publicdns" {
  description = "EC2 Instance Public DNS"
  value = aws_instance.myec2vm.public_dns
}

Datasource


# Get latest AMI ID for Amazon Linux2 OS
# Get Latest AWS AMI ID for Amazon2 Linux
data "aws_ami" "amzlinux2" {
  most_recent = true
  owners = [ "amazon" ]
  filter {
    name = "name"
    values = [ "amzn2-ami-hvm-*-gp2" ]
  }
  filter {
    name = "root-device-type"
    values = [ "ebs" ]
  }
  filter {
    name = "virtualization-type"
    values = [ "hvm" ]
  }
  filter {
    name = "architecture"
    values = [ "x86_64" ]
  }
}

Provisioners


#Local Exec:

resource "aws_instance" "web" {
 # …
 provisioner "local-exec" {
 command = "echo ${aws_instance.web.private_ip} >> private_ips.txt"
 }
}


# Remote Exec:
resource "aws_instance" "web" {
 # …
 provisioner "remote-exec" {
 …………………………...
 }
}

Modules

A module is a container for multiple resources that are used together.

Modules works on DRY Principle i.e Don't repeat yourself.

If we have a pattern which is repetitive, there is no need to write code again and again.

Why Terraform module?

As the Terraform configurations become more complex, modules can be used to organize and simplify them.

We can also share modules within your organization or with the Terraform community.

Every Terraform configuration has at least one module, called the root module, which consists of code files in your main working directory.


├── main.tf
├── modules
│   ├── api-1
│   │   ├── api1.tf
│   │   ├── user-data.tpl
│   │   └── variables.tf
│   ├── api-2
│   │   ├── api2.tf
│   │   └── variables.tf
│   └── network
│       └── aws
│           ├── network.tf
│           └── variables.tf
├── provider.tf
└── terraform.tfvars

Dynamic Block

Dynamic block is used to dynamically generate nested content within a resource.

Example: Let's say we are defining a security group and we need to define multiple ingress rules.

Here we only have two, but what if we needed 40 ingress rules to cover different ports? That would require 40 different blocks with the current syntax/structure of the code.

So we can use a dynamic block instead:

Taint

The terraform taint command tells Terraform that a specific resource is broken or needs to be replaced.

Example:

Let’s say we create a new EC2 resource in Terraform, and users make a lot of manual changes both to the infrastructure and inside the server. Now the EC2 resource will not match the Terraform code.

There are two ways to handle this:

Import the changes into Terraform.

Delete and recreate the resource.

With the terraform taint command, we can manually mark a Terraform-managed resource as tainted. This will force it to be destroyed and recreated on the next terraform apply like:

terraform taint aws_instance.my_ec2

Note how this applies to a specific resource rather than the whole template.

The terraform taint command does not modify the infrastructure directly. It merely alters the status of the resource in the state file to "tainted," marking it for destruction and recreation on the next terraform apply.

Why Use Taint?

When a resource declaration is modified, Terraform usually attempts to update the existing resource in place (although some changes can require destruction and recreation due to upstream API limitations).

In some cases, you might want a resource to be destroyed and recreated even when Terraform doesn’t think it’s necessary.

Note that the terraform taint command has been deprecated in Terraform v0.15.2 onwards.

The best practice now is to use terraform apply with the -replace flag.

Example: terraform apply -replace aws_instance.my_ec2

Local Values


locals {
  azs = data.aws_availability_zones.available.names
}

data "aws_availability_zones" "available" {}

resource "random_id" "random" {
  byte_length = 2
}

resource "aws_subnet" "mtc_public_subnet" {
  count                   = length(var.public_cidrs)
  vpc_id                  = aws_vpc.mtc_vpc.id
  cidr_block              = var.public_cidrs[count.index]
  map_public_ip_on_launch = true
  availability_zone       = local.azs[count.index] # Using local to reduce the code. 

  tags = {
    Name = "mtc_public_${count.index + 1}"
  }
}

resource "aws_subnet" "mtc_private_subnet" {
  count                   = length(var.private_cidrs)
  vpc_id                  = aws_vpc.mtc_vpc.id
  cidr_block              = var.private_cidrs[count.index]
  map_public_ip_on_launch = false
  availability_zone       = local.azs[count.index] # Using local to reduce the code.

  tags = {
    Name = "mtc_private_${count.index + 1}"
  }
}

Terraform Import

Write the configuration which matches the resource configuration as shown:


resource "aws_instance" "ec2_example" {

}

Run the below command to import the resource.


terraform import aws_instance.ec2_example i-097f1ec37854d01c2

Complete the code according to the imported the resource.


resource "aws_instance" "ec2_example" {
  ami                                  = "ami-06ce824c157700cd2"
  instance_type                        = "t2.micro"
  tags = {
    "Name" = "my-test-ec2"
  }
}

Verify the resource according to by running terraform plan and apply.

File Function


resource "aws_instance" "myec2vm" {
  instance_type = "t3.micro"
  user_data = file("${path.module}/app1-install.sh")
}

Security Groups


# Create Security Group - SSH Traffic
resource "aws_security_group" "vpc-ssh" {
  name        = "vpc-ssh"
  description = "Dev VPC SSH"
  ingress {
    description = "Allow Port 22"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    description = "Allow all ip and ports outbound"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "vpc-ssh"
  }
}

# Create Security Group - Web Traffic
resource "aws_security_group" "vpc-web" {
  name        = "vpc-web"
  description = "Dev VPC Web"
  ingress {
    description = "Allow Port 80"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    description = "Allow Port 443"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  egress {
    description = "Allow all ip and ports outbound"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "vpc-web"
  }
}