Terraform: General Concepts

General notes and prep for HCTA0-003.

Configuration Language

Uses a consistent (uniform) vocabulary to configure deployments and resources.
- (Input, Data, Local) Variables, Resources, Providers
Abstracts away many otherwise manual CLI operations.
Allows configuration reuse (through Modules and imports).
Typed configuration helps to validate and avoid errors/mistakes.
- Built in formatting and file validation.

Providers

(Mostly) Cloud Agnostic - the Configuration Language itself is Cloud Agnostic but you'll need to hook in Cloud-specific Providers.
Providers allow the same Configuration Language to deploy resources to many cloud environments (and even locally, on Docker).
Providers typically need an underlying CLI or SDK (such as awscli).

Stateful

Terraform uses the root of a dir as its Context.
Any file that's imported or present at root dir will be evaluated or considered automatically.
Terraform State tracks the definitions provided for within the .tf configs.
- Also data Variables retrieved from a Cloud Provider
- And the live status of any resources created, updated, running, or destroyed within the Cloud by those .tf configs.

Interactive

Supports interactive Shell commands:

terraform fmt - will format .tf files
terraform validate - will validate .tf returning Success! The configuration is valid.
terraform apply - deploys resources specified, configured, and defined in .tf files.
terraform plan - prints out the specified, configured, and defined resources without deploying them. Output can be saved into a file.
terraform show - displays the current State of the Terraform Context.

Resources and Links

Code samples:

https://github.com/Thoughtscript/terraform_2024

Terraform HCTA0-003: General Concepts

Backend

Terraform Backends specify where Terraform stores its State File.
Storing a State File in a remote location is a beneficial practice that can improve security.
Some Backends support multiple Workspaces.
- https://developer.hashicorp.com/terraform/language/state/workspaces#backends-supporting-multiple-workspaces

A Backend is configured within the terraform block with the keyword backend:

terraform {
  backend "remote" {
    organization = "example_corp"

    workspaces {
      name = "my-app-prod"
    }
  }
}

https://developer.hashicorp.com/terraform/language/backend

Workspaces

Support multiple deployments of the same configuration.
Each Workspace has its own State File.
If a Backend is configured, the State File will be created using the configured (remote) Backend.

terraform workspace new MY_WORKSPACE
terraform workspace select MY_WORKSPACE
terraform workspace show
terraform workspace list

https://spacelift.io/blog/terraform-workspaces

https://developer.hashicorp.com/terraform/language/state/workspaces

State

Community Terraform will persist Secrets to State in a plaintext and unecrypted way (although this can be masked in say the Terminal).
HCP Terraform will encrypt Secrets in State at rest.
Saved into a local terraform.tfstate file in the default local directory: terraform.tfstate.d/

Default Files and Directories

Files created by and used within Terraform:

.terraform.tfstate local State file saved into the default local directory: terraform.tfstate.d/
.terraform/providers is the default location where Providers are downloaded, saved, and cached locally.
secret.tfvars file where Secrets and sensitive values are stored.
credentials.tfrc.json - where API Tokens retrieved from terraform login are stored.

Important TF Variables

TF_VAR - Affixed. Format: TF_VAR_name.
- Specifies a Terraform Environment Variable.
- Note: it's not TF_ENV or TF_ENV_VAR!
TF_LOG - Specifies the Logging Visibility Level.
- Example: TF_LOG=TRACE

https://developer.hashicorp.com/terraform/cli/config/environment-variables

Logging Visibility Levels

In descending order:

TRACE
DEBUG
INFO
WARN
ERROR
OFF

User-Defined Functions

Terraform doesn't presently support User-Defined Functions:

"The Terraform language does not support user-defined functions, and so only the functions built in to the language are available for use."
One can combine pre-existing, in-built, Functions in creative ways but one cannot define say a Function or Function name.

https://developer.hashicorp.com/terraform/language/functions

Resources and Links

Code samples:

https://github.com/hashicorp/terraform-guides/tree/master

Terraform HCTA0-003: Sensitive Values

Secrets

Terraform supports the following ways to store and manage Secrets:

secret.tfvars file
sensitive function
sensitive keyword

Secrets and State

Secrets are always saved/stored in a State File as plaintext.
HCP Terraform will encrypt the State File itself and allow it to be saved to a remote location. Secrets will nevertheless be persisted in plaintext within that State File.
The sensitive keyword only suppresses/masks a value within the console. (It doesn't prevent the value from being persisted into a State File as plaintext.) (See below for more details.)

Sensitive Function

The sensitive function:

Treats a value as if it were marked with the sensitive keyword (below).
Allows values to be stored elsewhere (outside of a particular configuration file) in a manner akin to Pickling/Unpickling (in Python) or the serialized keyword (in Java).
The actual value will nevertheless be visible in the local State File and in some scenarios (like certain logging messages).

locals {
  sensitive_content = sensitive(file("${path.module}/sensitive.txt"))
}

Sensitive Keyword

The sensitive keyword suppresses the value of a Variable from a terraform plan or terraform apply output:

variable "user_information" {
  type = object({
    name    = string
    address = string
  })
  sensitive = true
}

There are some important limitations:

sensitive values may still appear in error messages or logs (regardless of their being suppressed in the CLI).
Any value that relies on a sensitive value will likewise be treated as sensitive.
A sensitive value may still be used or accessed by a person with the ability to do so.
- Access is not controlled or restricted by the sensitive keyword.
sensitive values are nevertheless persisted into a State File if no other backend or store has been configured.
- These values will be both persisted in a local file and unencrypted!
- So for Production environments, one will want to use something like HashiCorp Vault or AWS KMS (with at-rest encryption).
An entire block of values may be suppressed if the block is considered sensitive.
- Use the keyword sparingly and precisely.

https://developer.hashicorp.com/terraform/language/values/variables#suppressing-values-in-cli-output

View Sensitive Value

You may need to see a sensitive value in the CLI:

terraform output 'MY_RESOURCE_KIND.MY_RESOURCE_NAME'
terraform show or terraform show -json

Know too that the above represent limitations to the level of secrecy afforded by Terraform out-of-the-box.

Resources and Links

Terraform HCTA0-003: Syntax

Dyamic Blocks

dynamic blocks support for_each functionality when used in tandem with a list Type.

Example:

variable "settings" {
  type = list(map(string))
}

resource "aws_elastic_beanstalk_application" "tftest" {
  name        = "tf-test-name"
  description = "tf-test-desc"
}

resource "aws_elastic_beanstalk_environment" "tfenvtest" {
  name                = "tf-test-name"
  application         = "${aws_elastic_beanstalk_application.tftest.name}"
  solution_stack_name = "64bit Amazon Linux 2018.03 v2.11.4 running Go 1.12.6"

  dynamic "setting" {
    for_each = var.settings
    content {
      namespace = setting.value["namespace"]
      name = setting.value["name"]
      value = setting.value["value"]
    }
  }
}

https://developer.hashicorp.com/terraform/language/expressions/dynamic-blocks

Splat Expressions

Consider the following valid Terraform Comprehension-esque Expression: [for o in var.list : o.id].
It's equivalent to the more succinct Splat Expression: var.list[*].id.

https://developer.hashicorp.com/terraform/language/expressions/splat

Resources and Links

Code samples:

https://github.com/hashicorp/terraform-guides/tree/master

Terraform HCTA0-003: Modules

It might be intuitive to think of a Module that calls or references another as Subclassing the other. That's not the case:

A Child Module is a Module that's called or referenced (through a source value) from another Module.
The calling Module is the Parent Module.

Example:

# Must be named or have a main.tf in Module dir.
## Helpful AWS examples: https://github.com/terraform-aws-modules/terraform-aws-ecs/tree/master/modules/container-definition

# Input Variables
variable "policy_arn" {
  description = "Name of specified policy"
  type        = string
}

variable "policy_attachment_name" {
  description = "Name of policy attachment"
  type        = string
}

variable "input_aws_iam_user_name" {
  description = "Supplied AWS IAM User"
  type        = string
}

resource "aws_iam_policy_attachment" "this" {
  name       = var.policy_attachment_name
  users      = [var.input_aws_iam_user_name]
  policy_arn = var.policy_arn
}

# Output Variables
output "id" {
  description = "ID of the policy"
  value       = aws_iam_policy_attachment.this.id
}

https://github.com/Thoughtscript/terraform_2024/blob/main/terraform/modules/policy/main.tf

https://developer.hashicorp.com/terraform/tutorials/certification-associate-tutorials-003/module-create

https://developer.hashicorp.com/terraform/language/modules

Input Variables

Modules can define Input Variables that are Parameterized when the Module is imported and used.
These are akin to an Argument or Parameter in Object Oriented Programming Langauges.
Input Variables use the same variable block syntax as typical Variables but typically lack a value and typically have a default field.

https://developer.hashicorp.com/terraform/language/modules

Exporting Variables

A Module won't automatically have access to all the Variables of a child or called Module, only the exported ones.
Exported Variables are called Output Variables.
These are often placed in an outputs.tf file (within a Module directory).

These are defined using output blocks like so:

output "instance_ip_addr" {
  value = aws_instance.server.private_ip
}

https://developer.hashicorp.com/terraform/language/values/outputs

Resources and Links

Code samples:

Terraform HCTA0-003: Resource Drift

Importing

Terraform recommends using the import keyword to indicate that a Resource will come under Terraform management following terraform apply.

Example: a Resource might have been created manually with the need to add it to Terraform later.

import {
  to = aws_instance.example
  id = "i-abcd1234"
}

resource "aws_instance" "example" {
  name = "hashi"
  # (other resource arguments...)
}

https://developer.hashicorp.com/terraform/language/import

https://developer.hashicorp.com/terraform/tutorials/certification-associate-tutorials-003/resource-drift#import-the-security-group

Refresh

Some HCTA0-03 questions asked about terraform refresh which wasn't covered in the Udemy course I studied from.

terraform refresh is now deprecated.
Is an alias for terraform apply -refresh-only -auto-approve.
terraform apply -refresh-only is now officially recommended and prefered.

https://developer.hashicorp.com/terraform/cli/commands/refresh

Resources and Links

Terraform HCTA0-003: Commands

The big six:

terraform init
terrorm fmt
terraform validate
terraform plan
terraform apply
terraform destroy

Initialization and Setup

# Fetch Providers, dependencies, and external Modules
terraform init

# Download Modules or update them
## Akin to to 'npm install'
terraform get

# Just update the existing ones
terraform get -update

Workspaces

# Display all Workspaces
terraform workspace list

# Create a new Workspace MY_WORKSPACE_NAME
terraform workspace new MY_WORKSPACE_NAME

# Switch to an existing Workspace MY_WORKSPACE_NAME
terraform workspace select MY_WORKSPACE_NAME

Formatting and Validation

# Format Terraform files
terraform fmt

# Also subdirectories
terraform fmt --recursive

# Validate Terraform files for internally consistency
## Naming, proper initialization, etc.
terraform validate

Viewing State

# List the contents of State
terraform state list

# Display a single Resource in State
terraform state show 'MY_RESOURCE_TYPE.MY_RESOURCE_NAME'
## 'terraform show' doesn't allow this level of granularity

# Refresh State
terraform refresh

## The above is an Alias for:
terraform apply -refresh-only -auto-approve

# Display the Output Variables and Values of a Module and/or Resource
terraform output

Planning

# Refreshes State and creates the Execution Plan
## Without modifying the actual Resources
## Dry Run
terraform plan

## Create a Destroy Plan only
terraform plan -destroy

Modifying Resources

# Refresh State and Apply changes to actual Resources!!
terraform apply

# Destroy all Resources!!
terraform destroy

## Alias for:
terraform apply -destroy

# Replace a specific Resource
## Destroy and recreate it
terraform apply -replace="MY_RESOURCE_KIND.MY_RESOURCE_NAME"
## This supersedes 'terraform taint'

Unlock State

# Force State to be unlocked
terraform force-unlock

Console and Saving Outs

# Display State or Execution Plan as JSON
terraform show -json
## For Resource specific granulatity - 'terraform state show ...'

# Print out State or Execution Plan as JSON File
terraform show -json MY_FILENAME

# Print Execution Plan out
terraform plan -out=my/path/file.json

# Evaulate Expressions and Std Out.
terraform console

Credentials and Sensitive Values

# Log into HCP Terraform and obtain an API Token
terraform login

## Logout of HCP Terraform
terraform logout

# Reveal all Values included Sensitive ones
terraform show

# Display a potentially Sensitive value from an Output
terraform output 'MY_RESOURCE_KIND.MY_RESOURCE_NAME'