3.01 - Terraform State

Overview

Terraform state is a JSON file that maps your configuration to real-world infrastructure, acting as the single source of truth for everything Terraform has provisioned.

Abstract

Every time Terraform creates infrastructure, it records the result in a state file (terraform.tfstate). This file is how Terraform tracks what exists, detects drift, and determines what actions to take on subsequent plan and apply runs. State is non-optional — it is fundamental to how Terraform works.

Key Concepts

Concept	Description
`terraform.tfstate`	JSON file created after the first `terraform apply`; maps config to real resources
State refresh	Terraform reads current state into memory before every `plan` or `apply`
Single source of truth	State is the authoritative record of what Terraform has provisioned
Resource ID	Unique identifier assigned by Terraform/provider to each managed resource
Drift detection	Terraform compares state against config to identify changes that need to be applied
In-memory state	State is loaded into memory during plan/apply but only persisted after apply
Metadata tracking	State records resource dependencies so correct destroy order is preserved even after resources are removed from config
Cached attributes	State stores a local cache of resource attributes to avoid querying every provider on every command
`--refresh=false`	Flag that tells Terraform to use cached state rather than refreshing from providers — improves performance at scale
Remote state	State stored in a shared backend (S3, GCS, Consul, Terraform Cloud) for team collaboration and locking

The Terraform Workflow and State

Step 1 — `terraform init`

Downloads required provider plugins. No state file is created yet.

$ cd terraform-local-file
[terraform-local-file]$ terraform init

Initializing the backend...
Initializing provider plugins...
- Finding latest version of hashicorp/local...
- Installing hashicorp/local v1.4.0...
- Installed hashicorp/local v1.4.0 (signed by HashiCorp)

Terraform has been successfully initialized!

Directory contents at this point:

main.tf  variables.tf

No state file exists yet.

Step 2 — `terraform plan`

Terraform refreshes state in-memory before calculating the plan. On first run, no state exists, so Terraform determines all resources must be created.

[terraform-local-file]$ terraform plan

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan,
but will not be persisted to local or remote state storage.

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # local_file.pet will be created
  + resource "local_file" "pet" {
      + content              = "I love pets!"
      + directory_permission = "0777"
      + file_permission      = "0777"
      + filename             = "/root/pets.txt"
      + id                   = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

No state file is written during plan.

Step 3 — `terraform apply`

Executes the plan and creates the resource. A unique ID is assigned and a state file is written.

[terraform-local-file]$ terraform apply

  + content              = "I love pets!"
  + directory_permission = "0777"
  + file_permission      = "0777"
  + filename             = "/root/pets.txt"
  + id                   = (known after apply)

Plan: 1 to add, 0 to change, 0 to destroy.

Enter a value: yes

local_file.pet: Creating...
local_file.pet: Creation complete after 0s
[id=7e4db4fbfdbb108bdd04692602bae3e9bd1e1b68]

Verify the file was created:

$ cat /root/pets
I love pets!

Step 4 — Re-running `terraform apply`

Terraform refreshes state, finds the resource already exists with the same ID, and takes no action.

[terraform-local-file]$ terraform apply

local_file.pet: Refreshing state...
[id=7e4db4fbfdbb108bdd04692602bae3e9bd1e1b68]

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

This is only possible because the state file was written after the first apply.

The State File (`terraform.tfstate`)

After the first terraform apply, the directory now contains a state file:

main.tf  variables.tf  terraform.tfstate

The state file is a JSON document containing the complete record of all provisioned infrastructure:

[terraform-local-file]$ cat terraform.tfstate

{
  "version": 4,
  "terraform_version": "0.13.0",
  "serial": 1,
  "lineage": "e35dde72-a943-de50-3c8b-1df8986e5a31",
  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "local_file",
      "name": "pet",
      "provider": "provider[\"registry.terraform.io/hashicorp/local\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "content": "I love pets!",
            "content_base64": null,
            "directory_permission": "0777",
            "file_permission": "0777",
            "filename": "/root/pets.txt",
            "id": "7e4db4fbfdbb108bdd04692602bae3e9bd1e1b68",
            "sensitive_content": null
          }
        }
      ]
    }
  ]
}

The state file stores every attribute of every managed resource, including the resource ID, provider, type, logical name, and all configuration values.

State and Drift Detection

When you change a configuration value and re-run terraform plan or apply, Terraform compares the state file against the new configuration to detect drift.

Example: Changing content from "I love pets!" to "We love pets!" in variables.tf:

$ terraform plan

Refreshing Terraform state in-memory prior to plan...
local_file.pet: Refreshing state...
[id=7e4db4fbfdbb108bdd04692602bae3e9bd1e1b68]

Terraform detects the difference and proposes a replacement:

$ terraform apply

  # local_file.pet must be replaced
  -/+ resource "local_file" "pet" {
      ~ content = "I love pets!" -> "We love pets!"  # forces replacement
        directory_permission = "0777"
        file_permission      = "0777"
        filename             = "/root/pets.txt"
      ~ id      = "7e4db4fbfdbb108bdd04692602bae3e9bd1e1b68" -> (known after apply)
    }

After apply, the state file is updated with the new resource ID and content value — the old entry is gone and replaced:

"content": "We love pets!",
"id": "7e4db4fbfdbb108bdd04692602bae3e9bc4d1c14"

At this point, the configuration file and state file are back in sync — a subsequent apply shows no changes.

Purpose of Terraform State

Beyond mapping resources to config, state serves three additional critical purposes.

1. Tracking Metadata (Dependencies)

State records dependency relationships between resources. This matters most when resources are removed from the configuration — at that point the dependency information no longer exists in the config files, so Terraform reads it from state to determine the correct destruction order.

Example configuration with three resources:

resource "local_file" "pet" {
  filename = "/root/pet.txt"
  content  = "My favorite pet is ${random_pet.my-pet.id}!"
}

resource "random_pet" "my-pet" {
  length = 1
}

resource "local_file" "cat" {
  filename = "/root/cat.txt"
  content  = "I like cats too!"
}

Creation order on terraform apply:

local_file.cat and random_pet.my-pet are independent — they are created in parallel first. local_file.pet depends on random_pet.my-pet so it is created last.

local_file.cat: Creating...
random_pet.my-pet: Creating...
local_file.cat: Creation complete after 0s [id=fe448888891fc40342313bc44a1f1a8986520c89]
random_pet.my-pet: Creation complete after 0s [id=yak]

local_file.pet: Creating...
local_file.pet: Creation complete after 0s [id=28b373c6c1fa3fce132a518eadd0175c98f37f20]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

If local_file.pet and random_pet.my-pet are later removed from main.tf, Terraform checks state to find the stored dependency:

"dependencies": [
  "random_pet.my-pet"
]

This tells Terraform to destroy local_file.pet before random_pet.my-pet, preserving the correct destruction order even though the dependency is no longer in the config.

2. Performance

For large infrastructures with hundreds or thousands of resources across multiple cloud providers, refreshing state on every command by querying each provider API can take minutes. Terraform avoids this by caching all resource attribute values in the state file.

To skip the provider refresh entirely and rely on cached state, use the --refresh=false flag:

$ terraform plan --refresh=false

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  -/+ destroy and then create replacement

Terraform will perform the following actions:

  # local_file.cat must be replaced
  -/+ resource "local_file" "pet" {
      ~ content = "I like cats too!" -> "Dogs are awesome!"  # forces replacement
        directory_permission = "0777"
        file_permission      = "0777"
        filename             = "/root/pets.txt"
      ~ id      = "cba595b7d9f94ba1107a46f3f731912d95fb3d2c" -> (known after apply)
    }

Plan: 1 to add, 0 to change, 1 to destroy.

Terraform uses the cached attributes from state to compute the plan, with no provider API calls made.

Note

Use --refresh=false cautiously — if infrastructure has been changed outside Terraform, the cached state may be stale and the plan inaccurate.

3. Collaboration

By default, terraform.tfstate is stored locally in the configuration directory. This works for solo development but breaks down in a team:

Every team member needs the latest state before running Terraform.
Concurrent apply runs without coordination will produce conflicting state.
A state file on one person's laptop is invisible to everyone else.

The solution is a remote state backend — a shared, centralised location where all team members read and write state:

Remote Backend	Notes
AWS S3	Widely used; combine with DynamoDB for state locking
Google Cloud Storage	Native GCS backend with built-in locking
HashiCorp Consul	Key-value store with locking support
Terraform Cloud	Managed service with locking, history, and access controls built in

Remote backends ensure the state file is always up-to-date, shared securely, and protected against concurrent writes.

Tip

Remote state configuration and state locking are covered in detail in a later section on Terraform backends.

State Considerations

State Contains Sensitive Information

The state file stores every attribute of every provisioned resource in plain-text JSON — including values that should be treated as secrets. For cloud resources like an AWS EC2 instance, this includes:

AMI ID and instance type
Private and public IP addresses
Private DNS hostnames
SSH key pairs
Root block device details (disk size, encryption status, KMS key IDs)
For databases: initial passwords

{
  "mode": "managed",
  "type": "aws_instance",
  "name": "dev-ec2",
  "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
  "instances": [
    {
      "attributes": {
        "ami": "ami-0a634ae95e11c6f91",
        "primary_network_interface_id": "eni-0ccd57b1597e633e0",
        "private_dns": "ip-172-31-7-21.us-west-2.compute.internal",
        "private_ip": "172.31.7.21",
        "public_dns": "ec2-54-71-34-19.us-west-2.compute.amazonaws.com",
        "public_ip": "54.71.34.19",
        "root_block_device": [
          {
            "delete_on_termination": true,
            "device_name": "/dev/sda1",
            "encrypted": false,
            "iops": 100,
            "kms_key_id": "",
            "volume_size": 8
          }
        ]
      }
    }
  ]
}

Because state is stored as plain-text, it must always be kept in a secure storage location.

Configuration Files vs State Files — Different Storage Rules

Your Terraform directory contains two distinct categories of files with different storage requirements:

File type	Examples	Recommended storage
Configuration files	`main.tf`, `variables.tf`, `outputs.tf`	Version control (GitHub, GitLab, Bitbucket)
State files	`terraform.tfstate`, `terraform.tfstate.backup`	Remote backend only — never in Git

Configuration files are safe to commit and collaborate on via standard version control workflows. State files must not be committed to Git due to their sensitive content.

Never Edit State Manually

The state file is a JSON data structure intended for internal use by Terraform only. Manually editing it can corrupt the file and cause Terraform to lose track of managed resources entirely.

If you need to make changes to state (e.g. move, rename, or remove a resource from state), always use the dedicated terraform state subcommands:

terraform state list          # list all resources in state
terraform state show <addr>   # show details of a specific resource
terraform state mv <src> <dst> # rename or move a resource in state
terraform state rm <addr>     # remove a resource from state without destroying it

Warning

terraform state rm removes a resource from state tracking but does not destroy the real infrastructure. Use with caution.

Best Practices

Never edit terraform.tfstate manually — corruption will cause Terraform to lose track of real infrastructure.
Store state remotely (S3, GCS, Terraform Cloud) for team environments — local state is unsuitable for collaboration.
Enable state locking (supported by remote backends like S3 + DynamoDB) to prevent concurrent apply conflicts.
Use terraform state subcommands (list, show, mv, rm) for safe state manipulation.
Run terraform plan before apply to review drift detection results before making changes.

Security Best Practices

Security

State files can contain sensitive values (passwords, keys, tokens) in plaintext — treat them as secrets.
Never commit terraform.tfstate or terraform.tfstate.backup to version control. Add them to .gitignore.
Use remote backends with encryption at rest and restricted IAM access to protect state.
Enable versioning on S3 state buckets so you can recover from accidental state corruption.

Do and Don't

✅ Do	❌ Don't
Use a remote backend for team projects	Commit `terraform.tfstate` to Git
Use `terraform state` commands for state changes	Edit `terraform.tfstate` directly in a text editor
Enable state locking for concurrent access protection	Run `terraform apply` concurrently without locking
Back up state before destructive operations	Delete or overwrite state files without a backup
Treat state files as sensitive — restrict access	Store state in a publicly accessible location

Common Mistakes

Deleting or losing the state file — Terraform will no longer know what it manages and may try to recreate all resources, causing duplicates or errors.
Editing state manually — even small formatting errors can corrupt the file and break all Terraform operations.
Using local state in a team — two engineers running apply simultaneously without locking will produce conflicting state.
Forgetting that terraform plan does not write state — only apply persists state to disk.
Assuming state is always current — if someone modifies infrastructure outside Terraform, state can drift from reality until the next refresh.

Quick Recap

terraform.tfstate is created after the first terraform apply and never during plan or init.
It is a JSON file mapping configuration resources to real-world infrastructure attributes — stored in plain-text.
State contains sensitive information (IPs, key pairs, passwords) — never commit it to Git.
Store configuration files in version control (GitHub, GitLab, Bitbucket); store state in a remote backend.
Never edit state manually — use terraform state subcommands instead.
Terraform refreshes state in-memory on every plan and apply to detect drift.
If config and state match, Terraform takes no action on re-apply.
If config and state differ, Terraform creates an execution plan to reconcile them.
State tracks resource dependencies (metadata) — used to determine correct destroy order even when resources are removed from config.
State caches attribute values for performance — use --refresh=false to skip provider API calls at scale.
State enables team collaboration — use a remote backend (S3, GCS, Consul, Terraform Cloud) with locking for shared environments.
State is updated after every successful apply.
State is non-optional — it is fundamental to Terraform's operation.

Interview / Revision Notes

Q: What sensitive data can a Terraform state file contain?
A: IP addresses, private DNS names, SSH key pairs, AMI IDs, disk configuration, and database passwords — all stored in plain-text JSON.
Q: Should you store state files in Git?
A: No. State files contain sensitive information and should be stored in a remote backend (S3, GCS, Terraform Cloud), not in version control.
Q: What is the correct way to store Terraform config vs state files?
A: Config files (main.tf, variables.tf) go in version control. State files go in a remote backend only.
Q: How should you make changes to the Terraform state file?
A: Using terraform state subcommands (list, show, mv, rm) — never by editing the JSON directly.
Q: What does terraform state rm do?
A: Removes a resource from state tracking without destroying the real infrastructure. Terraform will no longer manage that resource.
Q: What is the Terraform state file?
A: A JSON file (terraform.tfstate) that maps configuration resources to real-world infrastructure. It is Terraform's single source of truth.
Q: When is the state file created?
A: After the first successful terraform apply. It is not created by init or plan.
Q: How does Terraform know if a resource already exists?
A: It reads the state file, refreshes state in-memory, and compares against the current configuration.
Q: What happens if you run terraform apply twice without any changes?
A: Terraform refreshes state, finds no differences, and reports 0 added, 0 changed, 0 destroyed.
Q: What happens when a configuration change forces resource replacement?
A: Terraform destroys the old resource and creates a new one with a new ID. The state file is updated to reflect the new resource.
Q: Why does Terraform need to track dependency metadata in state?
A: When resources are deleted from a config file, their dependency relationships are lost. Terraform reads the stored dependencies from state to determine the correct destruction order.
Q: What is the correct destruction order for a resource that depends on another?
A: The dependent resource is destroyed first, then the resource it depends on — the reverse of creation order.
Q: What does --refresh=false do?
A: Tells Terraform to skip querying providers for current resource state and instead rely on the cached attributes in the state file. Improves performance for large infrastructures.
Q: Why is local state unsuitable for team use?
A: It sits on one person's machine, is invisible to others, and has no locking — concurrent applies will produce conflicting state.
Q: What are examples of remote state backends?
A: AWS S3 (with DynamoDB for locking), Google Cloud Storage, HashiCorp Consul, and Terraform Cloud.
Q: Why should you never edit terraform.tfstate manually?
A: Manual edits can corrupt the file, causing Terraform to lose track of managed resources and produce incorrect plans.
Q: Where should state be stored in a team environment?
A: In a remote backend (e.g. AWS S3, Terraform Cloud) with state locking enabled to prevent concurrent modification conflicts.