One of the sensitive points in Terraform projects are the famous state files or tfstates ( https://developer.hashicorp.com/terraform/language/state ), and when it comes to more dynamic use as we will do, in pipelines and PODs, their control is a difficult task.
To make things easier, there are some Backend modules for Terraform, which allow states to be written to databases, for example, or Consul, etc.
In our case, I preferred to use PostgreSQL, as we already had an instance created and it was easier to control the schemas.
With this, we can always have PODs created and destroyed, but the states will always be persisted in the database.
For testing purposes, I will not use K8s, but Docker for ease. But nothing changes.
Database.
The first step is to create a database for the states in our PostgreSQL https://devops-db.com/vault-postgresql-as-backend/ .
psql -h localhost -p 5432 -U postgres
CREATE DATABASE db_terraform_backend;
CREATE USER terraform_backend WITH PASSWORD '1Ov6DWitPlq*QyYL';
## Change to new database
\c db_terraform_backend;
GRANT ALL PRIVILEGES ON DATABASE db_terraform_backend TO terraform_backend;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO terraform_backend;
GRANT USAGE, CREATE ON SCHEMA public TO terraform_backend;
SELECT * FROM pg_catalog.pg_tables where schemaname = 'terraform_remote_state';
schemaname | tablename | tableowner | tablespace | hasindexes | hasrules | hastriggers | rowsecurity
------------------------+-----------+-------------------+------------+------------+----------+-------------+-------------
terraform_remote_state | states | terraform_backend | | t | f | f | f
Terraform code.
The example of the terraform code that I show is as simple as possible, just to show, by a variable that is sent as a parameter, nothing more.
The important thing in the code is the first backend part in main.tf. There you need to have the PostgreSQL connection string.
The codes are in our repository: https://github.com/faustobranco/devops-db/tree/master/knowledge-base/terraform/postgresql_backend
main.tf
terraform {
backend "pg" {
conn_str = "postgres://terraform_backend:1Ov6DWitPlq*QyYL@postgresql.devops-db.internal:5432/db_terraform_backend?sslmode=disable"
schema_name = "terraform_remote_state"
}
}
variable "destination_email" { type = string }
results.tf
output "parameter_destination_email" {
value = var.destination_email
}Test.
To make it easier to use the code, I run the Docker commands in the same folder where the codes are. Something I won’t need in the pipeline codes, as I’ll check out the codes directly from Git to the POD.
So let’s init the directory:
docker run --rm -it -v $PWD:/postgresql -w /postgresql ubuntu_terraform:1.0.0 terraform init
Initializing the backend...
Initializing provider plugins...
Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.Now that the init is done, let’s create a plan. See that there is an indication of a new item "+ parameter_destination_email = "anyone@test.com""
docker run --rm -it -v $PWD:/postgresql -w /postgresql ubuntu_terraform:1.0.0 terraform plan -var destination_email="anyone@test.com"
Changes to Outputs:
+ parameter_destination_email = "anyone@test.com"
You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.Now we do the apply:
docker run --rm -it -v $PWD:/postgresql -w /postgresql ubuntu_terraform:1.0.0 terraform apply -var destination_email="anyone@test.com"
Changes to Outputs:
+ parameter_destination_email = "anyone@test.com"
You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
parameter_destination_email = "anyone@test.com"With this, tfstate should already be populated, not the file, but a table called state in PostgreSQL.
psql -h localhost -p 5432 -U postgres
Password for user postgres:
psql (17.2 (Ubuntu 17.2-1.pgdg22.04+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off, ALPN: postgresql)
Type "help" for help.
postgres=# \c db_terraform_backend;
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off, ALPN: postgresql)
You are now connected to database "db_terraform_backend" as user "postgres".
db_terraform_backend=# Select count(*) from terraform_remote_state.states;
count
-------
1
(1 row)If you still want to run the plan again:
docker run --rm -it -v $PWD:/postgresql -w /postgresql ubuntu_terraform:1.0.0 terraform plan -var destination_email="anyone@test.com"
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.