Optimising Terraform-based GCP Secret Definitions

GCP Secrets and Why to Set Them Up Using Terraform

When deploying software on cloud services, we often need some persistent variables whose values stay consistent unless manually changed. There are several reasons why we often don't want these in code:

• Anyone with access to the code can view and alter the variable contents (cybersecurity risks). Ideally we want to be able to define who has the permission to view the contents of these variables and who has the permissions to edit them.
• Leaked code leads to leaked variable values, this is especially risky when handling sensitive fields such as API keys and account credentials.
• Only engineers can make required changes to these variables, functional individuals are generally unable to unless trained.
• Changes are slow as they require code reviews.

That's where GCP secrets come in. They allow:

• Values of secrets to be altered only within the UI or through the CLI.
• The definition of specific IAM Roles that control who can view and who can edit said secrets.
• You can rest safe in the knowledge that they are protected by Google cybersecurity mechanisms.
• Code running within GCP can access these quuite easily.

The Normal Way of Setting Up GCP Secrets with Terraform

resource "google_secret_manager_secret" "my_secret_one" {
  secret_id = "my_secret_one"

  replication {
    user_managed {
      replicas {
        location = var.region
      }
    }
  }
}

resource "google_secret_manager_secret" "my_secret_two" {
  secret_id = "my_secret_two"

  replication {
    user_managed {
      replicas {
        location = var.region
      }
    }
  }
}

What's Wrong With This?

• Takes 10 lines of code to set up a single secret, makes Terraform files massive.
• Time-consuming to make similar changes to multiple secret names.
• Time-consuming to make similar changes to the replication strategies of multiple secrets.
• Doesn't include IAM roles.

A (potentially) Better Way

resource "google_secret_manager_secret" "my_secret_definitions" {
    for_each  = toset(local.all_secrets)
    secret_id = each.key
    replication {
        automatic = true
    }
}

resource "google_secret_manager_secret_iam_member" "my_secrets" {
    for_each  = toset(local.all_secrets)
    secret_id = each.key
    role      = "roles/secretmanager.secretAccessor"
    member    = "serviceAccount:${google_service_account.my_service_account.email}"
}

locals {
    favourite_animal_of_each_group_secret = [
        "bird",
        "mammal",
        "fish",
        "amphibians",
        "reptiles"
    ]

    name_of_friend = [
        "aaron",
        "peter",
        "jessica",
        "sophia",
        "terence",
    ]

    friend_phone_numbers = [for friend in local.name_of_friend : "${friend}-phone-number"]
    friend_addresses = [for friend in local.name_of_friend : "${friend}-current-address"]

    other_secrets = [
        "github-password",
        "myspace-password"
    ]

    all_secrets = concat(local.favourite_animal_of_each_type_secret, local.favourite_food_of_different_type_secret, local.friend_phone_numbers, local.friend_addresses, local.other_passwords)
}

The above lets me put together 17 secrets in less than 30 lines of code meanwhilst the previous implementation would have taken about 200 lines if you include spacing even though the previous implementation didn't even include IAM roles.

• We define the changeable parts of secret names inside local variable arrays. locals{test = ["my_test"]}
• We update all secret names within these arrays with the same prefix and suffix (which can be updated in a single location when required): [for t in local.test : "${t}-is-a-test"]
• We concatenate all the different arrays of secret names into a single array all_secrets = concat(local.test)
• We are able to create a secret and IAM role for each name defined in the concatenated array by doing

one for_each  = toset(local.all_secrets)
secret_id = each.key

Limitations

All secrets in a local variable array have to follow a similar naming convention.