Terraform and OpenTofu Container Runner

What are the Terraform and OpenTofu Container Runner Drivers? #

The Terraform and the OpenTofu Container Runner Drivers provide an easy, standardized and re-usable way to execute Terraform  or OpenTofu  code in your cluster by running a container image containing the Terraform or OpenTofu executable. You may bring your own image or use the default.

The current default images run these executable versions:

• Terraform: 1.5.7
• OpenTofu: 1.9.0

The Terraform and OpenTofu Container Runner Drivers are identical in structure and therefore covered on this common page.

This Driver is a virtual driver that wraps the Container Driver. Any Resources provisioned with it will therefore display the Container Driver as the Driver type being used.

This Driver supports only AKS, EKS, and GKE clusters as a target cluster for the execution of the Kubernetes Job created by the Container Driver.

This Driver is supported in Direct mode starting from Humanitec Operator version 0.3.11 for Helm Chart and 0.16.10 for App.

The minimum Kubernetes version is 1.29.

Why use the Terraform or OpenTofu Container Runner Driver? #

Using the Terraform or OpenTofu Container Runner Driver should ensure:

• Reusability: You may want to use a dedicated Kubernetes cluster to execute your container, e.g. to run your Terraform  or OpenTofu  code. By separating out the runner configuration, it can be injected into any number Resource Definitions that should use that specific cluster to provision Resources via these Drivers.
• Separation of concerns: As the configuration details are all in the config Resource Definition as shown in the example below, the Resource Definition based on the Terraform or OpenTofu Container Runner is completely oriented to provide the container specification and the Resource configuration.
• Integrate: It provides an easy way to execute your Terraform  or OpenTofu  code via Humanitec in your infrastructure.

Compared to the Terraform Driver or Terraform Runner Driver, the Terraform and OpenTofu Container Runner Drivers:

• Allow you to have more control of what is running in your cluster. You may choose to override the default image, but you are responsible for ensuring that the image includes the appropriate Terraform or OpenTofu executable and that your use complies with the respective vendor’s licensing terms.
• Has a better defined architecture and execution model compared to the other Terraform drivers. As described herefor the underlying Container Driver:
  • Executes a Kubernetes Job composed by different sidecar containers , each of them with a specific purpose, along with the main one.
  • Has to adhere to a specific interface to interact with the Platform Orchestrator.

How it works #

The Terraform and OpenTofu Container Runner Drivers wrap the Container Driver and:

• Inject the cluster configuration referencing a config Resource Definition via the config pattern, exactly as the Container Runner Driver does but matching a config Resource Definition with class default and a Resource ID of terraform-container-runner or opentofu-container-runner, respectively

• Injects a default image to be executed by the container:

  • Terraform: hashicorp/terraform:1.5.7 
  • OpenTofu: ghcr.io/humanitec/opentofu-container-runner:1.9.0 

  This image can be adjusted through the Resource Definition inputs.

• Handles container runner outputs to be compliant with the Container Driver Contract

• Offers a way to inject variables into your Terraform or OpenTofu code

Matching different cluster configuration #

By attaching the proper Matching Criteria to the config Resource Definition, a different Container Runner configuration can be matched e.g. per Environment Type so that the Kubernetes Job Container is executed in the proper cluster.

Using Terraform and OpenTofu backends #

By default, the Drivers inject the configuration of a Kubernetes Terraform backend  or Kubernetes OpenTofu backend  that:

• Creates Kubernetes secrets in the Runner namespace
• Uses the Resource’s Globally Unique Id as secret_suffix  via the context.res.guresid placeholder, to prevent any overlapping among resources

If you want to specify a different backend:

• Set the boolean input value use_default_backend to false
• Configure a different backend in your Terraform  or OpenTofu  code
• Configure the proper Kubernetes cluster permissions as shown below

See our Terraform backends example for ways to handle backends.

Kubernetes cluster setup #

Using the Terraform Container Runner Driver requires some additional setup on the target Kubernetes cluster so that the Platform Orchestrator may manage Kubernetes Jobs using the Driver. Refer to the “Container runner” instructions for the cluster type you are using:

• AKS
• EKS
• GKE

In addition, a service account must be created in the target namespace for the Job execution itself, and it must be configured to have the permissions needed by the K8s Job to complete its execution.

Use these settings if your Terraform or OpenTofu code is using a backend other than the default Kubernetes backend:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: humanitec-runner
  namespace: humanitec-runner
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: humanitec-runner
  name: humanitec-runner
rules:
- apiGroups: [""]
  resources: ["secrets", "configmaps"]
  verbs: ["create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: humanitec-runner
  namespace: humanitec-runner
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: humanitec-runner
subjects:
- kind: ServiceAccount
  name: humanitec-runner
  namespace: humanitec-runner

Use these settings if your Terraform or OpenTofu code is using the default Kubernetes backend:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: humanitec-runner
  namespace: humanitec-runner
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: humanitec-runner
  name: humanitec-runner
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["create"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["create", "get", "delete", "list", "update", "deletecollection"]
- apiGroups: ["coordination.k8s.io"]
  resources: ["leases"]
  verbs: ["create", "get", "list", "update", "watch"]    
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: humanitec-runner
  namespace: humanitec-runner
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: humanitec-runner
subjects:
- kind: ServiceAccount
  name: humanitec-runner
  namespace: humanitec-runner

Properties #

Inputs #

Values #

Script variables #

To supply OpenTofu or Terraform variables, the field variables described in values can be used, unless you want to inject a Resource Selector.

Due to some encoding limitations, OpenTofu or Terraform variables that take their value from a Resource Selector can not be placed in the variables inputs field. Instead, you can use the files section to create a variable definitions file (Terraform , OpenTofu ) and place those variables here like this:

# Sample Resource Definition snippet showing a variable definitions file for using a Resource Selector
entity:
  driver_inputs:
    values:
      # Defining regular variables
      variables:
        app_id: ${context.app.id}
        env_id: ${context.env.id}
      # Defining a variable definitions file using a Resource Selector
      files:
        "input_vars.auto.tfvars.json": |
          {
            "principals": ${resources['gcs.default<workload>k8s-service-account'].outputs.principal}
          }

Placeholders are resolved by the Platform Orchestrator before the Driver is called. This means that the content of the Placeholder will already be resolved in the Driver inputs that the Driver receives. This includes any placeholders in comments.

For Drivers that allow pulling code from an external source such as a Git repository, Placeholders will not be resolved in that external code.

Runner object #

The runner object provides all the information to properly build the Kubernetes Job that will execute the Terraform or OpenTofu code. It is a reduced version of the Job object in the Container Driver, as some of its properties are injected by the Terraform or OpenTofu Container Runner Driver.

credentials_config object #

The credentials_config object describes how the provider credentials should be made available to the runner container.

Source object #

The source object defines how the Driver uses IaC scripts that are stored in Git. The supplied repository must be accessible to the Container K8s Job and credentials must be supplied if necessary.

According to the value specified in the ref property, the git-init init container fetches the content in the repository at a specific reference:

• If ref is a git reference (e.g. refs/heads/my-branch, refs/tags/my-tag), that branch or tag is checked out.
• If ref is a valid commit SHA, that commit is checked out.
• If ref is empty, the HEAD of the repository is checked out (generally main or master).

This means that if ref is not a commit SHA, we can’t ensure that the scripts used to provision a resource match exactly the ones used to delete the same resource as a branch (or a tag) might have been updated compared to when the resource provisioning happened.

Secrets #

Runner object #

The runner object defines the environment variables with a sensitive value the container runner runs with.

Source object #

Credentials to be used to access the Git repository. The choice of credentials depends on the url format.

Examples #

In addition to the Container Driver examples page, we provide an example that consists of:

• A config Resource Definition. It matches a res_id of terraform-container-runner and produces a cluster object as its output.
• An s3 Resource Definition using the terraform-container-runner Driver that performs provisioning of an S3 bucket via the execution of Terraform code stored in a private GitHub repository. The Terraform code will be executed in the cluster specified by the config Resource Definition. Note that no reference to the config Resource is required.

The examples show the Terraform case. Change all occurrences of the substring terraform to opentofu when using OpenTofu.

In particular, the Driver to use is humanitec/opentofu-container-runner.

Config Resource Definition #

• CLI
• API
• Terraform

cat << "EOF" > config-terraform-runner.yaml
apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: config-terraform-container-runner
entity:
  name: Config For Container Driver Runner
  type: config
  driver_type: humanitec/template
  driver_inputs:
    values:
      templates:
        outputs:
          cluster:
            account: myOrg/myAccount
            cluster:
              name: my-eks-cluster
              region: eu-north-1
              cluster_type: eks
              loadbalancer: 10.10.10.10
          skip_permission_checks: false
        secrets: |
          agent_url: {{ .driver.secrets.agent_url }}
    secrets:
      agent_url: ${resources.agent.outputs.url}
  criteria:
  - env_type: development
    res_id: terraform-container-runner
EOF

humctl create -f config-terraform-container-runner.yaml
rm config-terraform-container-runner.yaml

curl https://api.humanitec.io/orgs/${HUMANITEC_ORG}/resources/defs \
  -X POST \
  -H "Authorization: Bearer ${HUMANITEC_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '
{
  "id": "config-terraform-runner",
  "name": "Config For Container Runner Driver",
  "type": "config",
  "criteria": [
    {
      "env_type": "development",
      "res_id": "terraform-container-runner"
    }
  ],
  "driver_type": "humanitec/template",
  "driver_inputs": {
    "values": {
      "templates": {
        "outputs": "skip_permission_checks: false\ncluster:\n  account: myOrg/myAccount\n  cluster:\n    loadbalancer: 10.10.10.10\n    name: my-eks-cluster\n    region: eu-north-1\n\n    cluster_type: eks\n",
        "secrets": {
          "agent_url": "{{ .driver.secrets.agent_url }}"
        }
      }
    },
    "secrets": {
      "agent_url": "${resources.agent.outputs.url}"
    }
  }
}'

resource "humanitec_resource_definition" "config-terraform-container-runner" {
  driver_type = "humanitec/template"
  id             = "config-terraform-container-runner"
  name           = "Config For Terraform Container Runner Driver"
  type           = "config"
  driver_inputs  = {
    values_string  = jsonencode({
      "templates" = {
        "outputs" = {
          "cluster" = {
            "account" = "myOrg/myAccount"
            "cluster" = {
              "cluster_type" = "eks"
              "loadbalancer" = "10.10.10.10"
              "name"         = "my-eks-cluster"
              "region"       = "eu-north-1"
            }
          }
          "skip_permission_checks" = false
        }
        "secrets" = "agent_url: {{ .driver.secrets.agent_url }}\n"
      }
    })
    secrets_string = jsonencode({
      "agent_url" = "$${resources.agent.outputs.url}"
    })
  }
}

resource "humanitec_resource_definition_criteria" "config-terraform-container-runner_criteria_0" {
  resource_definition_id = resource.humanitec_resource_definition.config-terraform-container-runner.id 
  env_type = "development"
  res_id = "terraform-container-runner"
}

Terraform Container Runner Resource Definition #

• CLI
• API
• Terraform

cat << "EOF" > s3-tf.yaml
apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: my-s3-bucket
entity:
  name: S3 Bucket
  type: s3
  driver_type: humanitec/terraform-container-runner
  driver_account: my-aws-account
  driver_inputs:
    values:
      source:
        ref: refs/heads/main
        url: https://my-domain.com/my-org/my-repo.git
        username: my-git-handler
        path: path/to/s3
      variables:
        bucket: ${context.app.id}-${context.env.id}
        region: eu-west-3
      credentials_config:
        environment:
          AWS_ACCESS_KEY_ID: AccessKeyId
          AWS_SECRET_ACCESS_KEY: SecretAccessKey
    secret_refs:
      source:
        password:
          store: my-store
          ref: path/to/git/password
  criteria:
  - env_type: development
EOF

humctl create -f s3-tf.yaml
rm s3-tf.yaml

curl https://api.humanitec.io/orgs/${HUMANITEC_ORG}/resources/defs \
  -X POST \
  -H "Authorization: Bearer ${HUMANITEC_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '
{
  "id": "my-s3-bucket",
  "name": "S3 Bucket",
  "type": "s3",
  "criteria": [
    {
      "env_type": "development"
    }
  ],
  "driver_type": "humanitec/container-runner",
  "driver_account": "my-aws-account",
  "driver_inputs": {
    "values": {
      "source": {
        "ref": "refs/heads/main",
        "url": "https://my-domain.com/my-org/my-repo.git",
        "username": "my-git-handler",
        "path: path/to/s3"
      },
      "variables": {
        "bucket": "${context.app.id}-${context.env.id}",
        "region": "eu-west-3"
      }
    },
    "secret_refs": {
      "source":{
        "password": {
          "store": "my-store",
          "ref": "path/to/git/password"
        },
        ...
      }
    }
  }
}'

resource "humanitec_resource_definition" "my-s3-bucket" {
  driver_type = "humanitec/terraform-container-runner"
  id             = "my-s3-bucket"
  name           = "S3 Bucket"
  type           = "s3"
  driver_account = "my-aws-account"
  driver_inputs  = {
    values_string  = jsonencode({
      "source" = {
        "ref" = "refs/heads/main"
        "url" = "https://my-domain.com/my-org/my-repo.git"
        "username" = "my-git-handler"
        "path" = "path/to/s3"
      }
      "variables" = {
        "bucket" = "$${context.app.id}-$${context.env.id}"
        "region" = "eu-west-3"
      }
      "credentials_config" = {
        "environment" = {
          "AWS_ACCESS_KEY_ID" = "AccessKeyId"
          "AWS_SECRET_ACCESS_KEY" = "SecretAccessKey"
        }
      }
    })
    secret_refs    = jsonencode({
      "source" = {
        "password" = {
          "store" = "my-store"
          "ref" = "path/to/git/password"
        }
      }
    })
  }
}

resource "humanitec_resource_definition_criteria" "my-s3-bucket_criteria_0" {
  resource_definition_id = resource.humanitec_resource_definition.my-s3-bucket.id 
  env_type = "development"
}