Private module sources

The Platform Orchestrator supports mechanisms to fetch module source code from private registries and Git repositories.

Without specific configuration, your runner can only fetch module sources that are public and accessible from its network location. For example, if the runner has internet egress, it may use modules from public OpenTofu/Terraform registries and public Git repositories. If the runner only has access to an internal network, then it may fetch module source code that is hosted in the network and does not require additional credentials.

However, most organizations will wish to access module source code that is private and requires credentials to access.

Private OpenTofu/Terraform registries

If your organization hosts its module source code within a private registry, you may use Environment Variable Credentials to specify the credential used to access the registry. These variables have a name that follows the pattern TF_TOKEN_<hostname> where the registry hostname has all dots replaced with underscores. For example, when accessing modules registered in the app.terraform.io registry, the variable name will appear as TF_TOKEN_app_terraform_io.

These variables can be set in your runner configuration. For security reasons, you should set this securely.

Private Git repositories using SSH

If you wish to use module source code from a private Git repository using the SSH protocol you can install SSH keys into the runner to allow it to fetch this content.

In your Git provider, create a scoped SSH key that has read-only access to the specific repositories you wish to use. For example, in GitHub, you may do this with an SSH deploy key or other similarly scoped user.

While you can manually mount the private key into the runner, and modify the GIT_SSH_COMMAND environment variable to point to it, the Platform Orchestrator provides a convenience feature in the runner via the RUNNER_GIT_SSH_KEYS environment variable. When you set RUNNER_GIT_SSH_KEYS to a new-line separated list of private SSH keys, the runner will detect these and automatically expose them securely to the OpenTofu/Terraform sub process.

For example, setting RUNNER_GIT_SSH_KEYS to the following will make two SSH keys available to the runner.

As with private registry tokens, you should set this securely.

This mechanism supports RSA, DSA, ECDSA, and Ed25519 private keys in PKCS#1, PKCS#8, OpenSSL, and OpenSSH formats. The keys must be unencrypted without passphrases.

Private Git repositories using HTTP/HTTPS

Private Git modules using an HTTP protocol require a username and password combination accessible by the Git processes within the runner. You can read more about this in the Git documentation . The Platform Orchestrator does not provide any special utilities to configure this.

For example, if you wish to use the token ghp_1234567890abcdefghijklmnopqrstuv for all access to github.com repositories. You can configure the following environment variables in your runner:

GIT_CONFIG_COUNT=1
GIT_CONFIG_KEY_0=url.https://[email protected]/.insteadOf (This one contains a sensitive value, so you should set this securely)
GIT_CONFIG_VALUE_0=https://github.com/

Setting sensitive runner environment variables

For `kubernetes-gke`, `kubernetes-eks`, `kubernetes`, and `kubernetes-agent` runner types

The runner_configuration.job.pod_template field contains a Kubernetes pod template you can set to extend the runtime configuration of the runner. The pod template expects a structure of pod spec with a container named main. You can set secret environment variables by referencing existing secrets within the same target namespace of the runner pod. For example, if you want to mount the value of the key field within a secret named my-secret to the environment variable TF_EXAMPLE, you can set the pod template as the following:

  runner_configuration = {
    job = {
      pod_template = jsonencode({
        spec = {
          containers = [
            {
              name = "main"
              env = [
                {
                  name = "TF_EXAMPLE"
                  valueFrom = {
                    secretKeyRef = {
                      name = "my-secret"
                      key  = "key"
                    }
                  }
                }
              ]
            }
          ]
        }
      })
    }
  }

runner_configuration:
  job:
    pod_template:
      spec:
        containers:
          - name: main
            env:
              name: TF_EXAMPLE
              valueFrom:
                secretKeyRef: 
                  name: my-secret
                  key: key

The service account used by the runner must have permissions to get the secret.

Environment variables that are not secret or sensitive can be set directly in the env structure.

For `serverless-ecs` runners

ECS runners support a runner_configuration.job.secrets field which may contain a mapping from environment variable key to AWS Secrets Manager Secret ARN or AWS Systems Manager Parameter ARN:

  runner_configuration = {
    job = {
      secrets = {
        TF_EXAMPLE   = "arn:aws:secretsmanager:eu-central-1:123456789012:secret:myapp/api-key-XyZ9Qw"
        TF_EXAMPLE_2 = "arn:aws:ssm:eu-central-1:123456789012:parameter/app/config/api-endpoint"
      }
    }
  }

runner_configuration:
  job:
    secrets:
      TF_EXAMPLE: arn:aws:secretsmanager:eu-central-1:123456789012:secret:myapp/api-key-XyZ9Qw
      TF_EXAMPLE_2: arn:aws:ssm:eu-central-1:123456789012:parameter/app/config/api-endpoint

We recommend using secrets manager ARNs for sensitive values.

The job.execution_role_arn of the ECS runner will be used to access these ARNs, therefor you will need to create an appropriate IAM policy to grant the required actions:

[
    {
        "Version": "2012-10-17",
        "Action": [
            "secretsmanager:GetSecretValue"
        ],
        "Resource": "arn:aws:secretsmanager:eu-central-1:123456789012:secret:myapp/*"
    },
    {
        "Version": "2012-10-17",
        "Action": [
            "ssm:GetParameter"
        ],
        "Resource": "arn:aws:ssm:eu-central-1:123456789012:parameter/app/config/api-endpoint"
    }
]

Private module sources

Private OpenTofu/Terraform registries #

Private Git repositories using SSH #

Private Git repositories using HTTP/HTTPS #

Setting sensitive runner environment variables #

For kubernetes-gke, kubernetes-eks, kubernetes, and kubernetes-agent runner types #

For serverless-ecs runners #