Secret references

Overview

Secret references allow developers and platform engineers to work with secrets without necessarily having access to them.

Enterprise organizations with a higher security posture often use a secret store solution for managing secret values. In some cases, this is to help developers and platform engineers who may not have access to the store, but need to incorporate secrets as part of the artifacts they manage. Such as Resource Definitions (where a secret is needed to provision a cloud resource) or Applications (an Application provides a shared secret value to its Workloads).

In other cases, enterprises may not wish to give the Humanitec Platform Orchestrator access to their secret store, not even write-only. This renders developers and platform engineers unable to supply the secret values they need to use via the Platform Orchestrator, even if they know them.

Secret references bridge that gap by allowing you to use an identifier for a secret that resolves the real value during deployment.

Using secret references is currently supported in:

Prerequisites

To get started using secret references, you need to install the Humanitec Operator. The Humanitec Operator is the component that has access to the secret store, and maps the secret references to real values inside your infrastructure.

You also need to connect at least one secret store to the Humanitec Operator and the Platform Orchestrator. See the Humanitec Operator architecture guide for more information on secret stores. To connect a particular secret store, follow the How-to guides for these secret store types:

Workflow

Below are the logical steps to take when using secret references:

Secret maintainers write secrets into a secret store connected to the Humanitec Operator and Platform Orchestrator.
(optional) Developers and/or platform engineers can write secrets into a secret store through Shared Values or Resource Definitions. See the section Writing Secrets via the Platform Orchestrator below for details.

%%{ init: { 'flowchart': { 'curve': 'step' } } }%%
flowchart LR
  subgraph actors[ ]
    secretsMaintainer(("fa:fa-user Secret\nMaintainer"))
    developer(("fa:fa-user Developer"))
    platformEngineer(("fa:fa-user Platform\nEngineer"))
  end
  platformOrchestrator{Platform\nOrchestrator}
  secretsMaintainer -->|fa:fa-lock Add secrets| secretStore
  developer -->|fa:fa-lock Add secrets as\nShared Values in App/Env| platformOrchestrator
  platformEngineer -->|fa:fa-lock Add secrets in\nResource Definitions| platformOrchestrator
  subgraph cloudAccount[Cloud Account]
    secretStore[(fa:fa-lock Secret store )] -.- humanitecOperator
    subgraph kubernetes[Kubernetes]
      humanitecOperator(Humanitec\nOperator)
    end
  end
  platformOrchestrator -->|fa:fa-lock Add secrets| secretStore
  
  classDef subgraph1 fill:#edf0f6,stroke:#cfd1d4,font-family:Akkurat
  classDef subgraph2 fill:#d6d8de,stroke:#cfd1d4,font-family:Akkurat
  class actors,cloudAccount subgraph1
  class kubernetes subgraph2
  linkStyle 0,1,2,3,4 stroke:#bbbbbb

Platform engineers configure Resource Definitions using secret references for any secret values.
Developers configure Shared Values in Applications and/or Application Environments using secret references for any secret values.

%%{ init: { 'flowchart': { 'curve': 'stepBefore' } } }%%
flowchart LR
  subgraph actors[ ]
    secretsMaintainer(("fa:fa-user Secret\nMaintainer"))
    developer(("fa:fa-user Developer"))
    platformEngineer(("fa:fa-user Platform\nEngineer"))
  end
  platformOrchestrator{Platform\nOrchestrator}
  secretsMaintainer
  developer -->|Configure Shared Values in App/Env\nusing Secret references| platformOrchestrator
  platformEngineer -->|Configure Resource Definitions\nusing Secret references| platformOrchestrator
  subgraph cloudAccount[Cloud Account]
    secretStore[(fa:fa-lock Secret store )] -.- humanitecOperator
    subgraph kubernetes[Kubernetes]
      humanitecOperator(Humanitec\nOperator)
    end
  end
  platformOrchestrator -.- secretStore

  classDef subgraph1 fill:#edf0f6,stroke:#cfd1d4,font-family:Akkurat
  classDef subgraph2 fill:#d6d8de,stroke:#cfd1d4,font-family:Akkurat
  class actors,cloudAccount subgraph1
  class kubernetes subgraph2
  linkStyle 0,1,2,3 stroke:#bbbbbb

Developers deploy Applications using Score. Score files do not change with the use of secret references.
The Platform Orchestrator creates custom resources (CRs) in the target Kubernetes cluster. Please refer to the Humanitec Operator architecture guide for pull-based alternatives.

%%{ init: { 'flowchart': { 'curve': 'stepBefore' } } }%%
flowchart LR
  subgraph actors[ ]
    secretsMaintainer(("fa:fa-user Secret\nMaintainer"))
    developer(("fa:fa-user Developer"))
    platformEngineer(("fa:fa-user Platform\nEngineer"))
  end
  platformOrchestrator{Platform\nOrchestrator}
  developer -->|Deploy Applications\nusing Score| platformOrchestrator
  subgraph cloudAccount[Cloud Account]
    secretStore[(fa:fa-lock Secret store )] -.- humanitecOperator
    subgraph kubernetes[Kubernetes]
      humanitecOperator(Humanitec\nOperator)
      crs[[CRs]] -.-> humanitecOperator
    end
  end
  platformOrchestrator -->|Deploy CRs\ninto cluster| crs

  classDef subgraph1 fill:#edf0f6,stroke:#cfd1d4,font-family:Akkurat
  classDef subgraph2 fill:#d6d8de,stroke:#cfd1d4,font-family:Akkurat
  class actors,cloudAccount subgraph1
  class kubernetes subgraph2

  linkStyle 0,1,2,3 stroke:#bbbbbb

The Humanitec Operator reads secrets from the secret store according to the secret references in the CRs as needed for Workload or resource provisioning.
The Humanitec Operator creates the required K8s manifests including Secrets according to the values it reads from the references in the CRs. It also calls any Drivers needed to provision Resources.

%%{ init: { 'flowchart': { 'curve': 'stepBefore' } } }%%
flowchart LR
  subgraph cloudAccount[Cloud Account]
    secretStore[(fa:fa-lock Secret store )] -->|Secrets| humanitecOperator
    subgraph kubernetes[Kubernetes]
      crs[[CRs]] -.->|Secret\nreferences| humanitecOperator
      humanitecOperator(Humanitec\nOperator) -->|fa:fa-lock Create\nmanifests| manifests[[fa:fa-lock Manifests]]
    end
    subgraph resources[Resources]
      direction TB
      infra1[/ /]
      infra2( )
      infra1 ~~~ infra2
    end
  end
  humanitecOperator -->|fa:fa-lock Call drivers| drivers(Drivers)
  drivers -->|fa:fa-lock Provision\nresources| resources

  classDef subgraph1 fill:#edf0f6,stroke:#cfd1d4,font-family:Akkurat
  classDef subgraph2 fill:#d6d8de,stroke:#cfd1d4,font-family:Akkurat
  class cloudAccount subgraph1
  class kubernetes,resources subgraph2

  linkStyle 0,1,2,3,4,5 stroke:#bbbbbb

The following examples illustrate how secret references are used in practice.

Examples: Resource Definitions

See the API documentation for all details about secret references in Resource Definitions.

Amazon S3 bucket

This Resource Definition describes an Amazon S3 bucket using the Terraform Driver. It references secrets in a secret store with the ID dev-vault to use them as driver inputs.

{
  "id": "s3-bucket-dev",
  "name": "s3-bucket-dev",
  "type": "s3",
  "driver_type": "humanitec/terraform",
  "driver_inputs": {
    "values": {
      "source": {
        "path": "test/terraform/s3",
        "rev": "main",
        "url": "https://github.com/my-org/terraform-s3.git"
      },
      "variables": {
        "REGION": "eu-central-1"
      }
    },
    "secret_refs": {
      "variables": {
        "ACCESS_KEY_ID": {
          "store": "dev-vault",
          "ref": "dev/aws/credentials/.access_key_id"
        },
        "SECRET_ACCESS_KEY": {
          "store": "dev-vault",
          "ref": "dev/aws/credentials/.secret_access_key"
        }
      }
    }
  }
}

Details to note are:

The driver_inputs / values section contains the non-secret inputs for the Driver.
The driver_inputs / secret_refs section contains the secret reference inputs for the Driver.
The variables section is specific to the Terraform Driver. It defines two variables ACCESS_KEY_ID and SECRET_ACCESS_KEY which the Driver needs as input to the Terraform code found at the source location above.
Each variable contains a store and ref attribute.
store corresponds to the ID of a secret store registered to the Humanitec Operator. The example uses HashiCorp Vault (see how to connect it) as the secret store type.
ref corresponds to the fully qualified name of the secret inside that secret store. For Vault, this must include the JSON path to the secret content, e.g. .access_key_id.
The secret references could optionally specify a version, which they do not, so the most recent version of each secret will be retrieved.

AWS Route 53 DNS Record

This Resource Definition describes a DNS Record in an AWS Route 53 service using the Humanitec built-in Driver. It references secrets in a secret store with the ID prod-awssm to use them as driver inputs.

{
  "id": "my-dns-prod",
  "name": "my-dns-prod",
  "type": "dns",
  "driver_type": "humanitec/dns-aws-route53",
  "driver_inputs": {
    "values": {
      "hosted_zone_id": "ABSDEFGH1234",
      "domain": "my-app.example.com",
      "template": "${context.app.id}-${context.env.id}"
    },
    "secret_refs": {
      "account": {
        "aws_access_key_id": {
            "store": "prod-awssm",
            "ref": "prod/aws/access_key_id"
        },
        "aws_secret_access_key":  {
          "store": "prod-awssm",
          "ref": "prod/aws/secret_access_key",
          "version": "2"
        }
      }
    }
  }
}

The setup is similar to the previous example. Note these differences:

The store is the ID of an AWS Secret store registered to the Humanitec Operator.
ref once more corresponds to the fully qualified name of the secret inside that secret store.
The secret reference aws_secret_access_key targets the specific version 2 of the secret.

Examples: Shared Values for Applications and Environments

See the API documentation for details about Shared Values in Applications and Environments.

Application Shared Value

This command adds a secret Shared Value named token to an Application named my-app. It references a secret store to obtain the value.

humctl api post /orgs/my-org/apps/my-app/values \
  -d '{
  "key": "token",
  "description": "my API token",
  "is_secret": true,
  "secret_ref": {
    "store": "dev-vault",
    "ref": "dev/api/.token"
  }
}'

curl https://api.humanitec.io/orgs/my-org/apps/my-app/values \
  -X POST \
  -H "Authorization: Bearer ${HUMANITEC_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
  "key": "token",
  "description": "my API token",
  "is_secret": true,
  "secret_ref": {
    "store": "dev-vault",
    "ref": "dev/api/.token"
  }
}'

resource "humanitec_value" "my-secret-ref-value" {
  app_id      = "my-app"
  description = "my API token"
  key         = "token"
  is_secret   = true

  secret_ref = {
    ref   = "dev/api/.token"
    store = "dev-vault"
  }
}

Details to note are:

The target URL " .../my-app/values" addresses the Shared Values of the Application my-app.
The secret_ref section contains the data for the secret reference.
store corresponds to the ID of a secret store registered to the Humanitec Operator. The example uses HashiCorp Vault (see how to connect it) as the secret store type.
ref corresponds to the fully qualified name of the secret inside that secret store. For Vault, this must include the JSON path to the secret content, e.g. .token.

Environment Shared Value

This command adds a secret Shared Value override in the development Environment for the Application level Share Value from the previous example. It references a secret store to obtain the value.

humctl api post /orgs/my-org/apps/my-app/envs/development/values \
  -d '{
  "key": "token",
  "description": "my API token",
  "is_secret": true,
  "secret_ref": {
    "store": "dev-vault",
    "ref": "dev/api/.token",
    "version": "2"
  }
}'

curl https://api.humanitec.io/orgs/my-org/apps/my-app/envs/development/values \
  -X POST \
  -H "Authorization: Bearer ${HUMANITEC_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
  "key": "token",
  "description": "my API token",
  "is_secret": true,
  "secret_ref": {
    "store": "dev-vault",
    "ref": "dev/api/.token",
    "version": "2"
  }
}'

Details to note are:

The message payload follows the same structure as for the previous Application level example.
The URL now targets the development environment underneath the my-app Application instead of the Application itself.
The secret reference requests a specific version of the secret.

Writing secrets via the Platform Orchestrator

You may optionally choose to allow users of the Platform Orchestrator to write secrets into the primary secret store via the Orchestrator. See the Humanitec Operator architecture guide for details.

When doing so, provide the secret value inside the secret reference definitions instead of a reference (store + ref).

Here’s an example of writing a secret to a secret store through an Application Shared Value:

humctl api post /orgs/my-org/apps/my-app/values \
  -d '{
  "key": "token",
  "description": "my API token",
  "is_secret": true,
  "secret_ref": {
    "value": "my-token-value"
  }
}'

curl https://api.humanitec.io/orgs/my-org/apps/my-app/values \
  -X POST \
  -H "Authorization: Bearer ${HUMANITEC_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
  "key": "token",
  "description": "my API token",
  "is_secret": true,
  "secret_ref": {
    "value": "my-token-value"
  }
}'

For Shared Values (see the API docs), the value property for writing secrets is deprecated. Use secret_ref / value instead as shown above.

Here’s an example of writing secrets to a secret store through a Resource Definition:

Create a file defining the Resource Definition you want to create:

cat << EOF > static-mysql-writesecret.yaml
apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: static-mysql-writesecret
entity:
  driver_type: humanitec/echo
  name: Static MySQL Write Secret
  type: mysql
  driver_inputs:
    secret_refs:
      password:
        value: my-super-secret-password
      username:
        value: my-username
    values:
      host: mysql.example.com
      name: db_name
      port: 3306
EOF

Use the humctl create command to create the Resource Definition in the Organization defined by your configured context:

humctl create -f static-mysql-writesecret.yaml
rm static-mysql-writesecret.yaml

curl https://api.humanitec.io/orgs/${HUMANITEC_ORG}/resources/defs \
  -H "Authorization: Bearer ${HUMANITEC_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
  "id": "static-mysql-writesecret",
  "name": "Static MySQL Write Secret",
  "type": "mysql",
  "driver_type": "humanitec/echo",
  "driver_inputs": {
    "values": {
      "name": "db_name",
      "host": "mysql.example.com",
      "port": 3306
    },
    "secret_refs": {
      "username": {
        "value": "my-username"
      },
      "password": {
        "value": "my-super-secret-password"
      }
    }
  }
}'

For Resource Definitions (see the API docs), the driver_inputs / secrets property for writing secrets is deprecated. Use driver_inputs / secret_refs instead as shown above.

Details to note when writing secrets are:

The store cannot be specified. The secret is always written to the primary store.
Also, no version can be specified. For subsequent updates using the same mechanism, use PATCH instead of POST, and provide the new value. This will create a new secret version in the secret store.

Secret ref format for secret store types

The format for the ref value of a secret reference varies based on the secret store type as follows:

Secret store type	`ref` format	Example
Azure Key Vault	`secretname`	`my-secret`
AWS Secrets Manager	`secretname`	`my-secret`
Google Secret Manager	`secretname`	`my-secret`
HashiCorp Vault	`path/below/secret/engine/.jsonpath`	`orgs/my-org/apps/my-app/secret_values/my-secret/.value`

Secret versions

Specifying a version in a secret reference is optional. If not specified, the most recent version is used.

Mixing secret stores

It’s possible to reference different secret stores, even of different types, within the same artifact such as a Resource Definition. Each secret reference independently specifies the secret store to use via the store’s ID.

This snippet from a Resource Definition describes two secrets coming from separate secret stores identified by their respective store ID:

    "secret_refs": {
      "username": {
        "store": "my-gsm",
        "ref": "development-username-mysql"
      },
      "password": {
        "store": "my-vault",
        "ref": "development/mysql/password/.value"
      }
    }

Secret references and Score

Since the use of secret references is an implementation detail taking place on the deployment side, it has no impact on the Workload specification expressed through Score.

Secret references for Driver cloud accounts

Some built-in Humanitec Drivers require cloud account credentials to provision their target resources, e.g. the AWS Route53 or the Google Cloud database Drivers such as PostgreSQL Cloud SQL.

You can maintain the required credentials in a secret store and then define a secret reference named account in the Resource Definition using one such Driver. E.g. for a Resource Definition using the PostgreSQL Cloud SQL Driver and credentials stored in a Google Cloud Secret Manager secret store, define the secret reference like this:

apiVersion: entity.humanitec.io/v1b1
kind: Definition
...
entity:
  ...
  driver_type: humanitec/postgres-cloudsql
  driver_inputs:
    secret_refs:
      account:
        store: my-gsm
        ref: cloudsql-account
      ...

When storing cloud account credentials in Google Cloud Secret Manager, convert a JSON representation to a String:

echo -n "$(cat credentials.json | jq -r tostring)" | \
gcloud secrets create cloudsql-account \
        --project my-gcp-project \
        --data-file=-

Limitations

Using placeholders is currently not possible in secret references.

Secret references

Overview #

Prerequisites #

Workflow #

Examples: Resource Definitions #

Amazon S3 bucket #

AWS Route 53 DNS Record #

Examples: Shared Values for Applications and Environments #

Application Shared Value #

Environment Shared Value #

Writing secrets via the Platform Orchestrator #

Secret ref format for secret store types #

Secret versions #

Mixing secret stores #

Secret references and Score #

Secret references for Driver cloud accounts #

Limitations #

Overview

Prerequisites

Workflow

Examples: Resource Definitions

Amazon S3 bucket

AWS Route 53 DNS Record

Examples: Shared Values for Applications and Environments

Application Shared Value

Environment Shared Value

Writing secrets via the Platform Orchestrator

Secret ref format for secret store types

Secret versions

Mixing secret stores

Secret references and Score

Secret references for Driver cloud accounts

Limitations