Connect to Azure Key Vault

This article describes how to connect an instance of Azure Key Vault as a secret store to your Humanitec Operator setup.

Before you begin

Before you begin, make sure you have the following resources and permissions:

You have an Azure Key Vault and permissions on it to create secrets and role assignments.
You have determined the required access level (read/write/no access) to the secret store for the Platform Orchestrator and the Humanitec Operator, respectively. See Architecture for guidance.
You have installed and configured the Humanitec Operator on an Azure Kubernetes Service (AKS) cluster.
You have access to the cluster via kubectl.
You have the az CLI installed and have authenticated against Azure.
You have reviewed the Implications of switching to Operator mode and prepared accordingly. Only relevant if this is the first secret store to be connected to your Organization.
(Recommended) Workload Identity is enabled on your AKS cluster.

Prepare your environment

Set the name and scope of your Key Vault instance:

export KEY_VAULT_NAME=<my-keyvault-name>
export KEY_VAULT_SCOPE=$(az keyvault show --name $KEY_VAULT_NAME --query id -o tsv)

Set the ID of the Key Vault’s Azure tenant:

export TENANT_ID=$(az keyvault show --name $KEY_VAULT_NAME --query properties.tenantId -o tsv)

Define an ID for your secret store. This ID will later match the secret store registrations in the Operator and in the Platform Orchestrator.

export SECRET_STORE_ID=my-akv

The following variables are only required when using workload identity. RESOURCE_GROUP_NAME and REGION will be used for the creation of the managed identity resource.

Define the name of your AKS cluster:

export AKS_NAME=<my-aks-cluster-name>

Define the Resource Group where to create the managed identity resource. It can be the one of your AKS cluster resource:

export RESOURCE_GROUP_NAME=<my-rg-name>

Define the region for the managed identity resource:

export REGION=<my-region>

Enable Workload Identity for the Humanitec Operator

Using Workload Identity (this section) is the recommended option to authenticate the Humanitec Operator. It frees you from the need to manage any service principal access credentials. If you do not wish to use Workload Identity, skip to the next section.

Workload Identity allows workloads in your AKS clusters to impersonate Microsoft Entra identities to access Microsoft Entra protected resources.

To grant the Operator access to the Key Vault, create a managed identity and then enable Workload Identity for the Humanitec Operator.

Prepare environment variables for the managed identity:

export HUMANITEC_OPERATOR_MI_NAME=humanitec-operator-identity
export FEDERATED_IDENTITY_CREDENTIAL_NAME=humanitec-operator-identity
export HUMANITEC_OPERATOR_NAMESPACE=humanitec-operator-system
export HUMANITEC_OPERATOR_SERVICE_ACCOUNT_NAME=humanitec-operator-controller-manager

Adjust the HUMANITEC_OPERATOR_NAMESPACE variable if you installed the Operator into a different namespace.

Get the cluster OIDC issuer:

AKS_OIDC_ISSUER="$(az aks show \
  -n ${AKS_NAME} \
  -g ${RESOURCE_GROUP_NAME} \
  --query "oidcIssuerProfile.issuerUrl" \
  -otsv)"

Create a managed identity:

az identity create \
  --name ${HUMANITEC_OPERATOR_MI_NAME} \
  --resource-group ${RESOURCE_GROUP_NAME} \
  --location ${REGION}

export HUMANITEC_OPERATOR_MI_CLIENT_ID="$(az identity show -g $RESOURCE_GROUP_NAME --name $HUMANITEC_OPERATOR_MI_NAME --query 'clientId' -o tsv)"

Create a federated identity for this managed identity:

az identity federated-credential create \
  --name ${FEDERATED_IDENTITY_CREDENTIAL_NAME} \
  --identity-name ${HUMANITEC_OPERATOR_MI_NAME} \
  --resource-group ${RESOURCE_GROUP_NAME} \
  --issuer "${AKS_OIDC_ISSUER}" \
  --subject system:serviceaccount:${HUMANITEC_OPERATOR_NAMESPACE}:${HUMANITEC_OPERATOR_SERVICE_ACCOUNT_NAME} \
  --audience api://AzureADTokenExchange

Upgrade the Helm installation of the Humanitec Operator to set the Annotation azure.workload.identity/client-id to the Kubernetes service account it uses and the Label azure.workload.identity/use on its Pod:

helm upgrade \
  humanitec-operator \
  oci://ghcr.io/humanitec/charts/humanitec-operator \
  -n humanitec-operator-system \
  --set "controllerManager.serviceAccount.annotations.azure\.workload\.identity/client-id=${HUMANITEC_OPERATOR_MI_CLIENT_ID}" \
  --set "controllerManager.podLabels.azure\.workload\.identity/use=true"

(Add the --version <version> parameter to pin the Helm chart to a particular version.)

If you are using an IaC tool to provision the Helm chart, make sure to add the new Helm value to the configuration.

Prepare service principal credentials for the Humanitec Operator

Using Workload Identity instead of a self-managed service principal is the recommended option to authenticate the Humanitec Operator. It frees you from the need to manage any service principal access credentials. To use Workload Identity, preform the steps in the previous section and skip this one.

To grant the Operator access to the Key Vault, create a service principal, create credentials for this service principal, and then place those credentials in a Kubernetes Secret. This Secret will then be referenced in the secret store definition.

Prepare environment variables for the service principal:

export KEY_VAULT_SP_NAME=humanitec-operator-sp-$KEY_VAULT_NAME

Create a service principal and capture its credentials:

export KEY_VAULT_SP_CREDENTIALS=$(az ad sp create-for-rbac \
       -n ${KEY_VAULT_SP_NAME})
export KEY_VAULT_SP_ID=$(echo ${KEY_VAULT_SP_CREDENTIALS} | jq -r .appId)
export KEY_VAULT_SP_PASSWORD=$(echo ${KEY_VAULT_SP_CREDENTIALS} | jq -r .password)

Create a Kubernetes Secret containing the service principal credentials:

export SECRET_NAME=humanitec-operator-sp-$KEY_VAULT_NAME

kubectl create secret generic ${SECRET_NAME} \
  -n humanitec-operator-system \
  --from-literal=ClientID=${KEY_VAULT_SP_ID} \
  --from-literal=ClientSecret=${KEY_VAULT_SP_PASSWORD}

Configure Key Vault access (Azure RBAC)

If you are using Azure RBAC for managing access to your Key Vault (recommended), perform the steps in this section. Otherwise, skip to the next section.

Define the required target role.

For read access (the Key Vault is not your default secret store):

export KEY_VAULT_ROLE="Key Vault Secrets User"

For read/write access (the Key Vault is your default secret store):

export KEY_VAULT_ROLE="Key Vault Secrets Officer"

Create the role assignment:

az role assignment create \
  --role "$KEY_VAULT_ROLE" \
  --assignee $HUMANITEC_OPERATOR_MI_CLIENT_ID \
  --scope $KEY_VAULT_SCOPE

az role assignment create \
  --role "$KEY_VAULT_ROLE" \
  --assignee $KEY_VAULT_SP_ID \
  --scope $KEY_VAULT_SCOPE

Configure Key Vault access (access policy)

If you are using the legacy access policy for managing access to your Key Vault, perform the steps in this section. Otherwise, skip to the next section.

Define the required permissions.

For read access (the Key Vault is not your default secret store):

export KEY_VAULT_SECRET_PERMISSIONS="get"

For read/write access (the Key Vault is your default secret store):

export KEY_VAULT_SECRET_PERMISSIONS="get set delete"

Set an access policy on your Key Vault. Choose the variant that matches the selected authentication method for the Humanitec Operator.

az keyvault set-policy \
  --name ${KEY_VAULT_NAME} \
  --secret-permissions ${KEY_VAULT_SECRET_PERMISSIONS} \
  --spn ${HUMANITEC_OPERATOR_MI_CLIENT_ID}

az keyvault set-policy \
  --name ${KEY_VAULT_NAME} \
  --secret-permissions ${KEY_VAULT_SECRET_PERMISSIONS} \
  --spn ${KEY_VAULT_SP_ID}

Register the secret store with the Operator

Create the secret store registration using the following command. Modify it according to your setup:

If this is not your default secret store, omit the label app.humanitec.io/default-store (What is the default secret store?).
Choose the variant that matches the selected authentication method for the Humanitec Operator.

kubectl apply -f - << EOF
apiVersion: humanitec.io/v1alpha1
kind: SecretStore
metadata:
  name: ${SECRET_STORE_ID}
  namespace: humanitec-operator-system
  labels:
    app.humanitec.io/default-store: "true"
spec:
  azurekv:
    url: https://${KEY_VAULT_NAME}.vault.azure.net/
    tenantID: ${TENANT_ID}
    auth: {}
EOF

kubectl apply -f - << EOF
apiVersion: humanitec.io/v1alpha1
kind: SecretStore
metadata:
  name: ${SECRET_STORE_ID}
  namespace: humanitec-operator-system
  labels:
    app.humanitec.io/default-store: "true"
spec:
  azurekv:
    url: https://${KEY_VAULT_NAME}.vault.azure.net/
    tenantID: ${TENANT_ID}
    auth:
     clientIDSecretRef:
       name: ${SECRET_NAME}
       key: ClientID
     clientSecretSecretRef:
       name: ${SECRET_NAME}
       key: ClientSecret
EOF

Confirm the secret store registration.

To confirm the secret store registration, and anytime you wish to check for registered secret stores, use the following command:

kubectl get secretstores -n humanitec-operator-system

Register the secret store with the Platform Orchestrator

The Platform Orchestrator automatically switches to “Humanitec Operator” mode when configured with an external secret store marked primary. See the Architecture guidance for the details and implications of that switch.

The Platform Orchestrator needs to know which secret store to reference in the custom resources it creates for the Operator. This step is therefore always required. To begin using your secret store, you must now register it with the Platform Orchestrator.

The ID of the secret store in the Platform Orchestrator must match the ID of the SecretStore object created in the cluster.

(Optional) Prepare write access for the Platform Orchestrator.

Skip this step if you do not wish to grant write access to your Key Vault via the Platform Orchestrator.

Create a service principal and capture its credentials:

export ORCHESTRATOR_SP_NAME=platform-orchestrator-sp-$KEY_VAULT_NAME
export ORCHESTRATOR_SP_CREDENTIALS=$(az ad sp create-for-rbac \
       -n ${ORCHESTRATOR_SP_NAME})
export ORCHESTRATOR_SP_ID=$(echo ${ORCHESTRATOR_SP_CREDENTIALS} | jq -r .appId)
export ORCHESTRATOR_SP_PASSWORD=$(echo ${ORCHESTRATOR_SP_CREDENTIALS} | jq -r .password)

If you are using Azure RBAC for managing access to your Key Vault, create a role assignment:

Option 1: Using a built-in role:
```
az role assignment create --role "Key Vault Secrets Officer" \
  --assignee ${ORCHESTRATOR_SP_ID} \
  --scope ${KEY_VAULT_SCOPE}
```
The Key Vault Secrets Officer role has read access to secrets as well, which is not required but it is the least privileged of all built-in roles with secrets write access. Consider creating a custom role for true least privilege, as shown next.

Option 2: Creating a least privilege Azure custom role with the required permissions only:

export KEY_VAULT_SUBSCRIPTION=$(az keyvault show -n ${KEY_VAULT_NAME} \
  --query "id" | \
  sed -n 's|^"/[^/]*/\([^/]*\)/[^/]*/.*$|\1|p')

az role definition create --role-definition '{
  "Name": "Key Vault Secrets Writer",
  "Description": "Write secrets and secret values into an Azure Key Vault.",
  "Actions": [
      "Microsoft.KeyVault/vaults/secrets/write"
  ],
  "DataActions": [
      "Microsoft.KeyVault/vaults/secrets/delete",
      "Microsoft.KeyVault/vaults/secrets/setSecret/action",
      "Microsoft.KeyVault/vaults/secrets/update/action"
  ],
  "AssignableScopes": [
    "/subscriptions/'${KEY_VAULT_SUBSCRIPTION}'"
  ]
}'

The example sets the AssignableScopes to the Key Vault’s parent Subscription. You may alter the scope depending on your mandate and the desired reach of the role.

Assign the role to the Platform Orchestrator service principal:

az role assignment create --role "Key Vault Secrets Writer" \
  --assignee ${ORCHESTRATOR_SP_ID} \
  --scope ${KEY_VAULT_SCOPE}

If you are using access policy for managing access to your Key Vault, set an access policy:

az keyvault set-policy \
  --name ${KEY_VAULT_NAME} \
  --secret-permissions set delete \
  --spn ${ORCHESTRATOR_SP_ID}

Set the ID of your Humanitec Organization (must be all lowercase):

export HUMANITEC_ORG=my-humanitec-org-id

Set a Humanitec API token:

export HUMANITEC_TOKEN=my-humanitec-api-token

Run this command to register the secret store. Modify it according to your setup:

Omit the "auth" part if you are not granting write access via the Platform Orchestrator.
Set "primary" to false if this is not your primary secret store (What is the primary secret store?).

humctl api post /orgs/${HUMANITEC_ORG}/secretstores \
  -d '{
  "id": "'"${SECRET_STORE_ID}"'",
  "primary": true,
  "azurekv": {
    "tenant_id": "'"${TENANT_ID}"'",
    "url": "https://'"${KEY_VAULT_NAME}"'.vault.azure.net/",
    "auth": {
      "client_id": "'"${ORCHESTRATOR_SP_ID}"'",
      "client_secret": "'"${ORCHESTRATOR_SP_PASSWORD}"'"
    }
  }
}'

curl https://api.humanitec.io/orgs/${HUMANITEC_ORG}/secretstores \
  -X POST \
  -H "Authorization: Bearer $HUMANITEC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "id": "'"${SECRET_STORE_ID}"'",
  "primary": true,
  "azurekv": {
    "tenant_id": "'"${TENANT_ID}"'",
    "url": "https://'"${KEY_VAULT_NAME}"'.vault.azure.net/",
    "auth": {
      "client_id": "'"${ORCHESTRATOR_SP_ID}"'",
      "client_secret": "'"${ORCHESTRATOR_SP_PASSWORD}"'"
    }
  }
}'

resource "humanitec_secretstore" "my_akv" {
  id      = "my-akv"
  primary = true
  azurekv = {
    url       = var.azure_keyvault_url
    tenant_id = var.azure_tenant_id
    auth = {
      client_id     = var.azure_client_id
      client_secret = var.azure_client_secret
    }
  }
}

If, at a later stage, you need to update the secret for an already registered secret store, the following command can be used.

humctl api patch /orgs/${HUMANITEC_ORG}/secretstores/${SECRET_STORE_ID} \
  -d '{
  "azurekv": {
    "auth": {
      "client_id": "'"${ORCHESTRATOR_SP_ID}"'",
      "client_secret": "'"${ORCHESTRATOR_SP_PASSWORD}"'"
    }
  }
}'

curl https://api.humanitec.io/orgs/${HUMANITEC_ORG}/secretstores/${SECRET_STORE_ID} \
  -X PATCH \
  -H "Authorization: Bearer $HUMANITEC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "azurekv": {
    "auth": {
      "client_id": "'"${ORCHESTRATOR_SP_ID}"'",
      "client_secret": "'"${ORCHESTRATOR_SP_PASSWORD}"'"
    }
  }
}'

Perform another terraform apply using the definition shown above, and current variable values.

Make sure all referenced environment variables are set according to the previous steps.

To disable access for the Platform Orchestrator, use the same command and set "client_secret": "xxx" in the "auth" section.

Confirm the secret store registration.

To confirm the secret store registration, and anytime you wish to check for registered secret stores, use the following command:

humctl api get /orgs/${HUMANITEC_ORG}/secretstores

curl -s https://api.humanitec.io/orgs/${HUMANITEC_ORG}/secretstores \
  -H "Authorization: Bearer $HUMANITEC_TOKEN" \
  | jq

Resource cookies

Resource cookies are stored in the Key Vault according to this pattern:

resources--active--<resource-id>--cookies--<driver>

Next steps

Test the Humanitec Operator installation using these test cases.
Perform the Update Resource Definitions for related Applications tutorial to verify your setup.
- Ensure the sample Applications are being deployed to your Orchestrator-enabled cluster by adjusting the matching criteria for the cluster Resource.
- Observe how custom resources of type workload and resource are being created on the cluster, and check their status sections.
- Observe how the Operator writes operational state (“resource cookies”) into your default secret store.

Connect to Azure Key Vault

Before you begin #

Prepare your environment #

Enable Workload Identity for the Humanitec Operator #

Prepare service principal credentials for the Humanitec Operator #

Configure Key Vault access (Azure RBAC) #

Configure Key Vault access (access policy) #

Register the secret store with the Operator #

Register the secret store with the Platform Orchestrator #

Resource cookies #

Next steps #