Resource Graph Patterns

Pattern name

Resource Type

Break A Loop Additional Resource

Break a loop with an additional resource

This example demonstrates how to break a loop where two resources have to both depend on each others outputs.

How the example works

There is often a mutual loop assigning a principal to a Kubernetes service account to enable Workload Identity. The Kubernetes service account often needs to be annotated with the principal and the principal needs some policy to allow it to be used by the Kubernetes Service Account.

In this example, we will simulate the graph using a k8s-service-account and a fake aws-role resource both implemented with the humanitec/template driver.

graph LR
    workload --> k8s-service-account
    k8s-service-account --> aws-role
    aws-role --> k8s-service-account

The loop arises because it is necessary to generate both the Kubernetes service account name and the aws-role dynamically. This is because these need to be unique for each Workload in the Application. Essentially, both resources require the same two pieces of information.

There are two ways to break the loop:

  1. Convention

    Decide that each resource “knows” how to generate both pieces of information. This can be achieved by using the context to provide the unique element.

    This has the downside that it is inflexible and limiting. For example, if a 3rd party system is used to issue principals, then this technique will not work.

  2. Add an additional resource

    Both the k8s-service-account and aws-role resources get both the service account name and role ID from a 3rd resource.

    This approach ensures consistency, does not rely on convention and allows for complex scenarios like getting IDs from a 3rd party system.

graph LR
    workload --> k8s-service-account
    k8s-service-account --> aws-role
    k8s-service-account --> config
    aws-role --> config

score.yaml (view on GitHub) :

apiVersion: score.dev/v1b1

metadata:
  name: example-workload

containers:
  busybox:
    image: busybox:latest

    variables:
      BUCKET_NAME: ${resources.my-s3.bucket}

    command:
      - /bin/sh
    args:
      - "-c"
      # This will output all of the environment variables in the container to
      # STDOUT every 15 seconds. This can be seen in the container logs in the
      # Humanitec UI.
      - "while true; do set; sleep 15; done"

resources:
  my-s3:
    type: s3

Resource Definitions

def-aws-role.yaml (view on GitHub) :

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: example-aws-role
entity:
  criteria:
    - app_id: example-break-a-loop-additional-resource
  driver_inputs:
    values:
      role_arn: ${resources['config.sa-name-role-id'].outputs.role_arn}
      sa_name: ${resources['config.sa-name-role-id'].outputs.sa_name}
      templates:
        outputs: |
          arn: {{ .drivers.role_arn }}
  driver_type: humanitec/template
  name: example-aws-role
  type: aws-role

def-config-name-id.yaml (view on GitHub) :

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: example-config-name-id
entity:
  criteria:
    # We only match to class and not res_id because the res_id changes for
    # each workload
    - class: sa-name-role-id
      app_id: example-break-a-loop-additional-resource
  driver_inputs:
    values:
      res_id: ${context.res.id}
      app_id: ${context.app.id}
      templates:
        init: |
          workload_id: {{ .driver.values.res_id | splitList "." | last }}
        outputs: |
          role_arn: "arn:aws:iam::123456789012:role/{{ .driver.values.app_id }}/sa-role-{{ .init.workload_id }}"
          sa_name: {{ .init.workload_id }}-sa
  driver_type: humanitec/template
  name: example-config-name-id
  type: config

def-k8s-service-account.yaml (view on GitHub) :

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: example-k8s-service-account
entity:
  criteria:
    - app_id: example-break-a-loop-additional-resource
  driver_inputs:
    values:
      role_arn: ${resources['config.sa-name-role-id'].outputs.role_arn}
      sa_name: ${resources['config.sa-name-role-id'].outputs.sa_name}
      templates:
        outputs: |
          name: {{ .driver.values.sa_name }}
        manifests: |
          service-account.yaml:
            location: namespace
            data: |
              apiVersion: v1
              kind: ServiceAccount
              metadata:
                name: {{ .driver.values.sa_name }}
                annotations:
                  eks.amazonaws.com/role-arn: {{ .driver.values.role_arn }}

  driver_type: humanitec/template
  name: example-k8s-service-account
  type: k8s-service-account

def-workload.yaml (view on GitHub) :

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: example-workload
entity:
  criteria:
    - app_id: example-break-a-loop-additional-resource
  driver_inputs:
    values:
      templates:
        outputs: |
          update: 
          - op: add
            path: /spec/serviceAccountName
            {{/*
              The resource reference does not specify ID or class so the ID and
              class of the workload being provisioned will be used.
            *//}
            value: ${resources.k8s-service-account.outputs.name}
  driver_type: humanitec/template
  name: example-workload
  type: workload

def-aws-role.tf (view on GitHub) :

resource "humanitec_resource_definition" "example-aws-role" {
  driver_type = "humanitec/template"
  id          = "example-aws-role"
  name        = "example-aws-role"
  type        = "aws-role"
  driver_inputs = {
    values_string = jsonencode({
      "role_arn" = "$${resources['config.sa-name-role-id'].outputs.role_arn}"
      "sa_name"  = "$${resources['config.sa-name-role-id'].outputs.sa_name}"
      "templates" = {
        "outputs" = "arn: {{ .drivers.role_arn }}\n"
      }
    })
  }
}

resource "humanitec_resource_definition_criteria" "example-aws-role_criteria_0" {
  resource_definition_id = resource.humanitec_resource_definition.example-aws-role.id
  app_id                 = "example-break-a-loop-additional-resource"
}

def-config-name-id.tf (view on GitHub) :

resource "humanitec_resource_definition" "example-config-name-id" {
  driver_type = "humanitec/template"
  id          = "example-config-name-id"
  name        = "example-config-name-id"
  type        = "config"
  driver_inputs = {
    values_string = jsonencode({
      "res_id" = "$${context.res.id}"
      "app_id" = "$${context.app.id}"
      "templates" = {
        "init"    = "workload_id: {{ .driver.values.res_id | splitList \".\" | last }}\n"
        "outputs" = <<END_OF_TEXT
role_arn: "arn:aws:iam::123456789012:role/{{ .driver.values.app_id }}/sa-role-{{ .init.workload_id }}"
sa_name: {{ .init.workload_id }}-sa
END_OF_TEXT
      }
    })
  }
}

resource "humanitec_resource_definition_criteria" "example-config-name-id_criteria_0" {
  resource_definition_id = resource.humanitec_resource_definition.example-config-name-id.id
  class                  = "sa-name-role-id"
  app_id                 = "example-break-a-loop-additional-resource"
}

def-k8s-service-account.tf (view on GitHub) :

resource "humanitec_resource_definition" "example-k8s-service-account" {
  driver_type = "humanitec/template"
  id          = "example-k8s-service-account"
  name        = "example-k8s-service-account"
  type        = "k8s-service-account"
  driver_inputs = {
    values_string = jsonencode({
      "role_arn" = "$${resources['config.sa-name-role-id'].outputs.role_arn}"
      "sa_name"  = "$${resources['config.sa-name-role-id'].outputs.sa_name}"
      "templates" = {
        "outputs"   = "name: {{ .driver.values.sa_name }}\n"
        "manifests" = <<END_OF_TEXT
service-account.yaml:
  location: namespace
  data: |
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: {{ .driver.values.sa_name }}
      annotations:
        eks.amazonaws.com/role-arn: {{ .driver.values.role_arn }}
END_OF_TEXT
      }
    })
  }
}

resource "humanitec_resource_definition_criteria" "example-k8s-service-account_criteria_0" {
  resource_definition_id = resource.humanitec_resource_definition.example-k8s-service-account.id
  app_id                 = "example-break-a-loop-additional-resource"
}

def-workload.tf (view on GitHub) :

resource "humanitec_resource_definition" "example-workload" {
  driver_type = "humanitec/template"
  id          = "example-workload"
  name        = "example-workload"
  type        = "workload"
  driver_inputs = {
    values_string = jsonencode({
      "templates" = {
        "outputs" = <<END_OF_TEXT
update: 
- op: add
  path: /spec/serviceAccountName
  {{/*
    The resource reference does not specify ID or class so the ID and
    class of the workload being provisioned will be used.
  *//}
  value: $${resources.k8s-service-account.outputs.name}
END_OF_TEXT
      }
    })
  }
}

resource "humanitec_resource_definition_criteria" "example-workload_criteria_0" {
  resource_definition_id = resource.humanitec_resource_definition.example-workload.id
  app_id                 = "example-break-a-loop-additional-resource"
}