Resource Graph Patterns

Resource Graph Patterns

This repo contains a set of examples of patterns that can be used when building Resource Graphs. Each pattern is explained along with use cases for when the pattern can be useful.

Running the examples

Prerequisites

In order to try out these examples, the following is necessary:

The environment must be configured with the following environment variables:

Variable Description
HUMANITEC_ORG The ID of the Humanitec Organization.
HUMANITEC_TOKEN The API Token for the service user with Administrator role on the Humanitec Organization
HUMANITEC_DEV This should be set to development

NOTE:

It is usually necessary to export the environment variable into the shell’s environment if you wish to use the CLIs interactively.

For example:

export HUMANITEC_ORG="my-org"

Break a loop additional resource

Break a loop with an additional resource

This example demonstrates how to break a loop where two resources have to both depend on each others outputs.

How the example works

There is often a mutual loop assigning a principal to a Kubernetes service account to enable Workload Identity. The Kubernetes service account often needs to be annotated with the principal and the principal needs some policy to allow it to be used by the Kubernetes Service Account.

In this example, we will simulate the graph using a k8s-service-account and a fake aws-role resource both implemented with the humanitec/template driver.

graph LR
    workload --> k8s-service-account
    k8s-service-account --> aws-role
    aws-role --> k8s-service-account

The loop arises because it is necessary to generate both the Kubernetes service account name and the aws-role dynamically. This is because these need to be unique for each Workload in the Application. Essentially, both resources require the same two pieces of information.

There are two ways to break the loop:

  1. Convention

    Decide that each resource “knows” how to generate both pieces of information. This can be achieved by using the context to provide the unique element.

    This has the downside that it is inflexible and limiting. For example, if a 3rd party system is used to issue principals, then this technique will not work.

  2. Add an additional resource

    Both the k8s-service-account and aws-role resources get both the service account name and role ID from a 3rd resource.

    This approach ensures consistency, does not rely on convention and allows for complex scenarios like getting IDs from a 3rd party system.

graph LR
    workload --> k8s-service-account
    k8s-service-account --> aws-role
    k8s-service-account --> config
    aws-role --> config

score.yaml(view on GitHub):

apiVersion: score.dev/v1b1

metadata:
  name: example-workload

containers:
  busybox:
    image: busybox:latest

    variables:
      BUCKET_NAME: ${resources.my-s3.bucket}

    command:
      - /bin/sh
    args:
      - "-c"
      # This will output all of the environment variables in the container to
      # STDOUT every 15 seconds. This can be seen in the container logs in the
      # Humanitec UI.
      - "while true; do set; sleep 15; done"

resources:
  my-s3:
    type: s3
    # Change the class to "two" 
    class: one

Resource definitions


aws-role-def.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: example-aws-role
entity:
  criteria:
    - app_id: example-break-a-loop-additional-resource
  driver_inputs:
    values:
      role_arn: ${resources["config.sa-name-role-id"].outputs.role_arn}
      sa_name: ${resources["config.sa-name-role-id"].outputs.sa_name}
      templates:
        outputs: |
          arn: {{ .drivers.role_arn }}
  driver_type: humanitec/template
  name: example-aws-role
  type: aws-role


config-name-id.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: example-config-name-id
entity:
  criteria:
    # We only match to class and not res_id because the res_id changes for
    # each workload
    - class: sa-name-role-id
      app_id: example-break-a-loop-additional-resource
  driver_inputs:
    values:
      res_id: ${context.res.id}
      app_id: ${context.app.id}
      templates:
        init: |
          workload_id: {{ .driver.values.res_id | splitList "." | last }}
        outputs: |
          role_arn: "arn:aws:iam::123456789012:role/{{ .driver.values.app_id }}/sa-role-{{ .init.workload_id }}"
          sa_name: {{ .init.workload_id }}-sa
  driver_type: humanitec/template
  name: example-config-name-id
  type: config


k8s-service-account-def.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: example-k8s-service-account
entity:
  criteria:
    - app_id: example-break-a-loop-additional-resource
  driver_inputs:
    values:
      role_arn: ${resources["config.sa-name-role-id"].outputs.role_arn}
      sa_name: ${resources["config.sa-name-role-id"].outputs.sa_name}
      templates:
        outputs: |
          name: {{ .driver.values.sa_name }}
        manifests: |
          service-account.yaml:
            location: namespace
            data: |
              apiVersion: v1
              kind: ServiceAccount
              metadata:
                name: {{ .driver.values.sa_name }}
                annotations:
                  eks.amazonaws.com/role-arn: {{ .driver.values.role_arn }}

  driver_type: humanitec/template
  name: example-k8s-service-account
  type: k8s-service-account


workload-def.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: example-workload
entity:
  criteria:
    - app_id: example-break-a-loop-additional-resource
  driver_inputs:
    values:
      templates:
        outputs: |
          update: 
          - op: add
            path: /spec/serviceAccountName
            {{/*
              The resource reference does not specify ID or class so the ID and
              class of the workload being provisioned will be used.
            *//}
            value: ${resources.k8s-service-account.outputs.name}
  driver_type: humanitec/template
  name: example-workload
  type: workload

Delegator resource

Delegator Resource

This example demonstrates how delegator Resource Definitions can be used to expose a shared base resource with different access policies.

How the example works

This example is made up of:

  • Two delegator s3 Resource Definitions
  • One base s3 Resource Definition
  • Two aws-policy Resource Definitions

and the resulting graph will look like:

flowchart LR
    WL_A[Workload A] -->|score resource dependency| DELEGATOR_RES_ADMIN(type: s3, id: s3-bucket, class: example-admin)
    WL_B[Workload B] -->|score resource dependency| DELEGATOR_RES_READ_ONLY(type: s3, id: s3-bucket, class: example-read-only)
    DELEGATOR_RES_ADMIN -->|co-provision| POL_ADMIN(type: aws-policy, class: s3-example-admin)
    DELEGATOR_RES_ADMIN -->|Resource Reference| BASE_RES(shared: s3-bucket, class: example)
    DELEGATOR_RES_READ_ONLY -->|co-provision| POL_READ_ONLY(type: aws-policy, class: s3-example-read-only)
    DELEGATOR_RES_READ_ONLY -->|Resource Reference| BASE_RES(type: s3, id: s3-bucket, class: example)

To keep the examples as simple as possible, the humanitec/echo driver is used. Checkout Resource Packs, if you are interested in examples with Resource Definitions that also include provisioning.

The s3 Resource Definition s3-example.yaml defines the underlying “base” resource and is matched as class: example.

The aws-policy Resource Definitions aws-policy-s3-example-admin.yaml and aws-policy-s3-example-read-only.yaml contain the different policies we want to make available. Those are matched as example-admin and example-read-only.

The s3 Resource Definitions s3-example-admin.yaml and s3-example-read-only.yaml are delegator resources that have two functions:

  • Co-provision the respective aws-policy Resource Definition.
  • Forward the outputs of the “base” resource using a Resource Reference.

When the workload defined in score-a.yaml now requests an s3 resource with class: example-admin, the Humanitec Platform Orchestrator creates the “base” s3 resource class: example and co-provisions the aws-policy resource class: s3-example-admin.

Similar to the first workload, score-b.yaml requests an s3 resource, but this time with class: example-read-only and here the Humanitec Platform Orchestrator creates the “base” s3 resource class: example and co-provisions the aws-policy resource class: s3-example-read-only.

As both workloads used the same s3 resource id shared.main-s3 via the annotation score.humanitec.io/resId in their Score files, they will use the same underlying s3 bucket, but each workload uses a different access policy. Look here to learn more about this annotation.

Run the demo

Prerequisites

See the prerequisites section in the README at the root of this section.

In addition, the environment variable HUMANITEC_APP should be set to example-delegator.

Cost

This example will result in a two Pods being deployed to a Kubernetes cluster.

Deploy the example

  1. Create a new app:

    humctl create app "${HUMANITEC_APP}"
    
  2. Register the Resource Definitions:

    humctl apply -f ./resource-definitions
    
  3. Deploy the Score workload A:

    score-humanitec delta --deploy --org "${HUMANITEC_ORG}" --app "${HUMANITEC_APP}" --env "${HUMANITEC_ENV}" --token "${HUMANITEC_TOKEN} --retry --file score-a.yaml
    
  4. Deploy the Score workload B:

    score-humanitec delta --deploy --org "${HUMANITEC_ORG}" --app "${HUMANITEC_APP}" --env "${HUMANITEC_ENV}" --token "${HUMANITEC_TOKEN} --retry --file score-b.yaml
    

Clean up the example

  1. Delete the Application:

    humctl delete app "${HUMANITEC_APP}"
    
  2. Delete the Resource Definitions:

    humctl delete -f ./resource-definitions
    

score-a.yaml(view on GitHub):

apiVersion: score.dev/v1b1
metadata:
  name: example-a

containers:
  busybox:
    image: busybox:latest

    command:
      - /bin/sh
    args:
      - "-c"
      # This will output all of the environment variables in the container to
      # STDOUT every 15 seconds. This can be seen in the container logs in the
      # Humanitec UI.
      - "while true; do set; sleep 15; done"
    variables:
      BUCKET_NAME: ${resources.s3.bucket}

resources:
  s3:
    metadata:
      annotations:
        score.humanitec.io/resId: shared.main-s3
    type: s3
    class: example-admin


score-b.yaml(view on GitHub):

apiVersion: score.dev/v1b1
metadata:
  name: example-b

containers:
  busybox:
    image: busybox:latest

    command:
      - /bin/sh
    args:
      - "-c"
      # This will output all of the environment variables in the container to
      # STDOUT every 15 seconds. This can be seen in the container logs in the
      # Humanitec UI.
      - "while true; do set; sleep 15; done"
    variables:
      BUCKET_NAME: ${resources.s3.bucket}

resources:
  s3:
    metadata:
      annotations:
        score.humanitec.io/resId: shared.main-s3
    type: s3
    class: example-read-only

Resource definitions


aws-policy-s3-example-admin.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: aws-policy-s3-example-admin
entity:
  criteria:
    - app_id: example-delegator
      class: s3-example-admin
  driver_inputs:
    values:
      arn: arn:aws:iam::aws:policy/AmazonS3FullAccess
  driver_type: humanitec/echo
  name: aws-policy-s3-example-admin
  type: aws-policy


aws-policy-s3-example-read-only.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: aws-policy-s3-example-read-only
entity:
  criteria:
    - app_id: example-delegator
      class: s3-example-read-only
  driver_inputs:
    values:
      arn: arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
  driver_type: humanitec/echo
  name: aws-policy-s3-example-read-only
  type: aws-policy


s3-example-admin.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: s3-example-admin
entity:
  criteria:
    - app_id: example-delegator
      class: example-admin
  driver_inputs:
    values:
      # This Resource reference to 's3.example' creates the dependency to the base resource
      bucket: ${resources['s3.example'].outputs.bucket}
  provision:
    aws-policy.s3-example-admin:
      is_dependent: false
      match_dependents: true
  driver_type: humanitec/echo
  name: s3-example-admin
  type: s3


s3-example-read-only.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: s3-example-read-only
entity:
  criteria:
    - app_id: example-delegator
      class: example-read-only
  driver_inputs:
    values:
      # This Resource reference to 's3.example' creates the dependency to the base resource
      bucket: ${resources['s3.example'].outputs.bucket}
  provision:
    aws-policy.s3-example-read-only:
      is_dependent: false
      match_dependents: true
  driver_type: humanitec/echo
  name: s3-example-read-only
  type: s3


s3-example.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: s3-example
entity:
  criteria:
    - app_id: example-delegator
      class: example
  driver_inputs:
    values:
      bucket: example-bucket
  driver_type: humanitec/echo
  name: s3-example
  type: s3

Propagate class

Propagate Class

This example demonstrates how Resource classes can be propagate via Resource References. It involves having a single Resource Definition that can be parameterized by referencing another Resource.

How the example works

This example is made up of 3 Resource Definitions: one s3 and two config Resource Definitions. To keep the examples as simple as possible, the humanitec/echo driver is used.

The s3 Resource Definition s3-def.yaml is configured to match to two classes one and two. It outputs its bucket output based on the Resource Reference ${resources.config#example.outputs.name}. This Resource Reference will cause a new resource to be provisioned and be replaced with the name output of that newly provisioned resource. As the Resource Definition only defines the type of the resource (config) and the ID of the resource (example), the class that the new resource will be provisioned with will be the same as the class of the s3 resource.

The one config Resource Definition (config-one-def.yaml) is configured to match the class one and the other config-two-def.yaml matches the class two.

The score.yaml file depends on a resource of type s3 bucket with a class of one. It outputs the bucket name in the environment variable BUCKET_NAME.

This means that an S3 bucket will be provisioned via the s3-def.yaml Resource Definition and the bucket name will be pulled from the Resource Definition config-one-def.yaml is configured to match the class one and so will be name-01.

If the class on the s3 resource is changed to two then the bucket name will be pulled from the Resource Definition config-two-def.yaml is configured to match the class two and so will be name-02.

Run the demo

Prerequisites

See the prerequisites section in the README at the root of this section.

In addition, the environment variable HUMANITEC_APP should be set to example-propagate-class.

Cost

This example will result in a single pod being deployed to a Kubernetes Cluster.

Deploy the example

  1. Create a new app:

    humctl create app "${HUMANITEC_APP}"
    
  2. Register the resource definitions:

    humctl apply -f ./resource-definitions
    
  3. Deploy the score workload:

    score-humanitec delta --deploy --org "${HUMANITEC_ORG}" --app "${HUMANITEC_APP}" --env "${HUMANITEC_ENV}" --token "${HUMANITEC_TOKEN}
    

Play with the demo

  1. In the Humanitec UI, visit the running deployment and look in the container logs for the line starting with BUCKET_NAME. It should have a value of name-01

  2. Change the class of the s3 resource in the score.yaml file from one to two.

  3. Redeploy the Score file:

    score-humanitec delta --deploy --org "${HUMANITEC_ORG}" --app "${HUMANITEC_APP}" --env "${HUMANITEC_ENV}"--token "${HUMANITEC_TOKEN}
    
  4. In the Humanitec UI, visit the running deployment and look in the container logs for the line starting with BUCKET_NAME. It should now have a value of name-02

Clean up the example

  1. Delete the Application:

    humctl delete app "${HUMANITEC_APP}"
    
  2. Delete the Resource Definitions:

    humctl delete -f ./resource-definitions
    

score.yaml(view on GitHub):

apiVersion: score.dev/v1b1

metadata:
  name: example-workload

containers:
  busybox:
    image: busybox:latest

    variables:
      BUCKET_NAME: ${resources.my-s3.bucket}

    command:
      - /bin/sh
    args:
      - "-c"
      # This will output all of the environment variables in the container to
      # STDOUT every 15 seconds. This can be seen in the container logs in the
      # Humanitec UI.
      - "while true; do set; sleep 15; done"

resources:
  my-s3:
    type: s3
    # Change the class to "two" 
    class: one

Resource definitions


config-one-def.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: example-config-one
entity:
  criteria:
    - class: one
      res_id: example
      app_id: example-propagate-class
  driver_inputs:
    values:
      name: name-01
  driver_type: humanitec/echo
  name: example-config-one
  type: config


config-two-def.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: example-config-two
entity:
  criteria:
    - class: two
      res_id: example
      app_id: example-propagate-class
  driver_inputs:
    values:
      name: name-02
  driver_type: humanitec/echo
  name: example-config-two
  type: config


s3-def.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: example-s3
entity:
  criteria:
    - class: one
      app_id: example-propagate-class
    - class: two
      app_id: example-propagate-class
  driver_inputs:
    values:
      bucket: ${resources.config#example.outputs.name}
  driver_type: humanitec/echo
  name: example-s3
  type: s3

Propagate id

Propagate Class

This example demonstrates how ID propagation through Resource References can be used to create a new instance of a resource for another resource. It involves provisioning a k8s-service-account resource for every workload resource provisioned.

How the example works

This example is made up of two Resource Definitions: one workload and two k8s-service-account Resource Definition. Both Resource Definitions use the humanitec/template driver.

A resource of type workload is automatically provisioned for each Workload in an Application in Humanitec. The workload will have the ID of modules.<workload id> and a class of default. This means that if an Application contains two workloads called workload-one and workload-two, two workload resources will be provisioned, one with ID modules.workload-one and the other with ID modules.workload-two.

The workload-def.yaml Resource Definition has a reference to a k8s-service-account resource. The Resource Reference does not specify either the class or the ID of the resource. This means that the k8s-service-account resource is provisioned with the same ID and class as the Workload.

Run the example

Prerequisites

See the prerequisites section in the README at the root of this repository.

In addition, the environment variable HUMANITEC_APP should be set to example-propagate-class.

Cost

This example will result in a single Pod being deployed to a Kubernetes Cluster.

Deploy the example

  1. Create a new Application:

    humctl create app "${HUMANITEC_APP}"
    
  2. Register the Resource Definitions:

    humctl apply -f ./resource-definitions
    
  3. Deploy the Score Workload:

    score-humanitec delta --deploy --org "${HUMANITEC_ORG}" --app "${HUMANITEC_APP}" --env "${HUMANITEC_ENV}" --token "${HUMANITEC_TOKEN}
    

Clean up the example

  1. Delete the Application

    humctl delete app "${HUMANITEC_APP}"
    
  2. Delete the Resource Definitions

    humctl delete -f ./resource-definitions
    

score.yaml(view on GitHub):

apiVersion: score.dev/v1b1

metadata:
  name: example-workload

containers:
  busybox:
    image: busybox:latest

    command:
      - /bin/sh
    args:
      - "-c"
      # This will output all of the environment variables in the container to
      # STDOUT every 15 seconds. This can be seen in the container logs in the
      # Humanitec UI.
      - "while true; do set; sleep 15; done"

Resource definitions


k8s-service-account-def.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: example-k8s-service-account
entity:
  criteria:
    - app_id: example-propagate-id
  driver_inputs:
    values:
      res_id: ${context.res.id}
      templates:
        init: |
          name: {{ .driver.values.res_id | splitList "." | last }}-sa
        outputs: |
          name: {{ .init.name }}
        manifests: |
          service-account.yaml:
            location: namespace
            data: |
              apiVersion: v1
              kind: ServiceAccount
              metadata:
                name: {{ .init.name }}

  driver_type: humanitec/template
  name: example-k8s-service-account
  type: k8s-service-account


workload-def.yaml(view on GitHub):

apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: example-workload
entity:
  criteria:
    - app_id: example-propagate-id
  driver_inputs:
    values:
      templates:
        outputs: |
          update: 
          - op: add
            path: /spec/serviceAccountName
            {{/*
              The resource reference does not specify ID or class so the ID and
              class of the workload being provisioned will be used.
            *//}
            value: ${resources.k8s-service-account.outputs.name}
  driver_type: humanitec/template
  name: example-workload
  type: workload

Top