Resource Graph

Resource Definitions can reference other Resource Definitions. This means that when a resource is provisioned, it might require other resources to be provisioned before it. The relationship between all the resources that need to be provisioned during a deployment is known as the Resource Graph. How a resource relates to or references other resources is defined by the Resource Definition.

The Resource Graph is a Directed Acyclic Graph (DAG) . This means that references that form a loop are not allowed. This is because a loop means that a resource would end up referencing itself. The graph is used to work out the order in which resources should be provisioned during a deployment. This is done by determining a Topological Order of the graph.

The shape of the Resource Graph for a particular deployment is affected by both the developer (via the Deployment Set or Score File) and by the platform team (via Resource Definitions). Developers can add resources to the graph via Resource Dependencies. Platform teams can add resources and additional connections in the graph via Resource References , Co-Provisioning and Resource Selectors .

Building and executing #

For every deployment, a Resource Graph is first built and then executed on.

For each deployment into an Environment, the Platform Orchestrator builds the Resource Graph using in the following steps:

1. Gather Types and IDs of all resources to be provisioned for the Deployment Set being deployed.

   This includes:

   • all private and shared resource dependencies in the deployment set
   • a workload resource type with ID modules.<WORKLOAD ID>
   • implicit resources such as base-env or k8s-cluster

   Any resources that are referenced via placeholders in a workload are added as dependencies of the relevant workload resource in the Graph.

2. For each resource to be provisioned, the Platform Orchestrator looks up the appropriate Resource Definition using Matching Criteria.

3. Each Resource Definition is analyzed to see if it adds new resources or connections to the Resource Graph.

   If a new resource is added to the graph, steps 2 and 2 are related to that new resource.

When the graph is ready to execute, a Topological Order of the graph is determined. The resources are then provisioned in that order. This means that each resource is provisioned before any resources that depend on it.

Basic graphs #

The simplest Resource Graph just involves dependent and implicit resources.

Dependent resources are resources that are directly required by a workload. Examples of dependent resources could include a private postgres database that a workload connects to or a shared dns that the workload is exposed under.

Implicit resources are provisioned automatically by the Platform Orchestrator. These include: workload, base-env, k8s-cluster and k8s-namespace resources. See the Resource Types for a complete list of types including all implicit ones.

Example #

Consider a simple Application made up of a single workload with a private resource dependency on a PostgreSQL database. In this case, there is one workload Resource depending on a postgres, a single base-env, a single k8s-cluster, and a single k8s-namespace. The latter three are implicit Types which are always automatically provisioned.

The base-env is unconnected. This means that it can be provisioned in any order. The postgres Resource is connected to the workload Resource. This means that it needs to be provisioned before the workload Resource is provisioned.

Why do I not see the cluster and namespace in the Resource Graph?

Some implicit Resources need to be technically provisioned before all others, so internally the Platform Orchestrator creates a separate Graph for them. This includes the k8s-cluster, k8s-namespace, and also an agent if there is one. These Resources are therefore not shown in the Resource Graph by default. They can still be referenced normally by another Resource and will then also be shown in the Graph.

Resource References #

A Resource Reference is a Placeholder that can be used in a Resource Definition to use the outputs of another Resource. The Resource Reference also adds the Resource to the Graph if it does not already exist. It becomes a direct dependency of the current Resource.

If the referenced Resource already exists in the Graph with the same Descriptor (same Type, Class, and ID), then the reference points to the same Resource, i.e. it is not added again.

Any Driver input in the Resource Definition can contain a Resource Reference.

Example:

        
        
      ${resources['dns.public#shared.api-dns'].outputs.host}

This reference resolves to the host value as outputted by the dns Resource with a Resource Class of public and a Resource ID of shared.api-dns.

A Resource that references another Resource will be provisioned after the Resource it references.

Some limitations apply when using the Humanitec Operator.

Resource Reference format #

Resources are referenced by the tuple (combination of) Resource Type, Resource Class, and Resource ID. If Resource ID is omitted, the Resource ID of the referencing Resource is used (propagated). If Resource Class is omitted, the Resource Class of the referencing Resource is used (propagated).

The Propagate class pattern demonstrates a use of this mechanism.

The Placeholder has the following format:

        
        
      resources.DESC.outputs.OUTPUT[.SUB_PROPERTY...]

where:

• DESC is the Resource Descriptor with some elements being optional:

  
          
          
        TYPE[.CLASS][#ID]
  

  where:

  • TYPE is the type of the Resource, e.g. dns
  • CLASS is the optional Resource Class of the Resource, e.g. public. If omitted or set to @ (“here”), the Class of the current Resource is used
  • ID is the optional Resource ID , e.g. shared.api-dns. If omitted or set to @ (“here”), the ID of the current Resource is used

• OUTPUT is an output from the Resource, e.g. host

• SUB_PROPERTY... can be further sub properties if the value of OUTPUT is a complex type.

As ID might contain ., it is common to write the placeholder using [] notation for the TYPE.CLASS#ID tuple as shown ealier:

        
        
      ${resources['dns.public#shared.api-dns'].outputs.host}

Resource Reference example #

The previous example can be extended as follows:

1. The base-env resource creates the PostgreSQL instance that the postgres resource needs.
2. The workload needs to run under a specific Kubernetes Service Account.

Both of these should not affect how the developer specifies their workload.

This can be modeled using Resource References.

1. Reference the base-env from the Postgres resource #

To get the outputs from the base-env the Resource Definition for the postgres resource needs to reference the outputs of the base-env resource:

        
        
      apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: postgres-def
entity:
  type: postgres
  name: postgres-def
  driver_type: humanitec/postgres
  driver_inputs:
    values:
      host: ${resources.base-env#base-env.outputs.pghost}
      name: ${resources.base-env#base-env.outputs.pgname}
      port: ${resources.base-env#base-env.outputs.pgport}
      append_host_to_user: false
    secrets:
      dbcredentials:
        username: ${resources.base-env#base-env.outputs.pguser}
        password: ${resources.base-env#base-env.outputs.pgpassword}
  criteria:
  - app_id: example-app

In this case, the Resource Type is base-env and the Resource ID is also base-env.

2. Provision a service account for the workload #

The workload needs to reference a resource of type k8s-service-account and retrieve the name output. This can then be used to set the serviceAccountName field in the workload.

        
        
      apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: workload-with-sa
entity:
  type: workload
  name: workload-with-sa
  driver_type: humanitec/echo
  driver_inputs:
    values:
      update:
      - op: add
        path: /spec/serviceAccountName
        value: ${resources.k8s-service-account.outputs.name}
  criteria:
  - app_id: example-app

Changes from the previous example:

• An additional Resource of type k8s-service-account with a Resource ID that is the same as the workload is created. It is depended on by the workload resource
• This is also a dependency added on the base-env from the postgres resource

3. More examples #

Our Example Library contains further examples making use of Resource References. Inspect their code to see the ${resources..} notation applied to solve the particular use case of each example.

• S3 Backends
• Credentials
• Labels
• Co-provisioning

Summary: Resource References #

Resource References can be used to:

• chain resources together such that the output of one resource can be used as the input of another
• provision new resources by adding them into the Resource Graph

Co-provision Resources #

Co-provisioning allows additional resources to be added to the Resource Graph without a Resource Reference required. These additional resources are listed in a Resource Definition and are added whenever that Resource Definition is matched.

It can be necessary to provision an additional resource that is not depended on by the resource it should be provisioned with. For example, an IAM access policy for a PostgreSQL database should depend on the PostgreSQL database rather than the other way around. This means that Resource References alone cannot be used to model this. Instead, a Resource Definition can define additional resources to be Co-provisioned. This co-provisioned resource can then use a Resource Reference to create a dependency in the opposite direction.

A Resource Definition can define that additional resources need to be provisioned when the one it defines is provisioned. These additional resources can be added with a range of different links to the resource being provisioned and its parent resources.

You define co-provisioning by adding a provision section to the Resource Definition like this:

        
        
      entity:
  ...
  provision:
    # Name the Resource Type of the resource that should be co-provisioned
    # Optionally specify Resource class and/or ID as <type>.<class>#<ID>, e.g. "aws-policy.strict#shared.policy1"
    aws-policy:
      is_dependent: false
      match_dependents: true

Specify these attributes to model the Resource Graph:

• Optionally specify a Resource Class and/or Resource ID for the co-provisioned Resource. This statement will co-provision a Resource of type aws-policy with the Class strict and the ID shared.policy-1:

  
          
          
        provision:
    aws-policy.strict#shared.policy-1:
    ...
  

  Omitting either attribute or setting it to @ (“here”) will use the value of the current Resource.

• is_dependent (true / false): states whether the co-provisioned resource depends on its co-provisioning resource.

  • Note that when the co-provisioned resource has a reference to its co-provisioning resource, there will be a dependency at any rate. Setting this attribute to true will introduce a dependency into the graph in all cases.

• match_dependents (true / false): states whether the resources which depend on the co-provisioning resource will depend on the co-provisioned resource as well.

  • This is useful if you want to traverse the Resource Graph using Resource Selectors to query the co-provisioned Resources.

In the following diagram, the resource R is depended on by its parent P. The resource R co-provisions an additional resource N.

Example: PostgreSQL and IAM Policy #

The previous example can be further extended as follows:

• An AWS IAM Policy (aws-policy) needs to be created for each postgres resource created. This needs the name of the database and so must depend on the postgres

This can be modeled by:

• co-provisioning an aws-policy in the postgres Resource Definition. This is done without specifying a Resource ID so that it uses the same Resource ID as the postgres resource.
• In the Resource Definition for the aws-policy, a Resource Reference is used to retrieve the database name.

        
        
      apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: postgres-def
entity:
  type: postgres
  name: postgres-def
  driver_type: humanitec/postgres
  driver_inputs:
    values:
      host: ${resources.base-env#base-env.outputs.pghost}
      name: ${resources.base-env#base-env.outputs.pgname}
      port: ${resources.base-env#base-env.outputs.pgport}
      append_host_to_user: false
    secrets:
      dbcredentials:
        username: ${resources.base-env#base-env.outputs.pguser}
        password: ${resources.base-env#base-env.outputs.pgpassword}
  provision:
    aws-policy:
      is_dependent: false
      match_dependents: true
  criteria:
  - app_id: example-app

        
        
      apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: iam-policy
entity:
  type: aws-policy
  name: iam-policy
  driver_type: my-org/aws-policy
  driver_inputs:
    values:
      db_name: ${resources.postgres.outputs.name}
  criteria:
  - app_id: example-app

Changes from the previous example:

• An additional Resource of type aws-policy with a Resource ID that is the same as the postgres is created
• The aws-policy has a Resource Reference to a Resource of type postgres without specifying the Resource ID. This matches the postgres Resource that co provisioned it because it defaults to use Resource ID of the Resource making the Resource Reference

Example: S3 and IAM Policy #

Our Example Library features another complete example on co-provisioning for S3 and IAM policies .

Summary: Co-provisioning #

Co-provisioning can be used to:

• Add additional Resources to the Resource Graph
• Define additional edges between the co-provisioned resource and the resource that provisioned it and the resources that it depends on

Resource Selectors #

Resource Selectors provide a way of querying the Resource Graph to find sets of Resources that have a particular Type and have a certain dependency. They are defined via Placeholders that expand to an array of values.

Example:

        
        
      ${resources['dns.public#shared.api-dns<route'].outputs.path}

This Resource Selector resolves like this:

1. Start with the Resource of type dns with a Class of public and a Resource ID of shared.api-dns
2. Find all route Resources that depend (<) on that Resource
3. For each route Resource, read its path output

The Resource Selector format extends that of a Resource Reference , but a Selector in different in several ways:

• Resource Selectors only add additional edges to Graph, while a Resource Reference adds a Resource and an edge to the Graph
• Resource Selectors can traverse the Graph doing more than one hop, starting anywhere, while a Resource Reference creates a direct dependency of the current Resource
• A Resource Selector always returns an array, while a Resource Reference matches a single Resource
• A Resource Selector may not find a matching Resource. It will then return an empty array [], while a Resource Reference always has a match because it will add the Resource to the Graph if it does not exist

Any Driver input in the Resource Definition can contain a Resource Reference.

Resource Selector Format #

Resource Selectors extend the Resource Reference format by appending either a > or a < symbol to the DESC to traverse the Graph.

        
        
      resources.DESC.outputs.OUTPUT[.SUB_PROPERTY...]

where:

• DESC is Resource Descriptor extended by a type selector:

  
          
          
        TYPE[.CLASS][#ID][><]SELECTED_TYPE[.SELECTED_CLASS][#SELECTED_ID]
  

  where:

  • TYPE is the type of the resource, e.g. dns

  • CLASS is the optional Resource Class of the Resource, e.g. public. Is set to the Class of the current Resource if omitted or set to @ (“here”)

  • ID is the optional Resource ID , e.g. shared.api-dns. Is set to the ID of the current Resource if omitted or set to @ (“here”)

  • >< is one of > or < indicating the direction of dependence:

    • > makes the selector output all the Resources with type SELECTED_TYPE that are depended on by the Resource identified by the Resource Descriptor
    • < makes the selector output all the Resources with type SELECTED_TYPE that are dependent on the Resource identified by the Resource Descriptor
    • The dependency may be direct or indirect, i.e. any depth in the Graph

  • SELECTED_TYPE is the Resource Type with the dependency on the Resource identified by the Resource Descriptor

  • SELECTED_CLASS is the optional Resource Class of the Resource with the dependency. If omitted, it matches any Class

  • SELECTED_CLASS is the optional Resource ID of the Resource with the dependency. If omitted, it matches any ID

• OUTPUT is an output from the resource, e.g. host

• SUB_PROPERTY... can be further sub properties if the value of OUTPUT is a complex type.

As Resource IDs might contain ., it is common to use [] notation for the selector as shown earlier:

        
        
      ${resources['dns.public#shared.api-dns<route'].outputs.path}

Advanced selectors #

Selectors can traverse the Graph more than one hop by chaining the <> notation, e.g.:

        
        
      ${resources['gcs.@#@<workload>k8s-service-account'].outputs.principal}

The Selector performs these steps:

1. Start with the gcs Resource of the same (@) Class and ID as the current Resource
2. Find all the workload Resources that depend on that gcs
3. Find all the k8s-service-account Resources that these workloads depend on
4. For each k8s-service-account, read its principal output

You may also qualify the inner Selector elements by Class and/or ID, e.g.:

        
        
      ${resources['s3.default<s3.s3-read-only<workload>aws-role'].outputs.arn}

The Selector performs these steps:

1. Start with the s3 Resource of Class default and the same ID as the current Resource
2. Find all the s3 Resources of Class s3-read-only that depend on that s3
3. Find all the workload Resources that depend on these s3 Resources
4. Find all the aws-role Resources that these workload Resources depend on
5. For each aws-role, read its arn output

Descriptor Resource needs to exist #

The Resource specified through the Descriptor, i.e. the first element of the Selector, needs to exist or the deployment will fail. In this example, there has to be a Resource of type dns, Class public, and ID shared.api-dns somewhere in the Graph:

        
        
      ${resources['dns.public#shared.api-dns<route'].outputs.path}

The resources following the first <> do not have to exist. If no route Resource exists depending on that dns in the example, this Selector will return [].

Resource Selector example #

The previous example can be further extended as follows:

• An AWS IAM Role (aws-role) needs to be created that the service account references. The role needs to access the ARNs of all the AWS IAM Policies that the workload depends on.

This can be modeled by:

• Referencing an aws-role from the k8s-service-account. This is done without specifying the ID in the Resource Reference Placeholder so that the aws-role inherits the Resource ID from the k8s-service-account.
• In the Resource Definition for the aws-role, a Resource Selector is used to retrieve the ARNs of all the policies that the workload resource depends on.

        
        
      apiVersion: entity.humanitec.io/v1b1
kind: Definition
metadata:
  id: iam-role
entity:
  type: aws-role
  name: iam-role
  driver_type: my-org/aws-role
  driver_inputs:
    values:
      arns: ${resources.workload>aws-policy.outputs.name}
  criteria:
  - app_id: example-app

Changes from the previous example:

• An additional Resource of type aws-role with a Resource ID that is the same as the workload is created
• The aws-role depends on all the aws-policy Resources that the workload depends on

Summary: Resource Selector #

The Resource Selector can be used to select sets of resources of a particular type out of the Resource Graph.

Working with current Class/ID and “@” #

All Resource Graph mechanisms - Resource References , Co-provisioning , and Resource Selectors - allow you to optionally specify a Resource Class and Resource ID in the Resource Descriptor, or to omit either or both.

If omitted, the Class/ID of the current Resource is used. This is useful for creating and working within a subset of coherent Resources sharing the same class and/or ID in the Graph.

Consider the final setup shown just previously. If two workloads each request a postgres Resource, then the Graph will contain two equivalent subsets of Resources linked by common IDs:

• The Resources depending on my-workload will have IDs derived from that workload’s ID, i.e. modules.my-workload*. They make up the upper structure in the Graph below
• The Resources depending on my-workload-2 will have IDs derived from that workload’s ID, i.e. modules.my-workload-2*. They make up the lower structure in the Graph below

Notably, the base-env is unique as it is referenced by its fixed ID base-env every time.

Resource References or Selectors using the ID of the current Resource can therefore specifically create or locate Resources within the same ID space of the Graph. The Resource Definition code itself can remain generic and work in any context.

To make the intent of using the current Class/ID explicit for human readers, you may specify “@” for “here” instead of omitting it. The following notations are equivalent:

        
        
      ${resources.dns.outputs.host}
${resources['dns'].outputs.host}
${resources['dns.@#@'].outputs.host}
${resources['dns.@'].outputs.host}
${resources['dns#@'].outputs.host}

        
        
      provision:
  aws-policy:
  aws-policy.@#@:
  aws-policy#@:
  aws-policy.@:
  ...

        
        
      ${resources.dns<route.outputs.path}
${resources['dns<route'].outputs.path}
${resources['dns.@#@<route'].outputs.path}
${resources['dns.@<route'].outputs.path}
${resources['dns#@<route'].outputs.path}

Resource Graph patterns #

See our example page of Resource Graph patterns that can be used when building Resource Graphs.