Architecture

The need to manage secrets #

The Humanitec Operator is motivated by the need to manage secrets in an Internal Developer Platform (IDP).

Secrets must be handled in various places when using the Humanitec Platform Orchestrator, e.g. as shared values and secrets for Applications, for certain types of Resource Definitions, or when configuring Drivers.

By default, the Humanitec SaaS solution uses its own secret store. Whenever it needs to manage secrets for later use, it saves them to that secret store and retrieves them as and when it needs them. This could be to populate a Kubernetes (K8s) Secret for an Application Deployment, or when calling a Driver that needs a secret to perform its task.

This may not be what you want. Many organizations want to have full control over their secrets and manage their own store, in their own infrastructure, to maintain their security posture. The Humanitec Operator introduces a new deployment model to make that possible. Let’s go through the different scenarios one by one.

Modes of operation: Direct Cluster #

This is how it works without the Humanitec Operator: when using full SaaS, the Humanitec Platform Orchestrator securely stores any secrets an Application needs in a separate, internal secret store. It inserts fully rendered K8s manifests into the target cluster when an Application is deployed, including any necessary K8s Secret manifests. The Platform Orchestrator then retrieves the secrets from its internal secret store to produce these Secret manifests, or when calling Drivers that require a secret.

Modes of operation: Humanitec Operator #

When secrets are stored inside your organization’s infrastructure, the Platform Orchestrator will not have access to them. The manifests deployed into the K8s cluster there cannot be rendered entirely by the Platform Orchestrator. Likewise, it cannot call any Driver requiring a secret.

The Platform Orchestrator automatically switches to Humanitec Operator mode when configured with an external secret store marked primary. See The primary secret store below for details.

It then renders the manifests only partially and generates a set of instructions outlining how to complete the rendering procedure. The target K8s cluster receives this information as a set of custom resources (CRs), based on CRDs that come with the Humanitec Operator installation.

The Humanitec Operator picks up these custom resources and completes the remaining rendering stages. This entails obtaining secrets from the secret store, calling any Drivers required to provision Resources, and creating the final K8s manifests.

You may optionally still allow users of the Platform Orchestrator to write secrets to the secret store using the Orchestrator just like in Direct Cluster mode. See The primary secret store below for details.

In any case, you always need to register the secret store with both the Platform Orchestrator and the Humanitec Operator. The instructions (CRs) the Platform Orchestrator creates reference a specific secret store by its ID which must match a secret store registration with the Humanitec Operator.

Modes of operation: Humanitec Operator with GitOps #

In the above “Humanitec Operator” mode the Platform Orchestrator still requires access to the API server of the target K8s Cluster to manage the custom resources.

If you do not wish to allow that, you need another way to write those custom resources into the cluster. This is where GitOps comes in. Here, the Platform Orchestrator writes the custom resource manifests to a Git repository instead of straight to the cluster, and a GitOps operator (like ArgoCD or Flux) performs the job of pulling those manifests into the cluster itself.

The Humanitec Operator will then again be able to execute the instructions represented by the custom resources just like before.

Speaking purely from the perspective of the Humanitec Operator, there is no difference between Operator mode with or without GitOps. All the Humanitec Operator sees is that someone creates custom resources in the cluster, and it executes the instructions they convey. In plain Humanitec Operator mode, that someone is the Platform Orchestrator, whereas with GitOps it is the GitOps operator (acting on behalf of the Platform Orchestrator).

You technically enable this mode by switching to Humanitec Operator mode and creating a Resource Definition for your Kubernetes cluster that uses a specific Driver named k8s-cluster-git on top of that.

The Humanitec Agent is another option that allows you to operate without the Platform Orchestrator accessing the API server.

Implications of switching to Humanitec Operator mode #

Switching from Direct Cluster to Operator mode has some implications to be aware of.

Existing secrets in Humanitec secret store #

Any secrets maintained while running the Platform Orchestrator in Direct Cluster mode have been stored in the secret store of the Humanitec SaaS system. Access to these secrets will be lost when you switch to Humanitec Operator mode.

If you wish to continue maintaining secrets via the Platform Orchestrator, you should re-save the Orchestrator objects containing the secrets. They will then be saved in the new primary secret store.

Default Humanitec Resource Definitions #

Some of the “Default Humanitec ...” Resource Definitions provided from the start in your Humanitec Organization will not work any longer. They require access to secrets maintained in the Humanitec secret store, which is not accessible to the Operator.

These Resource Definitions are affected:

• Default Humanitec DNS (Resource Type: dns. See DNS for an overview on DNS Driver types.)

Make sure you have provided replacement Resource Definitions for any Resource Types you need. Adjust matching criteria so that Applications deployed via the Humanitec Operator will no longer match the affected Default Resource Definitions. You can check their Active Resources to identity the relevant Applications.

Multiple secret stores #

While the descriptions above mention only one secret store for brevity, you can configure multiple secret stores for Humanitec Operator or GitOps mode. You may, for example, wish to use separate secret stores for different Applications, different Environments, or both.

You can mix and match different types of secret stores as long as your infrastructure can provide them, e.g. a HashiCorp Vault and a Google Secret Manager if you are running workloads on Google Cloud.

The secret store to use for reading a particular secret is part of its secret reference.

The primary secret store #

The primary secret store is a choice made for the secret store registrations on the Platform Orchestrator side.

At most one secret store can be marked primary. Defining a primary store has to two functions:

1. If users write secret values via the Platform Orchestrator, it will try to write them to the current primary store.
2. The Platform Orchestrator automatically switches to “Humanitec Operator” mode when configured with an external secret store marked primary.

If none of your registered secret stores are the primary store, the Humanitec-hosted secret store will become the primary, and any secret values written via the Platform Orchestrator will be saved to that store.

If you want to allow users to write secrets via the Platform Orchestrator to one of your own secret stores, designate the target secret store as primary and establish write access for the Platform Orchestrator. The individual How-to pages on connecting secret stores show how to do that.

The default secret store #

The default secret store is a choice made on the Humanitec Operator side.

Besides your own secrets, the Humanitec Operator needs to store “Resource cookies” (an operational state) internally in any one secret store. It will use the store marked as default among all secret store configurations for the Operator. There must be exactly one default secret store.

The default secret store can be a store you use for your regular secrets, or a separate store, at your discretion.

The individual How-to pages on connecting secret stores show how to declare a secret store the default.

Secret store access levels #

Depending on the mode of operation, you need to allow the appropriate levels of access for your secret store to the Humanitec Operator and/or the Platform Orchestrator.

In Direct Cluster mode, Humanitec’s own secret store is used. Users of the Platform Orchestrator use the Orchestrator as an interface to write secrets. Secret references cannot be used in Direct Cluster mode as we do not expose the internal structure of the secret store.

In Humanitec Operator or GitOps mode, your secret store - or stores - are used. These are your options regarding access levels:

• The Platform Orchestrator does not need read access to any store.
• The Platform Orchestrator needs write access only to the primary secret store if you wish to enable your users to write secrets through the Platform Orchestrator. Without write access, they’ll get an error if they try.
• Giving the Platform Orchestrator no access is a valid option. In this case, you must use secret references at all times.
• The Humanitec Operator needs read access to all secret stores.
• The Humanitec Operator needs write access only to the default secret store to write Resource cookies.

The Humanitec Operator in a self-hosting scenario #

When you’re self-hosting the Platform Orchestrator, using the Humanitec Operator is optional. The same considerations concerning the reachability of the cluster API server and secret store access apply. The only difference is that they are now evaluated inside the infrastructure you control.

Calling Drivers #

The Humanitec Operator will call Drivers to manage Resources. This might require sending secrets to Drivers as part of the payload. Depending on where those Drivers are hosted, those secrets - while being encrypted in transit - will leave your network. This will apply to any Humanitec-hosted drivers. It excludes the Echo and Template Drivers whose logic is executed inside the Operator itself.

If your security posture restricts you from sending secrets outside your own network, consider using self-hosted Drivers. Contact Humanitec support for details.

Next steps #

See how to install the Humanitec Operator in your infrastructure.