Architecture

The need to manage secrets #

The Humanitec Operator is motivated by the need to manage secrets in an Internal Developer Platform (IDP).

Secrets must be handled in various places when using the Humanitec Platform Orchestrator, e.g. as shared values and secrets for Applications, for certain types of Resource Definitions, or when configuring Drivers .

By default, the Humanitec SaaS solution uses its own secret store. Whenever it needs to manage secrets for later use, it saves them to that secret store and retrieves them as and when it needs them. This could be to populate a Kubernetes (K8s) Secret for an Application Deployment, or when calling a Driver that needs a secret to perform its task.

This may not be what you want. Many organizations want to have full control over their secrets and manage their own store, in their own infrastructure, to maintain their security posture. The Humanitec Operator introduces a new deployment model to make that possible. Let’s go through the different scenarios one by one.

Modes of operation: Legacy mode #

A newly created Humanitec Organization is in Legacy mode at the start. We recommend switching to one of the more secure modes early on to avoid the implications of a later change:

• Direct mode
• GitOps mode

This is how it works without the Humanitec Operator: when using full SaaS, the Humanitec Platform Orchestrator securely stores any secrets an Application needs in a separate, internal secret store. It inserts fully rendered K8s manifests into the target cluster when an Application is deployed, including any necessary K8s Secret manifests. The Platform Orchestrator then retrieves the secrets from its internal secret store to produce these Secret manifests, or when calling Drivers that require a secret.

Modes of operation: Direct mode #

This mode was previously known as “Humanitec Operator mode”.

In Direct mode, the Humanitec Platform Orchestrator has direct access to the target cluster to create Kubernetes objects.

When secrets are stored inside your organization’s infrastructure, the Platform Orchestrator will not have access to them. The manifests deployed into the K8s cluster therefore cannot be rendered entirely by the Platform Orchestrator. Likewise, it cannot call any Driver requiring a secret.

The Platform Orchestrator automatically switches to Direct mode when configured with an external secret store marked primary. See The primary secret store below for details.

It then renders the manifests only partially and generates a set of instructions outlining how to complete the rendering procedure. The target K8s cluster receives this information as a set of custom resources (CRs), based on CRDs that come with the Humanitec Operator installation.

The Humanitec Operator picks up these custom resources and completes the remaining rendering stages. This entails obtaining secrets from the secret store, calling any Drivers required to provision Resources, and creating the final K8s manifests.

Since the Platform Orchestrator can access the target cluster, it can provide you with runtime status information on your Deployments.

You may optionally still allow users of the Platform Orchestrator to write secrets to the secret store using the Orchestrator. See The primary secret store below for details.

In any case, you always need to register the secret store with both the Platform Orchestrator and the Humanitec Operator. The instructions (CRs) the Platform Orchestrator creates reference a specific secret store by its ID which must match a secret store registration with the Humanitec Operator.

Your cluster’s API server does not have to be exposed publicly. The Humanitec Agent can be used in conjunction with Direct mode to securely access a private cluster.

Modes of operation: GitOps mode #

This mode was previously known as “Humanitec Operator with GitOps mode”.

In the Direct mode shown above the Platform Orchestrator still requires access to the API server of the target K8s cluster to manage the custom resources.

If you do not wish to allow that, you need another way to write those custom resources into the cluster. This is where GitOps comes in. Here, the Platform Orchestrator writes the custom resource manifests to a Git repository instead of straight to the cluster, and a GitOps operator (like ArgoCD or Flux) performs the job of pulling those manifests into the cluster itself.

The Humanitec Operator will then again be able to execute the instructions represented by the custom resources just like before.

Speaking purely from the perspective of the Humanitec Operator, there is no difference between Direct mode and GitOps mode. All the Humanitec Operator sees is that someone creates custom resources in the cluster, and it executes the instructions they convey. In Direct mode, that someone is the Platform Orchestrator, whereas with GitOps mode it is the GitOps operator (acting on behalf of the Platform Orchestrator).

You technically enable this mode by switching to Direct mode and creating a Resource Definition for your Kubernetes cluster that uses a specific Driver named k8s-cluster-git on top of that.

Implications of switching from Legacy mode to Direct mode #

Switching from Legacy to Direct mode has some implications to be aware of. We therefore recommend to make the move to Direct mode early on.

Existing secrets in Humanitec secret store #

Any secrets maintained while running the Platform Orchestrator in Legacy mode have been stored in the secret store of the Humanitec SaaS system. Access to these secrets will be lost when you switch to Direct mode.

If you wish to continue maintaining secrets via the Platform Orchestrator, you should re-save the Orchestrator objects containing the secrets. They will then be saved in the new primary secret store.

Multiple secret stores #

While the descriptions above mention only one secret store for brevity, you can configure multiple secret stores for Direct mode or GitOps mode. You may, for example, wish to use separate secret stores for different Applications, different Environments, or both.

You can mix and match different types of secret stores as long as your infrastructure can provide them, e.g. a HashiCorp Vault and a Google Secret Manager if you are running workloads on Google Cloud.

The secret store to use for reading a particular secret is part of its secret reference.

The primary secret store #

The primary secret store is a choice made for the secret store registrations on the Platform Orchestrator side.

At most one secret store can be marked primary. Defining a primary store has to two functions:

1. If users write secret values via the Platform Orchestrator, it will try to write them to the current primary store.
2. The Platform Orchestrator automatically switches to “Humanitec Operator” mode when configured with an external secret store marked primary.

If none of your registered secret stores are the primary store, the Humanitec-hosted secret store will become the primary, and any secret values written via the Platform Orchestrator will be saved to that store.

If you want to allow users to write secrets via the Platform Orchestrator to one of your own secret stores, designate the target secret store as primary and establish write access for the Platform Orchestrator. The individual How-to pages on connecting secret stores show how to do that.

The default secret store #

The default secret store is a choice made on the Humanitec Operator side.

Besides your own secrets, the Humanitec Operator needs to store “Resource cookies” (an operational state) internally in any one secret store. It will use the store marked as default among all secret store configurations for the Operator. There must be exactly one default secret store.

The default secret store can be a store you use for your regular secrets, or a separate store, at your discretion.

The individual How-to pages on connecting secret stores show how to declare a secret store the default.

Secret store access levels #

Depending on the mode of operation, you need to allow the appropriate levels of access for your secret store to the Humanitec Operator and/or the Platform Orchestrator.

In Legacy mode, Humanitec’s own secret store is used. Users of the Platform Orchestrator use the Orchestrator as an interface to write secrets. Secret references cannot be used in Legacy mode as we do not expose the internal structure of the secret store.

In Direct mode or GitOps mode, your secret store - or stores - are used. These are your options regarding access levels:

• The Platform Orchestrator does not need read access to any store.
• The Platform Orchestrator needs write access only to the primary secret store if you wish to enable your users to write secrets through the Platform Orchestrator. Without write access, they’ll get an error if they try.
• Giving the Platform Orchestrator no access is a valid option. In this case, you must use secret references at all times.
• The Humanitec Operator needs read access to all secret stores.
• The Humanitec Operator needs write access only to the default secret store to write Resource cookies.

The Humanitec Operator in a self-hosting scenario #

When you’re self-hosting the Platform Orchestrator, using the Humanitec Operator is optional. The same considerations concerning the reachability of the cluster API server and secret store access apply. The only difference is that they are now evaluated inside the infrastructure you control.

Calling Drivers #

The Humanitec Operator will call Drivers to manage Resources. This might require sending secrets to Drivers as part of the payload. Depending on where those Drivers are hosted, those secrets - while being encrypted in transit - will leave your network. This will apply to any Humanitec-hosted drivers. It excludes the Echo and Template Drivers whose logic is executed inside the Operator itself.

If your security posture restricts you from sending secrets outside your own network, consider using self-hosted Drivers. Contact Humanitec support for details.

Next steps #

See how to install the Humanitec Operator in your infrastructure.