The need to manage secrets
The Humanitec Operator is motivated by the need to manage secrets in an Internal Developer Platform (IDP).
Secrets must be handled in various places when using the Humanitec Platform Orchestrator, e.g. as shared values and secrets for Applications, for certain types of Resource Definitions, or when configuring Drivers.
By default, the Humanitec SaaS solution uses its own secret store. Whenever it needs to manage secrets for later use, it saves them to that secret store and retrieves them as and when it needs them. This could be to populate a Kubernetes (K8s) Secret for an Application Deployment, or when calling a Driver that needs a secret to perform its task.
This may not be what you want. Many organizations want to have full control over their secrets and manage their own store, in their own infrastructure, to maintain their security posture. The Humanitec Operator introduces a new deployment model to make that possible. Let’s go through the different scenarios one by one.
Modes of operation: Direct Cluster
This is how it works without the Humanitec Operator: when using full SaaS, the Humanitec Platform Orchestrator securely stores any secrets an Application needs in a separate, internal secret store. It inserts fully rendered K8s manifests into the target cluster when an Application is deployed, including any necessary K8s Secret manifests. The Platform Orchestrator then retrieves the secrets from its internal secret store to produce these Secret manifests, or when calling Drivers that require a secret.
Modes of operation: Humanitec Operator
When secrets are stored inside your organization’s infrastructure, the Platform Orchestrator will not have access to them. The manifests deployed into the K8s cluster there cannot be rendered entirely by the Platform Orchestrator. Likewise, it cannot call any Driver requiring a secret.
The Platform Orchestrator automatically switches to Humanitec Operator mode when configured with an external secret store marked
primary. See The primary secret store below for details.
It then renders the manifests only partially and generates a set of instructions outlining how to complete the rendering procedure. The target K8s cluster receives this information as a set of custom resources (CRs), based on CRDs that come with the Humanitec Operator installation.
The Humanitec Operator picks up these custom resources and completes the remaining rendering stages. This entails obtaining secrets from the secret store, calling any Drivers required to provision Resources, and creating the final K8s manifests.
You may optionally still allow users of the Platform Orchestrator to write secrets to the secret store using the Orchestrator just like in Direct Cluster mode. See The primary secret store below for details.
In any case, you always need to register the secret store with both the Platform Orchestrator and the Humanitec Operator. The instructions (CRs) the Platform Orchestrator creates reference a specific secret store by its ID which must match a secret store registration with the Humanitec Operator.
Any secrets maintained while running the Platform Orchestrator in Direct Cluster mode have been stored in the secret store of the Humanitec SaaS system. Access to these secrets will be lost when you switch to Humanitec Operator mode.
If you wish to continue maintaining secrets via the Platform Orchestrator, you should re-save the Orchestrator objects containing the secrets. They will then be saved in the new
primary secret store.
Modes of operation: Humanitec Operator with GitOps
In the above “Humanitec Operator” mode the Platform Orchestrator still requires access to the API server of the target K8s Cluster to manage the custom resources.
If you do not wish to allow that, you need another way to write those custom resources into the cluster. This is where GitOps comes in. Here, the Platform Orchestrator writes the custom resource manifests to a Git repository instead of straight to the cluster, and a GitOps operator (like ArgoCD or Flux) performs the job of pulling those manifests into the cluster itself.
The Humanitec Operator will then again be able to execute the instructions represented by the custom resources just like before.
You technically enable this mode by switching to Humanitec Operator mode and creating a Resource Definition for your Kubernetes cluster that uses a specific Driver named
k8s-cluster-git on top of that.
Multiple secret stores
While the descriptions above mention only one secret store for brevity, you can configure multiple secret stores for Humanitec Operator or GitOps mode. You may, for example, wish to use separate secret stores for different Applications, different Environments, or both.
You can mix and match different types of secret stores as long as your infrastructure can provide them, e.g. a HashiCorp Vault and a Google Secret Manager if you are running workloads on Google Cloud.
The secret store to use for reading a particular secret is part of its secret reference.
The primary secret store
primary secret store is a choice made for the secret store registrations on the Platform Orchestrator side.
At most one secret store can be marked
primary. Defining a
primary store has to two functions:
- If users write secret values via the Platform Orchestrator, it will try to write them to the current
- The Platform Orchestrator automatically switches to “Humanitec Operator” mode when configured with an external secret store marked
primarystore, the Humanitec-hosted secret store will become the
primary, and any secret values written via the Platform Orchestrator will be saved to that store.
If you want to allow users to write secrets via the Platform Orchestrator to one of your own secret stores, designate the target secret store as
primary and establish write access for the Platform Orchestrator. The individual How-to pages on connecting secret stores show how to do that.
The default secret store
default secret store is a choice made on the Humanitec Operator side.
Besides your own secrets, the Humanitec Operator needs to store “Resource cookies” (an operational state) internally in any one secret store. It will use the store marked as
default among all secret store configurations for the Operator. There must be exactly one
default secret store.
default secret store can be a store you use for your regular secrets, or a separate store, at your discretion.
The individual How-to pages on connecting secret stores show how to declare a secret store the
Secret store access levels
Depending on the mode of operation, you need to allow the appropriate levels of access for your secret store to the Humanitec Operator and/or the Platform Orchestrator.
In Direct Cluster mode, Humanitec’s own secret store is used. Users of the Platform Orchestrator use the Orchestrator as an interface to write secrets. Secret references cannot be used in Direct Cluster mode as we do not expose the internal structure of the secret store.
In Humanitec Operator or GitOps mode, your secret store - or stores - are used. These are your options regarding access levels:
- The Platform Orchestrator does not need
readaccess to any store.
- The Platform Orchestrator needs
writeaccess only to the
primarysecret store if you wish to enable your users to write secrets through the Platform Orchestrator. Without
writeaccess, they’ll get an error if they try.
- Giving the Platform Orchestrator no access is a valid option. In this case, you must use secret references at all times.
- The Humanitec Operator needs
readaccess to all secret stores.
- The Humanitec Operator needs
writeaccess only to the
defaultsecret store to write Resource cookies.
The Humanitec Operator in a self-hosting scenario
When you’re self-hosting the Platform Orchestrator, using the Humanitec Operator is optional. The same considerations concerning the reachability of the cluster API server and secret store access apply. The only difference is that they are now evaluated inside the infrastructure you control.
The Humanitec Operator will call Drivers to manage Resources. This might require sending secrets to Drivers as part of the payload. Depending on where those Drivers are hosted, those secrets - while being encrypted in transit - will leave your network. This will apply to any Humanitec-hosted drivers. It excludes the Echo and Template Drivers whose logic is executed inside the Operator itself.
If your security posture restricts you from sending secrets outside your own network, consider using self-hosted Drivers. Contact Humanitec support for details.