Structure and integration points

Understanding the different planes of the reference architecture #

When McKinsey originally published the reference architecture they proposed five different planes that since then describe the different parts of a modern Internal Developer Platform (IDP). Let’s analyze them one by one.

Developer Control Plane #

This plane is the primary configuration layer and interaction point for the platform users. It harbors the following components:

• A Version Control System. GitHub is a prominent example, but this can be any system that contains two types of repositories:
  • Application Source Code
  • Platform Source Code, e.g. using Terraform
• Workload specifications. The reference architecture uses Score.
• A Portal for developers to interact with. It can be the Humanitec Platform Orchestrator UI, but you might also use Backstage or any other Portal on the market.

Following the design principle “everything as code” both app and platform source code are stored in Git. The platform source code represents the configuration of the platform and is maintained using the IaC framework Terraform. Terraform is used for both managing the Humanitec Resource Definitions using the Humanitec Terraform provider, and for configuring the different automation systems. The primary interaction method for developers is designed to be code-driven using the Workload spec Score, to describe the Workload and dependent Resources in abstract terms. Git integrates with the IDE, the CI pipeline, and the portal using the GitHub API.

The portal layer offers a user interface on top of all platform capabilities that acts like a single pane of glass, including shortcuts to scaffolding new services, metrics, service catalogs, and additional self-service actions. The portal integrates with the VCS through its API, using plugins where available, and equally to the Platform Orchestrator. It might also be pulling additional data directly from the CI pipelines or project management systems like JIRA.

Integration and Delivery Plane #

This plane is about building and storing the image, creating app and infra configs from the abstractions provided by the developers, and deploying the final state. It’s where the domains of developers and platform engineers meet.

This plane usually contains four different tools:

• A CI pipeline. It can be GitHub Actions or any CI tooling on the market.
• The image registry holding your container images. Again, this can be any registry on the market.
• An orchestrator which in our example is the Humanitec Platform Orchestrator.
• The CD system which can be the Platform Orchestrator’s deployment pipeline capabilities, an external system triggered by the Orchestrator using a webhook, or a setup in tandem with GitOps operators like ArgoCD.

The job of the CI pipeline is to build and test the developed code. It then pushes new images to the image registry.

At the end of its run, the pipeline may also notifies the Platform Orchestrator about the new image and submits the Workload specification (Score) to trigger a deployment using the newly generated image. An example pipeline in the form of a GitHub workflow is available here. The CI pipeline passes deployment metadata to the Platform Orchestrator, including image path, tags, and a deployment delta (basically the converted Score file in a format the Platform Orchestrator can interpret).

The registry itself does not have a direct integration with the Platform Orchestrator. You may choose to have the Platform Orchestrator provide registry credentials to CI/CD systems, or to workloads running in your Kubernetes cluster by injecting registry credential secrets. See the integration options for details.

Monitoring and Logging Plane #

The integration of monitoring and logging systems varies greatly depending on the system. Some require the use of sidecar containers to be included in all your Workloads, or adding labels and annotations to your Application configurations, or some other method. This plane however is not a focus of the reference architecture that you’re trialing today.

Security Plane #

The security plane of the reference architecture is focused on the secrets management system. The secrets manager stores configuration information such as database passwords, API keys, or TLS certificates needed by an Application at runtime. It allows the Platform Orchestrator to reference the secrets and inject them into the Workloads dynamically. You can learn more on secrets management and integration with other secrets management here.

The reference architecture sample implementations use the secrets store attached to the Humanitec SaaS system.

Resource Plane #

This plane is where the actual infrastructure exists including clusters, databases, storage or DNS services. The configuration of the Resources is managed by the Platform Orchestrator which dynamically creates app and infrastructure configurations with every deployment and creates, updates, or deletes dependent Resources as required.

Check the list of specific Resources created for each of the reference architecture implementations in the respective README.

Your platform interfaces #

The reference architecture allows the user or the organization to choose their preferred interface with your platform, which will offer varying degrees of flexibility and abstraction. How these abstractions are resolved depends on the context and the rules set by the platform team. We cover this in more detail going forward in this learning path. Our reference architecture comes with the following interface options:

Abstract code-based Workload specification (Score): code-based, abstract configuration of Workload and its dependencies. This specification prevents breaks in the workflows and can facilitate self-service actions such as changing Environment variables or removing/adding Resources. A Score specification may also be used for promoting an Application to the next Environment, spinning up a PR Environment, as well as adding meta-data to your deployment. It does this by adding or removing parts of the code and varying deployment metadata which can then be interpreted by the Platform Orchestrator, which executes the request.

Specific IaC and Resource Definitions (Terraform): code-based, specific Resource configurations and templates. These are usually maintained by a central Ops/platform team and are ideally applied across Applications to achieve standardization by design. Only if you need a very specific configuration for a Resource would you modify those files.

Orchestrator API: The Orchestrator API acts as the de facto platform API that allows you to pull data and intel about what’s going on inside the platform, and automate and trigger actions. The API covers 100% of the Platform Orchestrator capabilities and can be used for actions like spinning up Environments, rolling back, pulling information, running diffs between deployments, and much more.

Orchestrator CLI: The API and most commonly used combinations of requests are also packaged as a CLI.

Portal: The reference architecture uses the Humanitec Portal UI. You can also build your own user interface using a base product like Backstage, working the Platform Orchestrator API behind the scenes. The UI also allows you to perform most self-service actions and automation such as rolling back, running diffs, or adding Applications and Resources.