Overview
Introduction
Most Organizations have a working CI/CD setup in place already when starting to use the Humanitec products.
While there are some common patterns, they variety of real-world CI/CD processes is vast, as is the range of tools being used along the way.
The Humanitec products do not impose any particular workflow, or the use of any particular tool. Chances are you do not need to change your CI/CD processes a great deal. Rather, there are some natural extension points when using Score and the Platform Orchestrator that we have laid out in this guide.
Most importantly, your developer workflows will undergo little change. Once set up properly, application developers will likely enjoy more automation and more autonomy than before while working the software development lifecycle the familiar way.
Some code references on this page use GitHub as a sample CI/CD system. The patterns work just the same on any other tooling, including Bitbucket, CircleCI, GitLab, Jenkins, Codefresh, and more, and may easily be transferred.
Add Score to your source code and deploy continuously
The “CI” portion is probably the most uniform part of CI/CD workflows across organizations. Commonly, a git push
to a workload’s source repository triggers a pipeline with a series of build and test steps, and finally creates a new container image version that is pushed to a container registry.
If you are not just building, but also deploying continuously, then a CD pipeline is usually triggered to deploy the new image to the target environment, or there is a unified CI/CD pipeline. A common pattern will look like this:
flowchart LR
subgraph ciPipeline[CI pipeline]
direction LR
build((Build the\nworkload)) --> tests((Unit tests /\n smoke tests))
tests --> push((Push image\n to registry))
end
developer(fa:fa-person Developer ) -->|git push| ciPipeline
subgraph cdPipeline[CD pipeline]
deploy((Deploy))
end
subgraph infra[Infrastructure]
direction TB
deployedWorkload(Deployed Workload)
end
ciPipeline --> cdPipeline --> infra
You can now switch to deploying your workload along with the required resources via the Platform Orchestrator.
Perform these modifications:
- Add a Score file to the workload repository specifying the workload and its required resources, using the image
.
placeholder:
apiVersion: score.dev/v1b1
metadata:
name: my-app
...
containers:
main:
image: . # Set by pipeline
...
resources:
...
- Add these steps to the end of your CI pipeline, replacing the current CD step(s) if any:
- Install the
humctl
CLI- For GitHub, you can use the humanitec/setup-cli-action
- For any other runner, perform the CLI installation commands
- Use the CLI and the Score file to deploy your workload via the Platform Orchestrator using the
humctl score deploy
command, specifying the image version built earlier in the pipeline via the--image
flag - See this GitHub workflow for a complete example. The approach works likewise for any other pipeline runner
The exact steps of your pipeline up to the new steps will vary, but the entire flow will now look similar to this:
flowchart LR
subgraph ciCdPipeline[CI/CD pipeline]
direction LR
build((Build the\nworkload)) --> tests((Unit tests /\n smoke tests))
tests --> push((Push image\n to registry))
push --> installCLI((Install\nhumctl CLI))
installCLI --> deploy((humctl\nscore deploy))
end
subgraph infra[Infrastructure]
direction TB
deployedWorkload(Deployed Workload)
subgraph resources[Resources]
resource1[ ]
resource2[/ /]
end
deployedWorkload -.-> resources
end
developer(fa:fa-person Developer ) -->|git push| build
deploy --> platformOrchestrator{Platform\nOrchestrator}
platformOrchestrator --> infra
class installCLI,deploy,platformOrchestrator,deployedWorkload,resource1,resource2 highlight
class resources nested
Now, every git push
will trigger an updated deployment of your workload. And because the Score file is part of the code, developers control the workload specification including any required resources, which will also be provisioned by the Orchestrator in the way your platform team has configured it.
You can apply this approach to any branching pattern you are using, and deploy to a different Application Environment for each branch. The next section proposes an extremely useful implementation of this pattern.
Use ephemeral environments
Using this pattern is optional, but highly recommended as a next step after implementing your base CI integration.
An “ephemeral environment” is a temporary, on-the-fly instance of an application. It exists solely for the duration of a specific task, such as testing a feature or bug fix via a pull request. Once the task is done, the environment is torn down.
Ephemeral environments are sometimes called “dynamic previews”.
Ephemeral environments are not technically different from other Environments in the Platform Orchestrator. There is no technical flag marking an Environment as “ephemeral”. Their ephemeral nature is a human notion, signifying the limited time span of their existence.
As a developer, you thus get a full-fledged, isolated environment for testing your code, including its interactions with surrounding infrastructure, with zero effort.
To see how to implement ephemeral environments, refer to these guides:
- Ephemeral environments - topic deep dive and a quick guide specifically for users of GitHub
- Deploy ephemeral environments - extended tutorial with in-depth guidance, largely tooling-agnostic
Familiar developer workflow
Attaching the Humanitec tooling to your CI/CD setup leaves the familiar developer workflow unchanged. As a developer, you will continue to perform local development, create branches and PRs, push into branches, and finally merge them. The Humanitec tooling extends the workflow by automated features improving developers’ everday experience.
flowchart TB
subgraph devWorkflow
direction LR
deleteEphemeralEnv --> localDev
localDev(Local Development) --> creatPR(Create PR) --> createEphemeralEnv(Create\nephemeral\nenvironment) --> deployWorkloadArtefact(Deploy\nworkload\nartefact) --> testAndEvaluate(Develop,\ntest and\nevaluate) --> merge(Merge) --> deleteEphemeralEnv(Delete\nephemeral\nenvironment)
testAndEvaluate -->|git push| deployWorkloadArtefact
end
subgraph key[Key]
direction LR
humanitecExtensions(Humanitec\nextensions) ~~~ humanitecExternel(External to Humanitec)
end
devWorkflow ~~~ key
class humanitecExtensions,createEphemeralEnv,deployWorkloadArtefact,deleteEphemeralEnv highlight
class devWorkflow invisible
CD integration points
While CI setups are often fairly similar, the downstream CD setup varies greatly between organizations.
A common theme is an environment promotion from development to production, with any number of environments in between. An environment promotion is human-driven most of the time, often with an approval step involved.
When you switched to Platform Orchestrator deployments for your CI/CD development loop already and are using ephemeral environments, your setup may now look similar to this:
flowchart TB
subgraph ciLoop[CI/CD development loop using ephemeral environments]
direction LR
subgraph ciInnerLoop[CI/CD pipeline]
direction LR
deployWorkloadArtefact --> buildAndPush
buildAndPush(Build and\npush) --> deployWorkloadArtefact(humctl\nscore\ndeploy)
end
ciLoopPlatformOrchestrator{Platform\nOrchestrator}
subgraph devEnv[Development environment]
devEnvDeployedWorkload(Deployed\nworkload) -.-> devEnvResources(Resources)
end
subgraph ephemeralEnv[Ephemeral Environment]
ephemeralEnvDeployedWorkload(Deployed\nworkload) -.-> ephemeralEnvResources(Resources)
end
ciInnerLoop -->|--env=development| ciLoopPlatformOrchestrator --> devEnv
ciInnerLoop -->|--env=ephemeral| ciLoopPlatformOrchestrator --> ephemeralEnv
end
subgraph envPromotionStaging[CD staging]
direction LR
subgraph stagingInnerLoop[CD pipeline]
direction LR
promoteStaging(Promote) --> stagingDeployWorkloadArtefact(Deploy)
end
subgraph stagingEnv[Staging Environment]
stagingEnvDeployedWorkload(Deployed\nworkload)
end
stagingInnerLoop --> stagingCDtooling(CD tooling) --> stagingEnv
end
ciLoop -->|Environment promotion staging| envPromotionStaging
subgraph envPromotionProduction[CD Production]
direction LR
subgraph ProductionInnerLoop[CD pipeline]
direction LR
promoteProduction(Promote) --> approve(Approve) --> ProductionDeployWorkloadArtefact(Deploy)
end
subgraph ProductionEnv[Production Environment]
ProductionEnvDeployedWorkload(Deployed\nworkload)
end
ProductionInnerLoop --> ProductionCDtooling(CD tooling) --> ProductionEnv
end
envPromotionStaging -->|Environment promotion production| envPromotionProduction
class ciLoopPlatformOrchestrator highlight
The next step is to switch to the Platform Orchestrator for the downstream CD deployments.
Much like for the CI/CD development loop, using the Humanitec products does not impose any particular downstream CD approach. You can continue executing your environment promotion as you designed it.
Follow these steps for your downstream CD integration:
- Mirror your environments in the Platform Orchestrator as Application Environments
- Ensure that your platform team have prepared the provisioning of all the resources your workload requires by setting up the appropriate Resource Definitions
- Use the same Score file across all environments. If you need to make overrides, treat it as “development”, and apply overrides for downstream environments via the Platform Orchestrator like in this example
- Switch to deploying via Score and the Platform Orchestrator in your existing CD tooling using the
humctl
CLI, which you can install into your CD pipeline as shown above
You do not have to switch over to Platform Orchestrator deployment for all environments at once. Advance through your enviornment progression one by one to validate your setup, then proceed to the next.
Introducing the artefact endpoint
In many cases, using the humctl score deploy
command in your CD tooling will be all you need.
This command deploys to one specific Application Environment only by design and requires both Application (--app
) and Environment (--env
) to be named.
flowchart LR
subgraph pipeline[CD pipeline]
deploy(humctl\nscore\ndeploy)
end
subgraph env[Target environment]
devEnvDeployedWorkload(Deployed\nworkload) -.-> devEnvResources(Resources)
end
pipeline -->|--app=my-app\n--env=target-env| env
This also means that
- the CD tooling needs to know which Application Environment(s) to deploy to
- it must make separate calls for each target Environment
While this works well for simple and linear cases, you may have more complex setups where an updated workload needs to be deployed to several Environments or even several Applications. A network of interdependent pipelines can become hard to maintain.
In such scenarios, there is another option you can use which shifts control over to the Platform Orchestrator and can greatly simplify your CD setup. Basically, instead of submitting this chain of commands out of your CD tooling:
Deploy the workload defined by this Score file to Environment A of App1, and then to B of App1, and then to C of App2 …
you can submit this single command:
Deploy the workload defined by this Score file to every relevant Application and Environment
where “relevant” is decided by the Platform Orchestrator based on the workload’s attributes (name, source reference) provided with the command.
This option uses the CLI command humctl create artefact-version
(reference). It triggers all Humanitec Artefact Automation Pipelines in any Application whose triggers match the workload’s attributes. They will all be triggered asynchronously.
This approach shifts the responsibility for determining deployment targets from the CI/CD system into the Applications themselves and introduces a loose coupling between the two which can help untangle a network of CD pipeline dependencies.
This sample setup shows how a single command out of a CD pipeline triggers the deployment of an updated workload into two separate Applications, where App2
deploys into two regional Environments. The CD pipeline does not require any knowledge about each Application’s deployment pattern, and should it change, does not need to be adjusted.
---
title: Using the Artefact endpoint for a complex deployment scenario
---
flowchart LR
subgraph cdPipeline[CD pipeline]
deployWorkloadArtefact(humctl create\nartefact-version\n--type=workload)
end
subgraph app2[App2]
subgraph productionApp2EnvA[Production Environment Region A]
productionApp2EnvADeployedWorkload(Deployed\nworkload) -.-> productionApp2EnvAResources(Resources)
end
subgraph productionApp2EnvB[Production Environment Region B]
productionApp2EnvBDeployedWorkload(Deployed\nworkload) -.-> productionApp2EnvBResources(Resources)
end
artefactAutomationPipelineApp2(Artefact Automation Pipeline) -->|deploy| productionApp2EnvA
artefactAutomationPipelineApp2 -->|deploy| productionApp2EnvB
end
subgraph app1[App1]
subgraph productionApp1EnvA[Production Environment]
productionApp1EnvADeployedWorkload(Deployed\nworkload) -.-> productionApp1EnvAResources(Resources)
end
artefactAutomationPipelineApp1(Artefact Automation Pipeline) -->|deploy| productionApp1EnvA
end
cdPipeline -->|--name my-org/my-service\n--ref=refs/heads/main| platformOrchestrator{Platform\nOrchestrator}
platformOrchestrator -->|trigger| artefactAutomationPipelineApp1
platformOrchestrator -->|trigger| artefactAutomationPipelineApp2
humctl score deploy
command also triggers a Humanitec Pipeline of type Deployment Pipelines. There is a default one that is created and used if you do not set up your own. Unless you wish to customize the standard deployment, it is a detail you can safely ignore.Again, using the humctl create artefact-version
command (the “Artefact endpoint”) in place of humctl score deploy
(the “Deployment endpoint”) is optional and depends on your use case, and how you wish to distribute resposibilities among the teams involved. You may even use both approaches depending on the complexity of each deployment stage.
The table below summarizes the differences between the two variants.
Deployment endpoint | Artefact endpoint | |
---|---|---|
CLI command | humctl score deploy -f score.yaml --app my-app --env target-env |
humctl create artefact-version --type workload --spec score.yaml --name my-org/my-service --refs refs/heads/main --version 1.2.7 |
Reference | CLI Score integration | Workload artefacts |
Target Applications(s) | Must be known to pipeline/CD tooling Passed in as --app parameterExactly one |
Determined by Platform Orchestrator based on workload attributes (--name , --ref )Can be more than one |
Target Environment(s) | Must be known to pipeline/CD tooling Passed in as --env parameterExactly one |
Determined by Platform Orchestrator based on workload attributes (--name , --ref )Can be more than one |
Use of Humanitec Pipelines | Works with any Deployment Pipeline including the default one | Requires an Artefact Automation Pipeline |
Batching | By your pipeline/CD tooling | By Platform Orchestrator |
Approvals | In your pipeline/CD tooling | In your pipeline/CD tooling OR in Platform Orchestrator Pipeline (see approvals action) |
Responsibility | Centralized (CI/CD tooling) | Decentralized (Applications) |
Example | Deploy and run automated tests | Promote between Environments after approval |
The diagram below shows a sample setup where the ealier stages “development”, “ephemeral” and “staging” are handled using the Deployment endpoint, and the production stage using the Artefact endpoint. In particular, the production deployment is targeting two Environments representing different regions. Note that the diagram is omitting test steps, approvals, and other variations for brevity.
---
title: Sample CI/CD flow
---
flowchart TB
subgraph ciLoop[CI/CD base loop using ephemeral environments]
direction LR
subgraph ciInnerLoop[CI/CD inner loop]
deployWorkloadArtefact --> buildAndPush
buildAndPush(Build and\npush) --> deployWorkloadArtefact(humctl\nscore\ndeploy)
end
ciLoopPlatformOrchestrator{Platform\nOrchestrator}
subgraph devEnv[Development environment]
devEnvDeployedWorkload(Deployed\nworkload) -.-> devEnvResources(Resources)
end
subgraph ephemeralEnv[Ephemeral Environment]
ephemeralEnvDeployedWorkload(Deployed\nworkload) -.-> ephemeralEnvResources(Resources)
end
ciInnerLoop -->|--app=App1\n--env=development| ciLoopPlatformOrchestrator -->|Default\nDeployment\nPipeline| devEnv
ciInnerLoop -->|--app=App1\n--env=ephemeral| ciLoopPlatformOrchestrator -->|Default\nDeployment\nPipeline| ephemeralEnv
end
subgraph envPromotionStaging[CD staging]
direction LR
subgraph stagingInnerLoop[Staging loop]
direction TB
promoteStaging(Promote) --> stagingDeployWorkloadArtefact(humctl\nscore\ndeploy)
end
subgraph stagingEnv[Staging Environment]
stagingEnvDeployedWorkload(Deployed\nworkload) -.-> stagingEnvResources(Resources)
end
stagingInnerLoop -->|--app=App1\n--env=staging| stagingPlatformOrchestrator{Platform\nOrchestrator} -->|Default\nDeployment\nPipeline| stagingEnv
end
ciLoop -->|Environment promotion staging| envPromotionStaging
subgraph envPromotionProduction[CD production]
direction LR
subgraph productionInnerLoop[Production loop]
direction TB
promoteProduction(Promote) --> approve(Approve) --> productionDeployWorkloadArtefact(humctl create\nartefact-version\n--type=workload)
end
subgraph productionEnvA[Production Environment Region A]
productionEnvADeployedWorkload(Deployed\nworkload) -.-> productionEnvAResources(Resources)
end
subgraph productionEnvB[Production Environment Region B]
productionEnvBDeployedWorkload(Deployed\nworkload) -.-> productionEnvBResources(Resources)
end
productionInnerLoop -->|--name my-org/my-service\n--ref=refs/heads/main| productionPlatformOrchestrator{Platform\nOrchestrator} -->|Artefact\nAutomation\nPipeline| pipelineDummy( ) --> productionEnvA
pipelineDummy --> productionEnvB
end
envPromotionStaging -->|Environment promotion production| envPromotionProduction
class ciInnerLoop,devEnv,ephemeralEnv,stagingEnv,productionEnvA,productionEnvB nested
class stagingInnerLoop,productionInnerLoop,pipelineDummy invisible
class ciLoopPlatformOrchestrator,stagingPlatformOrchestrator,productionPlatformOrchestrator highlight
Recap
While every organization’s CI/CD setup is different, there are a number of clear patterns for integrating the Humanitec tooling into it.
The CI part can most often be augmented in a standard fashion by adding automated deployments via the Orchestrator. Offering ephemeral environments is a valuable next step.
The downstream CD portion can be extended gradually. You can add Orchestrator-based deployments to your setup while keeping workflows unchanged, and optionally shift more responsibility to the Humanitec tooling at your discretion in further steps.