Deploy an Amazon S3 Resource to production

Deploying an Application with an Amazon S3 Resource using Score and the Platform Orchestrator.

In the modern development landscape, seamlessly integrating infrastructure resources with application deployment is paramount. This tutorial offers a step-by-step guide to effortlessly integrate an Amazon S3 bucket — a widely-used storage solution — with your Application deployment.

While this tutorial specifies a Resource Type of s3, you can use any Resource Type that your platform team has created and your Application Workload needs.

Leveraging the capabilities of the Score CLI and the Humanitec platform, you’ll discover how to abstract infrastructure complexities, allowing you to remain focused on your application’s core functionalities.

Introduction

When it comes to connecting your application code to your infrastructure, you don’t want to worry about the details of the infrastructure, instead you want to focus on your application code. With Resources, you can define infrastructure components like databases, storage buckets, and more as Resources that your Workload can consume.

With Score, writing your infrastructure needs is as easy as declaring the resource type as s3, and the platform engineering team will provide the details of the Resource Type.

In this example, you’ll use an Amazon S3 bucket as your Resource Type and connect it to your Workload. We are not going to actually write to the bucket, instead, this example connects to the Resource and injects data (the bucket name) to the Workload.

If you are a platform engineer, see the documentation on how to provision Resource Definitions to enable this approach.

When creating an Amazon S3 Bucket or similar resources, the platform engineering team is responsible for the credentials and access provided to the developers.

Prerequisites

To get started with this tutorial, you’ll need:

The score-humanitec CLI installed locally.
An Application defined in the Orchestrator. If you do not yet have one, create an Application in one minute using the instructions in Applications.
The Orchestrator set up to provision resources of the desired type, e.g. Amazon S3, to two different regions for different Environments. This is a platform team task. The Provision Amazon S3 Buckets tutorial walks you through the process.
At least two Environment Types defined in your Organization, e.g. development and production.
(Optional) A Score file for your application. If present, you can extend your existing Score file as shown in this tutorial. Otherwise, we have provided a standalone example you can use.
(Optional) The AWS CLI installed locally.

Provide your Score file

As a developer, you’ll use Score to deploy your Application Resources to a development Environment to test out your application code, then push to production after testing. By declaring a resource in your Score file and using its output to connect your Workload.

If you have a Score file already, skip this step and proceed to deploying it.

To get started, you’ll need to write a Score file that defines a container and an image. In this example, you’ll use the image type of busybox and write a simple Hello World! command to see it run.

Create a score.yaml file with the following contents:

apiVersion: score.dev/v1b1

metadata:
  name: hello-world

containers:
  hello-world:
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo Hello World!; sleep 5; done"]

This is a simple setup; however the details for your Application are dependent on your Workload needs.

As a developer, you don’t need to worry about deployment mechanisms, Terraform, or Kubernetes. All you need is the abstract definitions of your Workload to deploy to the Platform Orchestrator.

Deploy your Workload to development

Declare your Organization name, Application name, and the target Environment development. Note that the values of the IDs must be all lowercase.

HUMANITEC_ORG=<my-humanitec-org-id>  # The ID of your Organization in the Platform Orchestrator
APP_ID=<my-humanitec-application-id> # The ID of your Application in the Platform Orchestrator

Declare the Humanitec API token. To create a new token, follow these steps:

Sign in to your Humanitec account.
From the left navigation menu, select API Tokens.
Enter a Token ID, and select Generate new token.

Make a note of the token, you may not be able to display it later. Then declare the token variable:

HUMANITEC_TOKEN=<my-humanitec-api-token>

Deploy your Application using score-humanitec, your Score file and the --deploy flag.

score-humanitec delta -f ./score.yaml \
  --org ${HUMANITEC_ORG} \
  --app ${APP_ID} \
  --env development \
  --token ${HUMANITEC_TOKEN} \
  --deploy

If the --env flag is not specified it will deploy to development by default.

The command will generate a Delta Set and deploy it to your Environment. For the example Score file, the output will look like this:

{
  "id": "123456789",
  "metadata": {
    "env_id": "development",
    "name": "Auto-deployment (SCORE)",
    "url": "https://app.humanitec.io/orgs/${HUMANITEC_ORG}/apps/${APP_ID}/envs/development/draft/123456789",
    "created_by": "123456789",
    "created_at": "2023-08-15T09:57:51.295817808Z",
    "last_modified_at": "2023-08-15T09:57:51.295817808Z"
  },
  "modules": {
    "add": {
      "hello-world": {
        "externals": null,
        "profile": "humanitec/default-module",
        "spec": {
          "containers": {
            "hello-world": {
              "args": [
                "-c",
                "while true; do echo Hello World!; sleep 5; done"
              ],
              "command": [
                "/bin/sh"
              ],
              "id": "hello-world",
              "image": "busybox"
            }
          }
        }
      }
    }
  }
}

Confirm the deployment using one of the following methods.

To confirm your deployment, review the status of your Workload in the Web UI.

Login to your Organization and select the Application name and Environment that you just deployed to.
1. In this example, the Environment should be set to development.
You should see an Active Deployment running in your Application, you should see the Workloads match the Workload declared in your Score file.
1. In this example, the name of the Workload matches what was declared in the metadata section of your Score file, hello-world.

To confirm your deployment, review the status of your Workload by the return value of the API.

curl -s https://api.humanitec.io/orgs/${HUMANITEC_ORG}/apps/${APP_ID}/envs/development/deploys \
  -H "Authorization: Bearer ${HUMANITEC_TOKEN}" \
  | jq '. | .[0:1]'

This command will use the API token to query the Platform Orchestrator for the latest deployment and output a list of objects that define your recent deployments in descending order of created at date.

The output uses jq to filter and slice the output to show the latest result, which is the first item in the array.

You should see an output similar to the following:

[
  {
    "set_id": "123456789-woP7I90U",
    "delta_id": "123456789-woP7I90U",
    "comment": "Auto-deployment (SCORE)",
    "id": "123456789",
    "env_id": "development",
    "created_at": "2023-08-14T14:09:39.764574Z",
    "created_by": "da8f8f94-4f70-4f1f-b3cd-fea74fc73d33",
    "status": "succeeded",
    "status_changed_at": "2023-08-14T14:09:56.384731Z",
    "from_id": "",
    "export_status": "",
    "export_file": "1Jys1xqUxHZ57SWRhO1MRmJyQGBnQlS9.zip"
  }
]

In this example, the status is succeeded and the env_id is development.

Now that you’ve confirmed you can deploy to your Environment, add a Resource to your Workload.

Add your Resource

Let’s add the Amazon S3 Bucket to the Score file, and then reference the resource in the Workload.

Update your score.yaml file to look like this:

apiVersion: score.dev/v1b1

metadata:
  name: hello-world

containers:
  hello-world:
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo Your bucket name: $$MY_BUCKET_NAME; sleep 5; done"]
    variables:
      MY_BUCKET_NAME: ${resources.my-bucket.bucket}

resources:
  my-bucket:
    type: s3

Let’s go through the changes one by one. If you are using your own Score file, add the code snippets to it at the corresponding locations.

resources:
  my-bucket:
    type: s3

Here, we added the resources section to the Score file.

It declares the my-bucket Resource. The name is ours to choose.
The Resource has a Resource type of s3.
Note that no further configuration is required for the bucket on the part of the developer. This has been taken care of by the Resource Definitions for the Resource Type s3 created by the platform team.

If your own Score file already has a resources section, just add the last two lines to it.

containers:
  ...
    variables:
      MY_BUCKET_NAME: ${resources.my-bucket.bucket}

Here, we reference the my-bucket resource in a container.

We set the value of the environment variable MY_BUCKET_NAME to the output bucket of the resource my-bucket.
At runtime, this will be the name of the bucket that was actually provisioned.
Available inputs and outputs are defined per Resource Type. bucket is one output of the Resource Type s3.

...
args: ["-c", "while true; do echo Your bucket name: $$MY_BUCKET_NAME; sleep 5; done"]
...

Here, we use the environment variable MY_BUCKET_NAME to generate container log output.

If you are using your own Score file, defining the args: attribute in this fashion be not be feasible.

If there is no easy way to expose the value of the MY_BUCKET_NAME variable in your setting, consider adding the entire hello-world container as an additional container to your Score file temporarily.

Deploy Workload and Resource to development

You can now deploy your updated Workload definition including the Amazon S3 bucket to your development Environment, using the same command as before.

score-humanitec delta -f ./score.yaml \
  --org ${HUMANITEC_ORG} \
  --app ${APP_ID}\
  --env development \
  --token ${HUMANITEC_TOKEN} \
  --deploy

For the example Score file, the output will look like this. Note that we now see the my-bucket element in the externals section, and the modifications to the container.

{
  "id": "1234567890",
  "metadata": {
    "env_id": "development",
    "name": "Auto-deployment (SCORE)",
    "url": "https://app.humanitec.io/orgs/${HUMANITEC_ORG}/apps/hello-world/envs/development/draft/03474eb10378a70fa6cde08304f7e255ac19f7f1",
    "created_by": "s-d3e70a0e-8b7d-40e9-a666-74548b7e06e0",
    "created_at": "2023-09-14T16:01:30.057233711Z",
    "last_modified_at": "2023-09-14T16:01:30.057233711Z"
  },
  "modules": {
    "add": {
      "hello-world": {
        "externals": {
          "my-bucket": {
            "type": "s3"
          }
        },
        "profile": "humanitec/default-module",
        "spec": {
          "containers": {
            "hello-world": {
              "args": [
                "-c",
                "while true; do echo Your bucket name: $$MY_BUCKET_NAME; sleep 5; done"
              ],
              "command": [
                "/bin/sh"
              ],
              "id": "hello-world",
              "image": "busybox",
              "variables": {
                "MY_BUCKET_NAME": "${externals.my-bucket.bucket}"
              }
            }
          }
        }
      }
    }
  }
}

Check the results in the Orchestrator UI:

From the left navigation menu, select Applcations.
Select the development environment of the Application your are using.
Select the active deployment.
Select the workload used on your Score file. For the example file, it is hello-world.
Under Resource Dependencies, you should now see the S3 bucket my-bucket. Its Values show the bucket name and the region. The region should be the one defined for development environments as per the Resource Definition for s3 type Resources.
Now select the container that received the environment variable. It is hello-world for the example Score file.
Check the log output of your container. For the example Score file, or if added this to your own container, the logs should now export the bucket name.
The container also shows the new variable MY_BUCKET_NAME.

(Optional) Verify the S3 bucket actually exists in the proper location via the aws CLI. Ues the bucket ID you saw in the UI and/or the log output:

aws s3api get-bucket-location --bucket <your-bucket-id>

You should see this output showing the location, which may be different from this one:

{
    "LocationConstraint": "eu-west-1"
}

Deploy Workload and Resource to production

Now that you’ve tested your resource is created in the development environment, push your changes to a production environment.

As the developer, you would follow your best practices, including testing, before deploying to production.

You can check available Environments for your Application like this:

curl -s "https://api.humanitec.io/orgs/${HUMANITEC_ORG}/apps/${APP_ID}/envs" \
  -H "Authorization: Bearer $HUMANITEC_TOKEN" \
  | jq -r ".[] | .id"

To quickly create a new Environment to deploy to, follow the instructions to Manage Environments. E.g. to create a new production Environment based on development via the CLI, use:

humctl create env /orgs/${HUMANITEC_ORG}/apps/${APP_ID}/envs/production \
  --type=production \
  --name=production \
  --from=/orgs/${HUMANITEC_ORG}/apps/${APP_ID}/envs/development \
  --token=${HUMANITEC_TOKEN}

Now perform the deployment using your Score file. Use the --env flag and choose the environment to deploy to. In this example, set your --env flag to production.

score-humanitec delta -f ./score.yaml \
  --org ${HUMANITEC_ORG} \
  --app ${APP_ID}\
  --env production \
  --token ${HUMANITEC_TOKEN} \
  --deploy

Check the log output of your Workload to see the container running.

You should see something similar to the following log output.

{
  "id": "1234567890",
  "metadata": {
    "env_id": "production",
    "name": "Auto-deployment (SCORE)",
    "url": "https://app.humanitec.io/orgs/${HUMANITEC_ORG}/apps/hello-world/envs/production/draft/1234567890",
    "created_by": "s-a3e60a0e-8a53-72e9-a266-74548b7e06a0",
    "created_at": "2023-09-15T08:18:48.466170354Z",
    "last_modified_at": "2023-09-15T08:18:48.466170354Z"
  },
  "modules": {
    "add": {
      "hello-world": {
        "externals": {
          "my-bucket": {
            "type": "s3"
          }
        },
        "profile": "humanitec/default-module",
        "spec": {
          "containers": {
            "hello-world": {
              "args": [
                "-c",
                "while true; do echo Your bucket name: $$MY_BUCKET_NAME; sleep 5; done"
              ],
              "command": [
                "/bin/sh"
              ],
              "id": "hello-world",
              "image": "busybox",
              "variables": {
                "MY_BUCKET_NAME": "${externals.my-bucket.bucket}"
              }
            }
          }
        }
      }
    }
  }
}

Repeat the steps from the development deployment to verify that a new S3 bucket was created in the proper region and its name is being injected into the container as a variable.

Troubleshooting

If you receive a Driver authentication error, your Cloud Account credentials might be set incorrectly. See Provision Amazon S3 Buckets for developers for more information.

Recap

You have successfully added a new Resource to your Workload and injected it into your container.

You have made the Platform Orchestrator provision not just different instances, but also different flavors of that resource for two environments of your Application.

Note that you used the same adjusted Score file for both environment deployments. Dynamic Configuration Management seperates the concerns of the developer (“I need an S3 bucket”) from the concerns of the platform engineers (“For development, we use buckets on that region/of that flavor, and for production, we use that other region/that other flavor”).

Although this specific example used an Amazon S3 bucket as your resource, you can provision any Resource Type that your platform team has created, which is discoverable in the Resource Management tab on the Web UI.

Deploy an Amazon S3 Resource to production

Introduction #

Prerequisites #

Provide your Score file #

Deploy your Workload to development #

Add your Resource #

Deploy Workload and Resource to development #

Deploy Workload and Resource to production #

Troubleshooting #

Recap #