Deploy an Amazon S3 Resource to production

Follow step-by-step instructions to deploy an Application with an Amazon S3 Resource using Score and the Platform Orchestrator.

Introduction

In the modern development landscape, seamlessly integrating infrastructure resources with application deployment is paramount. This tutorial offers a step-by-step guide to effortlessly integrate an Amazon S3 bucket — a widely-used storage solution — with your Application deployment.

While this tutorial specifies a Resource Type of s3, you can use any Resource Type that your platform team has created and your Application Workload needs.

Leveraging the capabilities of the Score CLI and the Humanitec platform, you’ll discover how to abstract infrastructure complexities, allowing you to remain focused on your application’s core functionalities. For example, with Score, writing your infrastructure needs is as easy as declaring the Resource Type as s3 (the platform engineering team will provide the details of the Resource Type). And with Resources, you can define infrastructure components like databases, storage buckets, and more, as Resources that your Workload can consume.

In this example, you’ll use an Amazon S3 bucket as your Resource Type and connect it to your Workload. We’re not going to actually write to the bucket, instead, this example connects to the Resource and injects data (the bucket name) to the Workload. When creating an Amazon S3 Bucket or similar resources, the platform engineering team is responsible for the credentials and access provided to the developers. If you’re a platform engineer, see the documentation on how to provision Resource Definitions to enable this approach.

Prerequisites

To get started with this tutorial, you’ll need:

  • The humctl Humanitec CLI installed locally.
  • An Application defined in the Orchestrator. If you don’t yet have one, create an Application in just one minute using the instructions in Applications.
  • The Platform Orchestrator set up to provision Resources of the desired type, e.g. Amazon S3, to two different regions for different Environments (this is a platform team task). The Provision Amazon S3 Buckets tutorial will walk your platform engineers through the process.
  • At least two Environment Types defined in your Organization, e.g. development and production.
  • (Optional) A Score file for your Application. If present, you can extend your existing Score file as shown in this tutorial. The outcome of the Scaffolding tutorial is a good starting point. Otherwise, we’ve provided a standalone example you can use.
  • (Optional) The AWS CLI installed locally.

Provide your Score file

As a developer, you’ll use Score to deploy your Application Resources to a development Environment to test out your Application code, then push to production after testing. By declaring a resource in your Score file and using its output to connect your Workload.

If you have a Score file already, skip this step and proceed to deploying it.

To get started, you’ll need to write a Score file that defines a container and an image. In this example, you’ll use the image type of busybox and write a simple Hello World! command to see it run.

Create a score.yaml file with the following contents:

apiVersion: score.dev/v1b1

metadata:
  name: hello-world

containers:
  hello-world:
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo Hello World!; sleep 5; done"]

Although this is a simple setup, the details for your Application are dependent on your Workload needs.

As a developer, you don’t need to worry about deployment mechanisms, Terraform, or Kubernetes. All you need is the abstract definitions of your Workload to deploy to the Platform Orchestrator.

Deploy your Workload to development

Declare your Organization name, Application name, and the target Environment development. Note that the values of the IDs must be all lowercase.

HUMANITEC_ORG=<my-humanitec-org-id>  # The ID of your Organization in the Platform Orchestrator
APP_ID=<my-humanitec-application-id> # The ID of your Application in the Platform Orchestrator

Declare the Humanitec API token. To create a new token, follow these steps:

  • Sign in to your Humanitec account.
  • From the left navigation menu, select API Tokens.
  • Enter a Token ID, and select Generate new token.

Make a note of the token, you may not be able to display it later. Then declare the token variable:

HUMANITEC_TOKEN=<my-humanitec-api-token>

Deploy your Application using humctl and your Score file.

humctl score deploy -f ./score.yaml \
  --org ${HUMANITEC_ORG} \
  --app ${APP_ID} \
  --env development \
  --token ${HUMANITEC_TOKEN}

If the --env flag is not specified it will deploy to development by default.

The command will generate a Delta Set and deploy it to your Environment. For the example Score file, the output will look like this:

{
  "id": "123456789",
  "metadata": {
    "env_id": "development",
    "name": "Auto-deployment (SCORE)",
    "url": "https://app.humanitec.io/orgs/${HUMANITEC_ORG}/apps/${APP_ID}/envs/development/draft/123456789",
    "created_by": "123456789",
    "created_at": "2023-08-15T09:57:51.295817808Z",
    "last_modified_at": "2023-08-15T09:57:51.295817808Z"
  },
  "modules": {
    "add": {
      "hello-world": {
        "externals": null,
        "profile": "humanitec/default-module",
        "spec": {
          "containers": {
            "hello-world": {
              "args": [
                "-c",
                "while true; do echo Hello World!; sleep 5; done"
              ],
              "command": [
                "/bin/sh"
              ],
              "id": "hello-world",
              "image": "busybox"
            }
          }
        }
      }
    }
  }
}

Confirm the deployment using one of the following methods.

  1. Login to your Organization and select the Application name and Environment that you just deployed to.
    • In this example, the Environment should be set to development.
  2. You should see an Active Deployment running in your Application, and you should see the Workloads match the Workload declared in your Score file.
    • In this example, the name of the Workload matches what was declared in the metadata section of your Score file, hello-world.

humctl get deployment --app ${APP_ID} --env development +0 -o json

You should see an output similar to the following:

{
  "apiVersion": "entity.humanitec.io/v1b1",
  "entity": {
    "comment": "Auto-deployment (SCORE)",
    "delta_id": "970bbdf0bf3a6b9a7b3933b6c91fdfe1ecfaed56",
    "set_id": "o2NbcHeBFwU1U-tPiv16NTxuFWsQAjLPartmD3Y1YhU",
    "value_set_version_id": "e52a8f11-3e36-45d2-bd0e-92045c70c017"
  },
  "kind": "Deployment",
  "metadata": {
    "app_id": "my-app",
    "env_id": "development",
    "id": "179b89653f2deb27",
    "org_id": "my-org"
  },
  "status": {
    "created_at": "2030-11-27T16:59:07.660545Z",
    "created_by": "s-bae60a0e-8a53-40e9-a666-74548b7606e0",
    "export_file": "XTtJC06waNzltFicqFFX5a32HTsL2J2l.zip",
    "export_status": "",
    "from_id": "",
    "status": "succeeded",
    "status_changed_at": "2030-11-27T16:59:15.195117Z"
  }
}


curl -s https://api.humanitec.io/orgs/${HUMANITEC_ORG}/apps/${APP_ID}/envs/development/deploys \
  -H "Authorization: Bearer ${HUMANITEC_TOKEN}" \
  | jq '. | .[0:1]'

The output uses jq to filter and slice the output to show the latest result, which is the first item in the array.

You should see an output similar to the following:

[
  {
    "set_id": "123456789-woP7I90U",
    "delta_id": "123456789-woP7I90U",
    "comment": "Auto-deployment (SCORE)",
    "id": "123456789",
    "env_id": "development",
    "created_at": "2030-08-14T14:09:39.764574Z",
    "created_by": "da8f8f94-4f70-4f1f-b3cd-fea74fc73d33",
    "status": "succeeded",
    "status_changed_at": "2030-08-14T14:09:56.384731Z",
    "from_id": "",
    "export_status": "",
    "export_file": "1Jys1xqUxHZ57SWRhO1MRmJyQGBnQlS9.zip",
    "pipeline": {
      "id": "default",
      "run_id": "c19a3be8-144f-4e3a-803b-a72bf2398a4c",
      "job_id": "deploy",
      "step_index": 1
    }
  }
]

The CLI and API commands will use the API token to query the Platform Orchestrator for the latest deployment, and output a list of objects that define your recent deployments in descending order of the “created at” date.

In this example, the status is succeeded and the env_id is development.

Now that you’ve confirmed you can deploy to your Environment, add a Resource to your Workload.

Add your Resource

Let’s add the Amazon S3 Bucket to the Score file, and then reference the resource in the Workload.

Update your score.yaml file to look like this:

apiVersion: score.dev/v1b1

metadata:
  name: hello-world

containers:
  hello-world:
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo Your bucket name: $$MY_BUCKET_NAME; sleep 5; done"]
    variables:
      MY_BUCKET_NAME: ${resources.my-bucket.bucket}

resources:
  my-bucket:
    type: s3

Let’s go through the changes one by one. If you’re using your own Score file, add the code snippets to it at the corresponding locations.

resources:
  my-bucket:
    type: s3

Here, we added the resources section to the Score file.

  • It declares the my-bucket Resource. The name is ours to choose.
  • The Resource has a Resource type of s3.
  • Note that no further configuration is required for the bucket on the part of the developer. This has been taken care of by the Resource Definitions for the Resource Type s3 created by the platform team.

If your own Score file already has a resources section, just add the last two lines to it.

containers:
  ...
    variables:
      MY_BUCKET_NAME: ${resources.my-bucket.bucket}

Here, we reference the my-bucket resource in a container.

  • We set the value of the environment variable MY_BUCKET_NAME to the output bucket of the resource my-bucket.
  • At runtime, this will be the name of the bucket that was actually provisioned.
  • Available inputs and outputs are defined per Resource Type. bucket is one output of the Resource Type s3.
...
args: ["-c", "while true; do echo Your bucket name: $$MY_BUCKET_NAME; sleep 5; done"]
...

Here, we use the environment variable MY_BUCKET_NAME to generate container log output.

Deploy Workload and Resource to development

You can now deploy your updated Workload definition including the Amazon S3 bucket to your development Environment, using the same command as before.

humctl score deploy -f ./score.yaml \
  --org ${HUMANITEC_ORG} \
  --app ${APP_ID}\
  --env development \
  --token ${HUMANITEC_TOKEN}

For the example Score file, the output will look like this. Note that we now see the my-bucket element in the externals section, and the modifications to the container.

{
  "id": "1234567890",
  "metadata": {
    "env_id": "development",
    "name": "Auto-deployment (SCORE)",
    "url": "https://app.humanitec.io/orgs/${HUMANITEC_ORG}/apps/hello-world/envs/development/draft/03474eb10378a70fa6cde08304f7e255ac19f7f1",
    "created_by": "s-d3e70a0e-8b7d-40e9-a666-74548b7e06e0",
    "created_at": "2023-09-14T16:01:30.057233711Z",
    "last_modified_at": "2023-09-14T16:01:30.057233711Z"
  },
  "modules": {
    "add": {
      "hello-world": {
        "externals": {
          "my-bucket": {
            "type": "s3"
          }
        },
        "profile": "humanitec/default-module",
        "spec": {
          "containers": {
            "hello-world": {
              "args": [
                "-c",
                "while true; do echo Your bucket name: $$MY_BUCKET_NAME; sleep 5; done"
              ],
              "command": [
                "/bin/sh"
              ],
              "id": "hello-world",
              "image": "busybox",
              "variables": {
                "MY_BUCKET_NAME": "${externals.my-bucket.bucket}"
              }
            }
          }
        }
      }
    }
  }
}

Check the results in the Platform Orchestrator UI:

  • From the left navigation menu, select Applications.
  • Select the development environment of the Application you’re using.
  • Select the active deployment.
  • Select the Workload used in your Score file. For the example file, it is hello-world.
  • Under Resource Dependencies, you should now see the S3 bucket my-bucket. Its Values show the bucket name and the region. The region should be the one defined for development environments as per the Resource Definition for s3 type Resources.
  • Now select the container that received the environment variable. It is hello-world for the example Score file.
  • Check the log output of your container. For the example Score file, or if added to your own container, the logs should now export the bucket name.
  • The container also shows the new variable MY_BUCKET_NAME.

(Optional) Verify the S3 bucket actually exists in the proper location via the aws CLI. Use the bucket ID you saw in the UI and/or the log output:

aws s3api get-bucket-location --bucket <your-bucket-id>

You should see this output showing the location, which may be different from this one:

{
    "LocationConstraint": "eu-west-1"
}

Deploy Workload and Resource to production

Now that you’ve tested your resource is created in the development environment, push your changes to a production Environment.

As the developer, you would follow your best practices, including testing, before deploying to production.

You can check available Environments for your Application like this:

humctl get environment --app ${APP_ID}


curl -s "https://api.humanitec.io/orgs/${HUMANITEC_ORG}/apps/${APP_ID}/envs" \
  -H "Authorization: Bearer $HUMANITEC_TOKEN" \
  | jq -r ".[] | .id"

To quickly create a new Environment to deploy to, follow the instructions to Manage Environments. E.g. to create a new production Environment based on development via the CLI, use:

humctl create env production --org ${HUMANITEC_ORG} --app ${APP_ID} --env production \
  --type production \
  --name production \
  --from development \
  --token ${HUMANITEC_TOKEN}

Now perform the deployment using your Score file. Use the --env flag and choose the environment to deploy to. In this example, set your --env flag to production.

humctl score deploy -f ./score.yaml \
  --org ${HUMANITEC_ORG} \
  --app ${APP_ID}\
  --env production \
  --token ${HUMANITEC_TOKEN}

Check the log output of your Workload to see the container running.

You should see something similar to the following log output.

{
  "id": "1234567890",
  "metadata": {
    "env_id": "production",
    "name": "Auto-deployment (SCORE)",
    "url": "https://app.humanitec.io/orgs/${HUMANITEC_ORG}/apps/hello-world/envs/production/draft/1234567890",
    "created_by": "s-a3e60a0e-8a53-72e9-a266-74548b7e06a0",
    "created_at": "2023-09-15T08:18:48.466170354Z",
    "last_modified_at": "2023-09-15T08:18:48.466170354Z"
  },
  "modules": {
    "add": {
      "hello-world": {
        "externals": {
          "my-bucket": {
            "type": "s3"
          }
        },
        "profile": "humanitec/default-module",
        "spec": {
          "containers": {
            "hello-world": {
              "args": [
                "-c",
                "while true; do echo Your bucket name: $$MY_BUCKET_NAME; sleep 5; done"
              ],
              "command": [
                "/bin/sh"
              ],
              "id": "hello-world",
              "image": "busybox",
              "variables": {
                "MY_BUCKET_NAME": "${externals.my-bucket.bucket}"
              }
            }
          }
        }
      }
    }
  }
}

Repeat the steps from the development deployment to verify that a new S3 bucket was created in the proper region and that its name is being injected into the container as a variable.

Troubleshooting

If you receive a Driver authentication error, your Cloud Account credentials might be set incorrectly. See Provision Amazon S3 Buckets for developers for more information.

Recap

Congrats! You’ve successfully completed the tutorial on how to deploy an Amazon S3 Resource to production. In this tutorial you added a new Resource to your Workload and injected it into your container. You made the Platform Orchestrator provision not just different instances, but also different flavors of that resource for two Environments of your Application.

It’s important to remember that you used the same adjusted Score file for both environment deployments. Dynamic Configuration Management separates the concerns of the developer (“I need an S3 bucket”) from the concerns of the platform engineers (“For development, we use buckets on that region/of that flavor, and for production, we use that other region/that other flavor”).

Although this specific example used an Amazon S3 bucket as your Resource, you can provision any Resource Type that your platform team has created, which is discoverable in the Resource Management tab on the Web UI.

Next steps

Now that you’re familiar with deploying an Amazon S3 Resource to production, learn how to perform daily activities like debug, rollback, diffs, and logs in this tutorial.

For more information on handling resources, see our Overview and Resource Definitions page.

Provision Amazon S3 Buckets

Set up the reference architecture in your cloud