Audit logs

Overview

Audit and compliance are important concepts for larger companies, especially in regulated industries such as finance, utilities and the public sector. The ability to know what happened, when, and who is responsible is the essence of an audit. Having a comprehensive audit allows companies to show they are fulfilling compliance regimes by acting as proof that actions have taken place as intended.

Organizations need the ability to view all activities that impact their environment. Specifically, the action, what it acted on, who carried it out, and exactly when it happened.

The Platform Orchestrator’s audit logs help you meet those requirements. They may be queried via the Platform Orchestrator API for your Organization, and processed according to your needs.

Available data

The audit logs are aligned with the Platform Orchestrator API. They represent requests on all entity types defined in the API, such as Applications, Resource Definitions, or Deployments.

An audit log entry is a JSON structure like this:

{
  "at": "2030-01-12T11:47:39.508355Z",
  "org_id": "my-org",
  "request_method": "POST",
  "request_path": "/orgs/my-org/resources/defs",
  "response_status": 200,
  "user_id": "s-d35b0d0e-8a53-40e9-a329-64e48b7e92e0"
}

Refer to the API Definition of audit logs for details on the individual parts. These general rules apply to all logs:

  • Only requests that change data are logged, i.e. successful create, modify, or delete requests. This excludes all read requests.
  • The audit log stores only authenticated and authorized requests.
  • The request methods are generally used like this:
    • POST: creating an entity. The request target can have the character of a functional endpoint, e.g. creating a pipeline run.
    • PUT: fully updating an existing entity.
    • PATCH: partially updating an existing entity.
    • DELETE: deleting an entity.
  • The audit log does not store the request bodies (payloads).
  • No IP addresses are logged.

There is a delay of up to five minutes between a request and its visibility in the audit log.

Data retention

The audit log keeps data for at least 365 days after its creation.

Deleting audit log data is not possible.

Prerequisites

To query audit logs, you need to have the following prerequisites:

  • Administrator permissions in the target Organization.
  • The CLI installed.
  • These environment variables set:
export HUMANITEC_TOKEN=<api-token>
export HUMANITEC_ORG=<humanitec-org-id>

To see how to obtain an API token, go here.

Query audit logs

You can query audit logs via the Platform Orchestrator API. The base command is:

humctl api get "/orgs/${HUMANITEC_ORG}/audit-logs"

The API returns an array of JSON structures from newest to oldest and may return large quantities of data, depending on the size of the Organization. So it’s best to use the “from” and “to” query parameters to limit the returned data to the time window of interest:

humctl api get \
  "/orgs/${HUMANITEC_ORG}/audit-logs?from=2030-12-29T00:00:00Z&to=2030-12-30T00:00:00Z"

Advanced results filtering can be achieved using standard shell tooling like jq. For common use cases head to the below examples section.

Working with pagination

To ensure reliable performance, the audit log API is paginated. Each request will return a maximum number of entries as defined via the per_page parameter. The default is 100.

humctl api get "/orgs/${HUMANITEC_ORG}/audit-logs?per_page=500"

The “link” response header contains the link which can be used to retrieve the next page.

Use this script to retrieve all paginated data. It will create a file page*.json per page in the current directory.

#!/usr/bin/env bash
set -euo pipefail

# Adjust this link to add additional filter parameters the API provides
link='</orgs/'${HUMANITEC_ORG}'/audit-logs?per_page=500>;rel="next"'

i=0
while [[ $(echo ${link} | wc -c) -gt 10 ]]; do
  link=$(echo ${link} | egrep -o '/orgs/[^>]+')
  echo "Fetching page${i}.json"
	link=$(curl -s https://api.humanitec.io${link} -w '%header{link}' -H "Authorization: Bearer ${HUMANITEC_TOKEN}" -o page${i}.json)
  i=$((i+1))
done

The script requires a curl version >= 7.83 to support the -w '%header{}' feature.

You can then combine the pages together using the jq utility into a final results.json file:

cat page*.json | jq -s 'add | sort_by(.at)' > results.json

Continue to filter the results as needed. The following examples cover some common use cases.

Examples

The following examples omit pagination for brevity.

If you’re using the above scripting example and have created a results file, you can pipe the file content to the same jq queries for further filtering.

  • All logs after a cutoff date:

    humctl api get "/orgs/${HUMANITEC_ORG}/audit-logs?from=2030-01-11T00:00:00Z"

  • All logs in a certain time window:

    humctl api get "/orgs/${HUMANITEC_ORG}/audit-logs?from=2030-12-29T00:00:00Z&to=2030-12-30T00:00:00Z"

  • All entity deletions after a cutoff date:

    humctl api get "/orgs/${HUMANITEC_ORG}/audit-logs?from=2030-01-11T00:00:00Z" | \
  jq '. | map( . | select ( .request_method == "DELETE" ) )'

  • All creations of new Resource Definitions:

    humctl api get "/orgs/${HUMANITEC_ORG}/audit-logs" | \
  jq '. | map( . | select(.request_method == "POST" ) | select( .request_path | startswith ("/orgs/'${HUMANITEC_ORG}'/resources/defs") ) )'

    Note: an update of an existing Resource Definition would be a PUT request.

  • All Deployments into the Application “my-app”:

    humctl api get "/orgs/${HUMANITEC_ORG}/audit-logs" | \
  jq '. | map( . | select( .request_method == "POST" ) | select( .request_path | startswith("/orgs/'${HUMANITEC_ORG}'/apps/my-app/envs/development/deploys") ))'

  • All deletions of Applications:

    humctl api get "/orgs/${HUMANITEC_ORG}/audit-logs" |  \
  jq '. | map( . | select( .request_method == "DELETE" ) | select( .request_path | test("^/orgs/'${HUMANITEC_ORG}'/apps/[^/]+$") ))'

    Note: A startswith selection on /orgs/${HUMANITEC_ORG}/apps is not sufficient to select only the Application entities themselves. It would also match subordinate entities such as Application Values at /orgs/${HUMANITEC_ORG}/apps/my-app/values. The example therefore uses a test selection with a regular expression containing the line ending “$”.

  • Updates and deletion of the Resource Definition “my-resdef”:

    humctl api get "/orgs/${HUMANITEC_ORG}/audit-logs" | \
  jq '. | map( . | select( .request_method | test("PUT|DELETE") ) | select( .request_path | test("^/orgs/'${HUMANITEC_ORG}'/resources/defs/my-resdef$") ))'

    Note: Similar to the previous example, we need to use a regular expression specifying the line ending to exclude subordinate entities to the Resource Definition, such as Matching Criteria at /orgs/${HUMANITEC_ORG}/resources/defs/my-resdef/criteria.

  • All deletions of secret stores made by user “s-d35b0d0e-8a53-40e9-a329-64e48b7e92e0”:

    humctl api get "/orgs/${HUMANITEC_ORG}/audit-logs" | \
  jq '. | map( . | select( .request_method == "DELETE" ) | select( .user_id == "s-d35b0d0e-8a53-40e9-a329-64e48b7e92e0" ) | select ( .request_path | startswith("/orgs/'${HUMANITEC_ORG}'/secretstores") ))'

Next steps

Explore the Platform Orchestrator API documentation to find the entities you wish to audit.

Secret references