Mask multiple lines text in GitHub Actions Workflow
Tags: githubactions
Published: 2024-05-26 13:36:10

---
title: Mask multiple lines text in GitHub Actions Workflow
published: true
description: 
tags: [GitHubActions]
# cover_image: https://direct_url_to_image.jpg
# Use a ratio of 100:42 for best results.
# published_at: 2024-05-26 13:24 +0000
---

This article is a translation of an article I originally wrote in Japanese, translated into English using ChatGPT with some modifications:

* [GitHub Actions の Workflow 内で複数行の文字列をマスクする](https://blog.yuyat.jp/post/mask-multiple-lines-text-in-github-actions-workflow/)

---

## tl; dr

Although it might seem like a bit of bad practice, the content of `$multiple_lines_text` can be masked with the following one-liner:

```yaml
- run: |
    echo "::add-mask::$(echo "$multiple_lines_text" | sed ':a;N;$!ba;s/%/%25/g' | sed ':a;N;$!ba;s/\r/%0D/g' | sed ':a;N;$!ba;s/\n/%0A/g')"
```

## `add-mask` Command in GitHub Actions

GitHub Actions has a feature called workflow commands.

* [Workflow commands for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions)

The `add-mask` command is used to mask specific strings in subsequent outputs. For example, by executing the following command, the string "Hello, World!" will be masked as "***" in subsequent outputs.

```
echo "::add-mask::Hello, World!"
```

Values retrieved from secrets can be masked this way, but using the `add-mask` command allows dynamically specifying strings to be masked.

Additionally, this command is implemented in the NPM package [@actions/core](https://github.com/actions/toolkit/tree/main/packages/core). By using `core.setSecret('Hello, World!');` in JavaScript code, the same masking can be achieved.

The `setSecret` function essentially builds and outputs the `::add-mask::~~~` string to the standard output, so the mechanism is identical.

## Issues When Masking Multiple Lines

If you try to mask a string like `Foo\nBar\nBaz` in a shell script without any considerations, it would look like this:

```
::add-mask::Foo
Bar
Baz
```

The command breaks due to the line breaks, and only "Foo" gets masked. This can lead to unwanted side effects like the string "Foo Fighters" being masked as "*** Fighters".

The workflow commands documentation briefly mentions handling multiple strings:

* [Multiline strings](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#multiline-strings)

However, these techniques are meant for inputting multiline environment variables or output values into `$GITHUB_ENV` or `$GITHUB_OUTPUT`, and based on my tests, they don’t work with workflow commands.

## Examining the `core.setSecret` Code

By examining the `core.setSecret` function in @actions/core, it appears that the values to be masked are escaped using a function called `escapeData`.

https://github.com/actions/toolkit/blob/d1df13e178816d69d96bdc5c753b36a66ad03728/packages/core/src/command.ts#L80-L85

```ts
function escapeData(s: any): string {
  return toCommandValue(s)
    .replace(/%/g, '%25')
    .replace(/\r/g, '%0D')
    .replace(/\n/g, '%0A')
}
```

The `toCommandValue` function simply returns the string as-is if it’s already a string, so it can be ignored.

https://github.com/actions/toolkit/blob/d1df13e178816d69d96bdc5c753b36a66ad03728/packages/core/src/utils.ts#L11-L18

Then, the `replace` method replaces `%` with `%25`, `\r` with `%0D`, and `\n` with `%0A`.

By executing a command like this, `Foo\nBar\nBaz` gets masked correctly:

```sh
echo "::add-mask::Foo%0ABar%0ABaz"
```

After asking ChatGPT about how to perform these replacements in a shell script, I arrived at the following conclusion:

```sh
echo "::add-mask::$(echo "$multiple_lines_text" | sed ':a;N;$!ba;s/%/%25/g' | sed ':a;N;$!ba;s/\r/%0D/g' | sed ':a;N;$!ba;s/\n/%0A/g')"
```

While the `s/%/%25/g` part is clear, the preceding `:a;N;$!ba;` part is obscure. I understand from ChatGPT that it’s necessary for handling line breaks, but I can’t explain it in detail here.

## When Is This Necessary?

This article is written for those wondering how to mask multiline strings, regardless of the purpose. For me, there was a specific use case: safely storing and using a GitHub App's Private Key.

To achieve this, I believe the following steps are necessary:

1. Encode the Private Key in Base64 and store it in AWS Secrets Manager.
2. Use OIDC for authentication with AWS.
3. Retrieve the secret in the workflow, decode it from Base64 to get the Private Key.

Assuming AWS usage, the steps can be adapted similarly for Google Cloud using Secret Manager, etc. (not verified).

This can be implemented in a workflow as follows:

```yaml
steps:
  - name: Configure AWS credentials
    id: aws-credentials
    uses: aws-actions/configure-aws-credentials@v4
    with:
      role-to-assume: arn:aws:iam::012345678901:role/role-name
      aws-region: ap-northeast-1

  - name: Retrieve secret from AWS Secrets Manager
    id: aws-secrets
    run: |
      secrets=$(aws secretsmanager get-secret-value --secret-id secret-name --query SecretString --output text)
      gh_app_private_key="$(echo "$secrets" | jq .GH_APP_PRIVATE_KEY_BASE64 -r | base64 -d)"
      echo "::add-mask::$(echo "$gh_app_private_key" | sed ':a;N;$!ba;s/%/%25/g' | sed ':a;N;$!ba;s/\r/%0D/g' | sed ':a;N;$!ba;s/\n/%0A/g')"
      echo "gh-app-private-key<<__EOF__"$'
'"$gh_app_private_key"$'
'__EOF__ >> "$GITHUB_OUTPUT"

  - uses: actions/create-github-app-token@v1
    id: app-token
    with:
      app-id: ${{ vars.GH_APP_ID }}
      private-key: ${{ steps.aws-secrets.outputs.gh-app-private-key }}
```

Further details are explained in the Q&A format below.

### Why Not Store It in Organization or Repository Secrets?

Because it is considered unsafe.

With some specific conditions, the secret's value can be exposed if someone can create a Pull Request and place a GitHub Actions workflow file in the repository.

For more details, refer to this presentation from a recent event. (company blog)

* [AWS知見共有会でTerraformのCI/CDパイプラインのセキュリティ等について発表してきました + GitHub新機能Push rulesについて](https://tech.layerx.co.jp/entry/scalable-and-secure-infrastructure-as-code-pipeline-for-a-compound-startup) (in Japanese)

### Why Do You Need to Mask the Private Key?

Without masking, the Private Key passed as input to `actions/create-github-app-token@v1` can be viewed in the GitHub Actions UI.

### Why Encode the Private Key in Base64 for AWS Secrets Manager?

It’s not mandatory but simplifies storing it without line breaks.

While AWS Secrets Manager can store secrets with line breaks, the key/value mode in the Management Console does not handle line breaks well. Plaintext mode allows entering line breaks, but it’s cumbersome. Therefore, encoding it in Base64 for storage without line breaks is more convenient.