Repository: kestra-io/kestra
Issue: Newlines break secrets
Labels: bug
quick-win

### Explain the bug



Base64 encoding can introduce newline characters, e.g.,

```sh
$ echo -n "y0Dqk9Zg17IuRbYrWtEJFFCtILvmdKeHEefr1IEYpFAQq6Crtb4wVVZ3hEVcGsRgmGbKWfQn8ZgPwUo5ocn0B5VRONCJwwvS" | base64
eTBEcWs5WmcxN0l1UmJZcld0RUpGRkN0SUx2bWRLZUhFZWZyMUlFWXBGQVFxNkNydGI0d1ZWWjNo
RVZjR3NSZ21HYktXZlFuOFpnUHdVbzVvY24wQjVWUk9OQ0p3d3ZT
```

These decode fine in the command line

```sh
$ export foo="$(echo -n 'y0Dqk9Zg17IuRbYrWtEJFFCtILvmdKeHEefr1IEYpFAQq6Crtb4wVVZ3hEVcGsRgmGbKWfQn8ZgPwUo5ocn0B5VRONCJwwvS' | base64)"
$ echo "$foo" | base64 -d
y0Dqk9Zg17IuRbYrWtEJFFCtILvmdKeHEefr1IEYpFAQq6Crtb4wVVZ3hEVcGsRgmGbKWfQn8ZgPwUo5ocn0B5VRONCJwwvS
```

But Kestra is not able to decode newline-containing encoded secrets.

As workarounds, `-w 0` option can be appended to `base64` command during encoding, or the newline characters can be manually removed from the encoded base64 string.

Kestra should probably not break when newlines are encountered in encoded secrets.


### Environment Information

- Kestra Version: 0.12.1
- Operating System and Java Version (if not using Kestra Docker image):


Issue comments:
Can you give us more information about the issue.

- How do you set this env variable that includes a new line? Newline on itself is cumbersome in the shell.
- What is the error in Kestra? Do you have any logs?

Pull request review comments:

Patch that resolved the issue:
diff --git a/build.gradle b/build.gradle
index f838393f30..cb2d0d623f 100644
--- a/build.gradle
+++ b/build.gradle
@@ -192,6 +192,7 @@ subprojects {
         systemProperty 'user.country', 'US'
 
         environment 'SECRET_MY_SECRET', "{\"secretKey\":\"secretValue\"}".bytes.encodeBase64().toString()
+        environment 'SECRET_NEW_LINE', "cGFzc3dvcmR2ZXJ5dmVyeXZleXJsb25ncGFzc3dvcmR2ZXJ5dmVyeXZleXJsb25ncGFzc3dvcmR2\nZXJ5dmVyeXZleXJsb25ncGFzc3dvcmR2ZXJ5dmVyeXZleXJsb25ncGFzc3dvcmR2ZXJ5dmVyeXZl\neXJsb25n"
         environment 'SECRET_WEBHOOK_KEY', "secretKey".bytes.encodeBase64().toString()
         environment 'SECRET_NON_B64_SECRET', "some secret value"
         environment 'KESTRA_TEST1', "true"
diff --git a/core/src/main/java/io/kestra/core/secret/SecretService.java b/core/src/main/java/io/kestra/core/secret/SecretService.java
index dc57bb3754..a852f4cc1a 100644
--- a/core/src/main/java/io/kestra/core/secret/SecretService.java
+++ b/core/src/main/java/io/kestra/core/secret/SecretService.java
@@ -23,7 +23,8 @@ private void postConstruct() {
             .filter(entry -> entry.getKey().startsWith(SECRET_PREFIX))
             .<Map.Entry<String, String>>mapMulti((entry, consumer) -> {
                 try {
-                    consumer.accept(Map.entry(entry.getKey(), new String(Base64.getDecoder().decode(entry.getValue()))));
+                    String value = entry.getValue().replaceAll("\\R", "");
+                    consumer.accept(Map.entry(entry.getKey(), new String(Base64.getDecoder().decode(value))));
                 } catch (Exception e) {
                     log.error("Could not decode secret '{}', make sure it is Base64-encoded: {}", entry.getKey(), e.getMessage());
                 }


Test patch:
diff --git a/core/src/test/java/io/kestra/core/secret/SecretFunctionTest.java b/core/src/test/java/io/kestra/core/secret/SecretFunctionTest.java
index 406f4c67d4..398bf3d745 100644
--- a/core/src/test/java/io/kestra/core/secret/SecretFunctionTest.java
+++ b/core/src/test/java/io/kestra/core/secret/SecretFunctionTest.java
@@ -26,9 +26,11 @@ public class SecretFunctionTest extends AbstractMemoryRunnerTest {
 
     @Test
     @EnabledIfEnvironmentVariable(named = "SECRET_MY_SECRET", matches = ".*")
+    @EnabledIfEnvironmentVariable(named = "SECRET_NEW_LINE", matches = ".*")
     void getSecret() throws TimeoutException {
         Execution execution = runnerUtils.runOne(null, "io.kestra.tests", "secrets");
         assertThat(execution.getTaskRunList().get(0).getOutputs().get("value"), is("secretValue"));
+        assertThat(execution.getTaskRunList().get(1).getOutputs().get("value"), is("passwordveryveryveyrlongpasswordveryveryveyrlongpasswordveryveryveyrlongpasswordveryveryveyrlongpasswordveryveryveyrlong"));
     }
 
     @Test
diff --git a/core/src/test/resources/flows/valids/secrets.yaml b/core/src/test/resources/flows/valids/secrets.yaml
index 2f5108bd35..0bbe91b092 100644
--- a/core/src/test/resources/flows/valids/secrets.yaml
+++ b/core/src/test/resources/flows/valids/secrets.yaml
@@ -4,4 +4,7 @@ namespace: io.kestra.tests
 tasks:
   - id: get-secret
     type: io.kestra.core.tasks.debugs.Return
-    format: "{{json(secret('my_secret')).secretKey}}"
\ No newline at end of file
+    format: "{{json(secret('my_secret')).secretKey}}"
+  - id: get-multiline-secret
+    type: io.kestra.core.tasks.debugs.Return
+    format: "{{json(secret('new_line'))}}"
\ No newline at end of file