Repository: Azure/WALinuxAgent
Pull request #2811 (2023-04-25 14:44:12)

Redact access tokens from extension's output
Redact URIs that look like SAS tokens when capturing the stdout/stderr of extensions.

codecov[bot]: ## [Codecov](https://codecov.io/gh/Azure/WALinuxAgent/pull/2811?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Azure) Report
> Merging [#2811](https://codecov.io/gh/Azure/WALinuxAgent/pull/2811?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Azure) (60cf7c1) into [develop](https://codecov.io/gh/Azure/WALinuxAgent/commit/4ba20de52653286f535dd617ba2486fc6809120c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Azure) (4ba20de) will **increase** coverage by `0.01%`.
> The diff coverage is `100.00%`.

```diff
@@             Coverage Diff             @@
##           develop    #2811      +/-   ##
===========================================
+ Coverage    72.01%   72.02%   +0.01%     
===========================================
  Files          104      104              
  Lines        15870    15874       +4     
  Branches      2273     2273              
===========================================
+ Hits         11429    11434       +5     
  Misses        3916     3916              
+ Partials       525      524       -1     
```


| [Impacted Files](https://codecov.io/gh/Azure/WALinuxAgent/pull/2811?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Azure) | Coverage Δ | |
|---|---|---|
| [...urelinuxagent/common/utils/extensionprocessutil.py](https://codecov.io/gh/Azure/WALinuxAgent/pull/2811?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Azure#diff-YXp1cmVsaW51eGFnZW50L2NvbW1vbi91dGlscy9leHRlbnNpb25wcm9jZXNzdXRpbC5weQ==) | `97.14% <100.00%> (+0.17%)` | :arrow_up: |

... and [1 file with indirect coverage changes](https://codecov.io/gh/Azure/WALinuxAgent/pull/2811/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Azure)

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Azure). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Azure)


Patch:
diff --git a/azurelinuxagent/common/utils/extensionprocessutil.py b/azurelinuxagent/common/utils/extensionprocessutil.py
index 9038f614..137f3aa2 100644
--- a/azurelinuxagent/common/utils/extensionprocessutil.py
+++ b/azurelinuxagent/common/utils/extensionprocessutil.py
@@ -18,6 +18,7 @@
 #
 
 import os
+import re
 import signal
 import time
 
@@ -87,6 +88,9 @@ def handle_process_completion(process, command, timeout, stdout, stderr, error_c
     return process_output
 
 
+SAS_TOKEN_RE = re.compile(r'(https://\S+\?)((sv|st|se|sr|sp|sip|spr|sig)=\S+)+', flags=re.IGNORECASE)
+
+
 def read_output(stdout, stderr):
     """
     Read the output of the process sent to stdout and stderr and trim them to the max appropriate length.
@@ -103,7 +107,11 @@ def read_output(stdout, stderr):
         stderr = ustr(stderr.read(TELEMETRY_MESSAGE_MAX_LEN), encoding='utf-8',
                       errors='backslashreplace')
 
-        return format_stdout_stderr(stdout, stderr)
+        def redact(s):
+            # redact query strings that look like SAS tokens
+            return SAS_TOKEN_RE.sub(r'\1<redacted>', s)
+
+        return format_stdout_stderr(redact(stdout), redact(stderr))
     except Exception as e:
         return format_stdout_stderr("", "Cannot read stdout/stderr: {0}".format(ustr(e)))
 


Test patch:
diff --git a/tests/ga/test_extension.py b/tests/ga/test_extension.py
index 2272a190..10f44274 100644
--- a/tests/ga/test_extension.py
+++ b/tests/ga/test_extension.py
@@ -3418,6 +3418,33 @@ class TestExtension(TestExtensionBase, HttpRequestPredicates):
                     self._assert_handler_status(protocol.report_vm_status, "Ready", 1, "1.0.0")
                     self.assertEqual("1", protocol.report_vm_status.call_args[0][0].vmAgent.vm_artifacts_aggregate_status.goal_state_aggregate_status.in_svd_seq_no, "SVD sequence number mismatch")
 
+    def test_it_should_redact_access_tokens_in_extension_output(self):
+        original = r'''ONE https://foo.blob.core.windows.net/bar?sv=2000&ss=bfqt&srt=sco&sp=rw&se=2025&st=2022&spr=https&sig=SI%3D
+            TWO:HTTPS://bar.blob.core.com/foo/bar/foo.txt?sv=2018&sr=b&sig=Yx%3D&st=2023%3A52Z&se=9999%3A59%3A59Z&sp=r TWO
+            https://bar.com/foo?uid=2018&sr=b THREE'''
+        expected = r'''ONE https://foo.blob.core.windows.net/bar?<redacted>
+            TWO:HTTPS://bar.blob.core.com/foo/bar/foo.txt?<redacted> TWO
+            https://bar.com/foo?uid=2018&sr=b THREE'''
+
+        with mock_wire_protocol(mockwiredata.DATA_FILE) as protocol:
+            exthandlers_handler = get_exthandlers_handler(protocol)
+
+            original_popen = subprocess.Popen
+
+            def mock_popen(cmd, *args, **kwargs):
+                if cmd.endswith("sample.py -enable"):
+                    cmd = "echo '{0}'; >&2 echo '{0}'; exit 1".format(original)
+                return original_popen(cmd, *args, **kwargs)
+
+            with patch.object(subprocess, 'Popen', side_effect=mock_popen):
+                exthandlers_handler.run()
+
+            status = exthandlers_handler.report_ext_handlers_status()
+            self.assertEqual(1, len(status.vmAgent.extensionHandlers), 'Expected exactly 1 extension status')
+            message = status.vmAgent.extensionHandlers[0].message
+            self.assertIn('[stdout]\n{0}'.format(expected), message, "The extension's stdout was not redacted correctly")
+            self.assertIn('[stderr]\n{0}'.format(expected), message, "The extension's stderr was not redacted correctly")
+
 
 if __name__ == '__main__':
     unittest.main()