Repository: aws/sagemaker-python-sdk

Issue:
Mask creds in docker compose file for local mode
**Describe the feature you'd like**
Currently, [line 673](https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/local/image.py#L673) in `sagemaker/local/image.py` logs AWS creds in the docker compose file. This could cause a credentials leak if it is running in a ci pipeline, during presentations/demos and for live streamers or youtubers creating sagemaker content. 

I don't feel this is a outright security issue, just unexpected logging that could have adverse impact in certain situations. 

One has to explicitly provide fake creds and local data to get around this. This limits the ability to use data from s3 and use other aws services in jobs.

**How would this feature be used? Please describe.**
Credentials should be masked in the logs to prevent a possible leak. This could be accomplished with a regex replace of the yaml content.  I'm happy to submit a PR for this, but wanted to get thoughts because there may be better alternatives to regex. I'm not a fan of regex but it was the best I could come up with since the container needs the creds.

**Describe alternatives you've considered**
Currently patching out this logging in a ci pipeline.


Discussion:
> Credentials should be masked in the logs to prevent a possible leak. This could be accomplished with a regex replace of the yaml content. I'm happy to submit a PR for this, but wanted to get thoughts because there may be better alternatives to regex. I'm not a fan of regex but it was the best I could come up with since the container needs the creds.

Thanks for the suggestion! Masking the credentials definitely makes sense. It may be possible to use [PyYAML](https://pypi.org/project/PyYAML/) for this purpose instead of a regex-- I'd recommend looking into this. Please feel free to submit a PR and we'll be happy to review.

> > Credentials should be masked in the logs to prevent a possible leak. This could be accomplished with a regex replace of the yaml content. I'm happy to submit a PR for this, but wanted to get thoughts because there may be better alternatives to regex. I'm not a fan of regex but it was the best I could come up with since the container needs the creds.
> 
> Thanks for the suggestion! Masking the credentials definitely makes sense. It may be possible to use [PyYAML](https://pypi.org/project/PyYAML/) for this purpose instead of a regex-- I'd recommend looking into this. Please feel free to submit a PR and we'll be happy to review.

Excellent suggestion, that is way better. I'll take a stab at it over the weekend.
Think I have something working. The environment turned out to be a list of strings so the  simple solution was to set each item in the "environment" list to "[Masked]" for all services (on a separate deepcopy for logging).

One question: where is logging configured for local mode/test suite? I can't seem to track it down. I'm only getting WARNING level logging when running the test suite. When I run jobs in local mode, it returns INFO level logging. Makes using caplog to check the INFO logs a bit hard in the tests.

Any thoughts appreciated. Will submit PR once I can get assert on the info logs.

Resolving patch:
diff --git a/src/sagemaker/local/image.py b/src/sagemaker/local/image.py
--- a/src/sagemaker/local/image.py
+++ b/src/sagemaker/local/image.py
@@ -14,6 +14,7 @@
 from __future__ import absolute_import
 
 import base64
+import copy
 import errno
 import json
 import logging
@@ -670,7 +671,13 @@ def _generate_compose_file(self, command, additional_volumes=None, additional_en
             raise e
 
         yaml_content = yaml.dump(content, default_flow_style=False)
-        logger.info("docker compose file: \n%s", yaml_content)
+        # Mask all environment vars for logging, could contain secrects.
+        masked_content = copy.deepcopy(content)
+        for _, service_data in masked_content["services"].items():
+            service_data["environment"] = ["[Masked]" for _ in service_data["environment"]]
+
+        masked_content_for_logging = yaml.dump(masked_content, default_flow_style=False)
+        logger.info("docker compose file: \n%s", masked_content_for_logging)
         with open(docker_compose_path, "w") as f:
             f.write(yaml_content)
 


Test patch:
diff --git a/tests/unit/test_image.py b/tests/unit/test_image.py
--- a/tests/unit/test_image.py
+++ b/tests/unit/test_image.py
@@ -19,6 +19,7 @@
 from botocore.credentials import Credentials
 
 import base64
+import logging
 import json
 import os
 import subprocess
@@ -332,12 +333,14 @@ def test_check_output():
 @patch("sagemaker.local.data.get_data_source_instance")
 @patch("subprocess.Popen")
 def test_train(
-    popen, get_data_source_instance, retrieve_artifacts, cleanup, tmpdir, sagemaker_session
+    popen, get_data_source_instance, retrieve_artifacts, cleanup, tmpdir, sagemaker_session, caplog
 ):
     data_source = Mock()
     data_source.get_root_dir.return_value = "foo"
     get_data_source_instance.return_value = data_source
 
+    caplog.set_level(logging.INFO)
+
     directories = [str(tmpdir.mkdir("container-root")), str(tmpdir.mkdir("data"))]
     with patch(
         "sagemaker.local.image._SageMakerContainer._create_tmp_folder", side_effect=directories
@@ -388,6 +391,7 @@ def test_train(
 
     retrieve_artifacts.assert_called_once()
     cleanup.assert_called_once()
+    assert "[Masked]" in caplog.text
 
 
 @patch("sagemaker.local.local_session.LocalSession", Mock())
@@ -579,7 +583,8 @@ def test_container_does_not_enable_nvidia_docker_for_cpu_containers(sagemaker_se
 @patch("sagemaker.local.image._SageMakerContainer._prepare_serving_volumes", Mock(return_value=[]))
 @patch("shutil.copy", Mock())
 @patch("shutil.copytree", Mock())
-def test_serve(tmpdir, sagemaker_session):
+def test_serve(tmpdir, sagemaker_session, caplog):
+    caplog.set_level(logging.INFO)
     with patch(
         "sagemaker.local.image._SageMakerContainer._create_tmp_folder",
         return_value=str(tmpdir.mkdir("container-root")),
@@ -601,6 +606,7 @@ def test_serve(tmpdir, sagemaker_session):
             for h in sagemaker_container.hosts:
                 assert config["services"][h]["image"] == image
                 assert config["services"][h]["command"] == "serve"
+    assert "[Masked]" in caplog.text
 
 
 @patch("sagemaker.local.image._HostingContainer.run", Mock())