Repository: great-expectations/great_expectations
Pull request: [BUGFIX] Stop masking config strings
This should be unneeded, and results in warnings if users don't have sqlalchemy supported (e.g. for ABS)

- [ ] Description of PR changes above includes a link to [an existing GitHub issue](https://github.com/great-expectations/great_expectations/issues)
- [ ] PR title is prefixed with one of: [BUGFIX], [FEATURE], [DOCS], [MAINTENANCE], [CONTRIB]
- [ ] Code is linted - run `invoke lint` (uses `ruff format` + `ruff check`)
- [ ] Appropriate tests and docs have been updated

For more information about contributing, visit our [community resources](https://docs.greatexpectations.io/docs/core/introduction/community_resources#contribute-code-or-documentation).

After you submit your PR, keep the page open and **monitor the statuses of the various checks made by our continuous integration process at the bottom of the page. Please fix any issues that come up** and [reach out on Slack](https://greatexpectations.io/slack) if you need help. Thanks for contributing!


Problem statement:
URL Masking not working for azure blob storage connection string
**Describe the bug**
I am currently getting constant warnings about not having SQL Alchemy installed when the mask_db_url for the following config_variables.yml entry:

AZURE_STORAGE_CONNECTION_STRING: "DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=<redacted>;AccountKey=<redacted>"

I have removed the account name and account key for this example

**To Reproduce**

Include above entry in config_variables.yml

Here is my great_expectations.yml config:

```
config_variables_file_path: uncommitted/config_variables.yml
plugins_directory: plugins/
stores:
# Stores are configurable places to store things like Expectations, Validations
# Data Docs, and more. These are for advanced users only - most users can simply
# leave this section alone.
  expectations_store:
    class_name: ExpectationsStore
    store_backend:
      class_name: TupleFilesystemStoreBackend
      base_directory: expectations/

  validation_results_store:
    class_name: ValidationResultsStore
    store_backend:
      class_name: TupleFilesystemStoreBackend
      base_directory: uncommitted/validations/

  checkpoint_store:
    class_name: CheckpointStore
    store_backend:
      class_name: TupleFilesystemStoreBackend
      suppress_store_backend_id: true
      base_directory: checkpoints/

  validation_definition_store:
    class_name: ValidationDefinitionStore
    store_backend:
      class_name: TupleFilesystemStoreBackend
      base_directory: validation_definitions/

expectations_store_name: expectations_store
validation_results_store_name: validation_results_store
checkpoint_store_name: checkpoint_store

data_docs_sites:
  local_site:
    class_name: SiteBuilder
    show_how_to_buttons: true
    store_backend:
      class_name: TupleFilesystemStoreBackend
      base_directory: uncommitted/data_docs/local_site/
    site_index_builder:
      class_name: DefaultSiteIndexBuilder

  azure_docs_site:
    class_name: SiteBuilder
    store_backend:
      class_name: TupleAzureBlobStoreBackend
      container: \$web
      connection_string: ${AZURE_STORAGE_CONNECTION_STRING}
    site_index_builder:
      class_name: DefaultSiteIndexBuilder

  my_data_docs_site:
    class_name: SiteBuilder
    site_index_builder:
      class_name: DefaultSiteIndexBuilder
    store_backend:
      class_name: TupleFilesystemStoreBackend
      base_directory: /dbfs/mnt/web/
fluent_datasources:
  spark_generic_technical_columns_template_suite:
    type: spark
    id: 1b8640f5-1461-4acc-b48f-509b50487332
    assets:
      dataframe_asset_generic_technical_columns_template_suite:
        type: dataframe
        id: 65b363e5-4f4c-4ec1-b989-951e1c3f24ce
        batch_metadata: {}
        batch_definitions:
          generic_technical_columns_template_batch_definition:
            id: 136d1a85-e2d4-4543-b239-9afecec2859b
            partitioner:
  spark_sap_s4_sap_s4_x8_openhub_sap_bw_sap_bw_commitments_management_line_items_suite:
    type: spark
    id: 1a329ac9-da93-43ad-8588-46519e410315
    assets:
      dataframe_asset_sap_s4_sap_s4_x8_openhub_sap_bw_sap_bw_commitments_management_line_items_suite:
        type: dataframe
        id: 64cde889-e875-48ad-ad00-8da951924121
        batch_metadata: {}
        batch_definitions:
          sap_s4_sap_s4_x8_openhub_sap_bw_sap_bw_commitments_management_line_items_batch_definition:
            id: b532cafb-d79a-4652-9b0f-dea16541c701
            partitioner:
analytics_enabled:
data_context_id: 9b7e3388-812e-4325-a3bc-7f48736d52e7
```

I get this warning message all the time for most operations using the GE Framework. Simplest example would be:

ge_context = ge.get_context(context_root_dir="/dbfs/mnt/config/gx/")

I get the output:

/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/great_expectations/data_context/util.py:196: UserWarning: SQLAlchemy is not installed, using urlparse to mask database url password which ignores **kwargs.
  warnings.warn(

**Expected behavior**
Looking at the source code for the mask_db_url function I see that there is a special case included for the azure blob storage strings _obfuscate_azure_blobstore_connection_string. But the function seems to fail the check if url.startswith("DefaultEndpointsProtocol") for some reason.

**Environment (please complete the following information):**
 - Operating System: Linux
 - Great Expectations Version: 1.2.0
 - Data Source: Spark
 - Cloud environment: Databricks

Hints from issue discussion:
Hi there! Thanks for bringing this to our attention. I've shared it with the team, so please keep an eye out for updates on this issue.

Commit message: Update docstring

Patch to review:
diff --git a/great_expectations/data_context/util.py b/great_expectations/data_context/util.py
index 1c02d27153da..35e2ae8403f3 100644
--- a/great_expectations/data_context/util.py
+++ b/great_expectations/data_context/util.py
@@ -171,7 +171,7 @@ class PasswordMasker:
     @classmethod
     def mask_db_url(cls, url: str, use_urlparse: bool = False, **kwargs) -> str:
         """
-        Mask password in database url.
+        Mask password in database url unless it is a substitution string, e.g. ConfigStr.
         Uses sqlalchemy engine parsing if sqlalchemy is installed, otherwise defaults to using urlparse from the stdlib which does not handle kwargs.
         Args:
             url: Database url e.g. "postgresql+psycopg2://username:password@host:65432/database"
@@ -181,8 +181,19 @@ def mask_db_url(cls, url: str, use_urlparse: bool = False, **kwargs) -> str:
         Returns:
             url with password masked e.g. "postgresql+psycopg2://username:***@host:65432/database"
         """  # noqa: E501
+
+        from great_expectations.datasource.fluent.config_str import ConfigStr
+
+        try:
+            ConfigStr.validate_template_str_format(url)
+            is_config_str = True
+        except ValueError:
+            is_config_str = False
+
         if url.startswith("DefaultEndpointsProtocol"):
             return cls._obfuscate_azure_blobstore_connection_string(url)
+        elif is_config_str:
+            return url
         elif sa is not None and use_urlparse is False:
             try:
                 engine = sa.create_engine(url, **kwargs)
diff --git a/great_expectations/datasource/fluent/config_str.py b/great_expectations/datasource/fluent/config_str.py
index b9b68de01954..d41396aa4c9f 100644
--- a/great_expectations/datasource/fluent/config_str.py
+++ b/great_expectations/datasource/fluent/config_str.py
@@ -71,7 +71,7 @@ def str_contains_config_template(cls, v: str) -> bool:
         return TEMPLATE_STR_REGEX.search(v) is not None
 
     @classmethod
-    def _validate_template_str_format(cls, v: str) -> str | None:
+    def validate_template_str_format(cls, v: str) -> str | None:
         if cls.str_contains_config_template(v):
             return v
         raise ValueError(
@@ -85,7 +85,7 @@ def __get_validators__(cls):
         # one or more validators may be yielded which will be called in the
         # order to validate the input, each validator will receive as an input
         # the value returned from the previous validator
-        yield cls._validate_template_str_format
+        yield cls.validate_template_str_format
         yield cls.validate
 
     @classmethod
@@ -228,7 +228,7 @@ def __get_validators__(cls):
         # one or more validators may be yielded which will be called in the
         # order to validate the input, each validator will receive as an input
         # the value returned from the previous validator
-        yield ConfigStr._validate_template_str_format
+        yield ConfigStr.validate_template_str_format
         yield cls.validate  # equivalent to AnyUrl.validate
 
     @classmethod
diff --git a/tests/data_context/test_data_context_utils.py b/tests/data_context/test_data_context_utils.py
index c2d6e593f0fb..9200851f410c 100644
--- a/tests/data_context/test_data_context_utils.py
+++ b/tests/data_context/test_data_context_utils.py
@@ -50,6 +50,13 @@ def test_load_class_raises_error_when_module_name_is_not_string():
             load_class(bad_input, "great_expectations.datasource")
 
 
+@pytest.mark.unit
+def test_mask_db_url__does_not_mask_config_strings():
+    config_str = "${MY_DB_URL}"
+    output = PasswordMasker.mask_db_url(config_str)
+    assert output == config_str
+
+
 @pytest.mark.filesystem
 @pytest.mark.filterwarnings(
     "ignore:SQLAlchemy is not installed*:UserWarning:great_expectations.data_context.util"


Code review comments:
diff_hunk: @@ -181,8 +181,19 @@ def mask_db_url(cls, url: str, use_urlparse: bool = False, **kwargs) -> str:
         Returns:
             url with password masked e.g. "postgresql+psycopg2://username:***@host:65432/database"
         """  # noqa: E501
+
+        from great_expectations.datasource.fluent.config_str import ConfigStr
+
+        try:
+            ConfigStr.validate_template_str_format(url)
path: great_expectations/data_context/util.py
text: @user1:
Can we call `ConfigStr.str_contains_config_template` instead? That seems to do the same thing as `validate_template_str_format` except returns a bool and doesn't raise. You could then remove the `try/except` here.

@author:
Oh yeah, that's much better. Thanks!