Repository: great-expectations/great_expectations

Issue:
SQLAlchemy warning when not using SQL
**Describe the bug**
I'm using Azure Blob Storage as a backend and getting the following warning in the new version of ge 0.15.4. Don't think this warning showed up in versions prior to 0.15.

`WARNING:great_expectations.data_context.util:Something went wrong when trying to use SQLAlchemy to obfuscate URL: Could not parse rfc1738 URL from string 'BlobEndpoint=xxxxxx'`

**Expected behavior**
No warning of this is type should show when TupleAzureBlobStoreBackend backend is used.

**Environment (please complete the following information):**
 - System: Azure
 - Great Expectations Version: 0.15.4




Discussion:
Im getting the same error when using TupleAzureBlobStoreBackend
I am getting the same error. I've included my data source config below

```python
datasources={
        "pandas": DatasourceConfig(
            class_name="Datasource",
            execution_engine={
                "class_name": "PandasExecutionEngine"
            },
            data_connectors={
                "azure_data_configured": {
                    "class_name": "ConfiguredAssetAzureDataConnector",
                    "container": "\\$web",
                    "assets": {
                        "blob_csv": {
                            "pattern" : r".csv$",
                    }},
                    "azure_options": {
                        "conn_str": "DefaultEndpointsProtocol=https;AccountName=<ACCOUNT>;AccountKey<KEY>;EndpointSuffix=core.windows.net",
                    }
                }
            },
        )
    },
```
This looks like it's the result of an attempt to sanitize the `connection_string`, which is one of the `cls.URL_KEYS`, here:

https://github.com/great-expectations/great_expectations/blob/bd29694677eb83d2461a3b13ea07b060b14b2ed7/great_expectations/data_context/util.py#L601-L614

In the same way that sqlite urls are skipped, perhaps Azure connection strings should be skipped as well?  

https://github.com/great-expectations/great_expectations/blob/bd29694677eb83d2461a3b13ea07b060b14b2ed7/great_expectations/data_context/util.py#L569-L571

Else custom logic might be needed to mask them instead of treating them like a URL.
I was able to add the following snippet as a workaround right after the sqlite check in `_mask_db_url_no_sa` for now.  

```python
# Parse Azure Connection Strings in a special way
azure_conn_str_re = re.compile("(DefaultEndpointsProtocol=(http|https));(AccountName=([a-zA-Z0-9]+));(AccountKey=)([a-zA-Z0-9=\+]+);(EndpointSuffix=([a-zA-Z\.]+))")
if azure_conn_str_re.match(url):
    m = azure_conn_str_re.match(url)
    return f"DefaultEndpointsProtocol={m.group(2)};AccountName={m.group(4)};AccountKey=***;EndpointSuffix={m.group(7)}"
```

It's not very elegant in that it still results in warnings from the initial SQLAlchemy parsing attempt.  One underlying issue is that it would seem other DB (e.g. BigQuery) usage of `connection_string` indeed use URLs, however Azure's is not a valid URL as the original error states.
Have the same problem - any ETA on this?
Same for me, my credentials are being written to the logs.
This issue still occurs in great expectations 0.15.41
I am having the same issue. Could someone provide some ideas? Thank you!
Hey folks, thanks for your context & patience here. Core team is taking this on ASAP. Look for updates here very soon. 🚀 
Hey @mateoxc, thank you for your reply. Really appreciate it. 
```stores:
expectations_AZ_store:
    class_name: ExpectationsStore
    store_backend:
      class_name: TupleAzureBlobStoreBackend
      container: <blob-container>
      prefix: expectations
      connection_string: ${AZURE_STORAGE_CONNECTION_STRING}
```
This is exactly what I am using to design my DataContextConfig which created the above mentioned SQL error.
Hey @mingyy,
I confused the threads, this issue still occurs

Resolving patch:
diff --git a/great_expectations/data_context/util.py b/great_expectations/data_context/util.py
--- a/great_expectations/data_context/util.py
+++ b/great_expectations/data_context/util.py
@@ -1,7 +1,10 @@
+from __future__ import annotations
+
 import copy
 import inspect
 import logging
 import pathlib
+import re
 import warnings
 from typing import Any, Optional
 from urllib.parse import urlparse
@@ -9,6 +12,7 @@
 import pyparsing as pp
 
 from great_expectations.alias_types import PathStr  # noqa: TCH001
+from great_expectations.exceptions import StoreConfigurationError
 from great_expectations.types import safe_deep_copy
 from great_expectations.util import load_class, verify_dynamic_loading_support
 
@@ -183,7 +187,9 @@ def mask_db_url(cls, url: str, use_urlparse: bool = False, **kwargs) -> str:
         Returns:
             url with password masked e.g. "postgresql+psycopg2://username:***@host:65432/database"
         """
-        if sa is not None and use_urlparse is False:
+        if url.startswith("DefaultEndpointsProtocol"):
+            return cls._obfuscate_azure_blobstore_connection_string(url)
+        elif sa is not None and use_urlparse is False:
             try:
                 engine = sa.create_engine(url, **kwargs)
                 return engine.url.__repr__()
@@ -198,6 +204,25 @@ def mask_db_url(cls, url: str, use_urlparse: bool = False, **kwargs) -> str:
             )
         return cls._mask_db_url_no_sa(url=url)
 
+    @classmethod
+    def _obfuscate_azure_blobstore_connection_string(cls, url: str) -> str:
+        # Parse Azure Connection Strings
+        azure_conn_str_re = re.compile(
+            "(DefaultEndpointsProtocol=(http|https));(AccountName=([a-zA-Z0-9]+));(AccountKey=)(.+);(EndpointSuffix=([a-zA-Z\\.]+))"
+        )
+        try:
+            matched: re.Match[str] | None = azure_conn_str_re.match(url)
+            if not matched:
+                raise StoreConfigurationError(
+                    f"The URL for the Azure connection-string, was not configured properly. Please check and try again: {url} "
+                )
+            res = f"DefaultEndpointsProtocol={matched.group(2)};AccountName={matched.group(4)};AccountKey=***;EndpointSuffix={matched.group(8)}"
+            return res
+        except Exception as e:
+            raise StoreConfigurationError(
+                f"Something went wrong when trying to obfuscate URL for Azure connection-string. Please check your configuration: {e}"
+            )
+
     @classmethod
     def _mask_db_url_no_sa(cls, url: str) -> str:
         # oracle+cx_oracle does not parse well using urlparse, parse as oracle then swap back


Test patch:
diff --git a/tests/data_context/test_data_context_utils.py b/tests/data_context/test_data_context_utils.py
--- a/tests/data_context/test_data_context_utils.py
+++ b/tests/data_context/test_data_context_utils.py
@@ -7,6 +7,7 @@
     PasswordMasker,
     parse_substitution_variable,
 )
+from great_expectations.exceptions.exceptions import StoreConfigurationError
 from great_expectations.types import safe_deep_copy
 from great_expectations.util import load_class
 
@@ -283,6 +284,22 @@ def test_password_masker_mask_db_url(monkeypatch, tmp_path):
     assert PasswordMasker.mask_db_url("sqlite://", use_urlparse=True) == "sqlite://"
 
 
+def test_sanitize_config_azure_blob_store():
+    azure_url: str = "DefaultEndpointsProtocol=https;AccountName=iamname;AccountKey=i_am_account_key;EndpointSuffix=core.windows.net"
+    assert (
+        PasswordMasker.mask_db_url(azure_url)
+        == "DefaultEndpointsProtocol=https;AccountName=iamname;AccountKey=***;EndpointSuffix=core.windows.net"
+    )
+
+    azure_wrong_url: str = "DefaultEndpointsProtocol=i_dont_work;AccountName=iamname;AccountKey=i_am_account_key;EndpointSuffix=core.windows.net"
+    with pytest.raises(StoreConfigurationError):
+        PasswordMasker.mask_db_url(azure_wrong_url)
+
+    azure_missing_fields: str = "DefaultEndpointsProtocol=i_dont_work;AccountName=iamname;EndpointSuffix=core.windows.net"
+    with pytest.raises(StoreConfigurationError):
+        PasswordMasker.mask_db_url(azure_missing_fields)
+
+
 def test_sanitize_config_raises_exception_with_bad_input(
     basic_data_context_config,
 ):