Repository: mlflow/mlflow

Issue:
[FR] AzureBlobArtifactRepository connection using Service Principal credentials
## Willingness to contribute
The MLflow Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature (either as an MLflow Plugin or an enhancement to the MLflow code base)?

- [x] Yes. I can contribute this feature independently.
- [x] Yes. I would be willing to contribute this feature with guidance from the MLflow community.
- [ ] No. I cannot contribute this feature at this time.

## Proposal Summary

Allow the user to connect with the AzureBlobArtifactRepository using service principal credentials taken from the following environment variables:
* AZURE_TENANT_ID
* AZURE_CLIENT_ID
* AZURE_CLIENT_SECRET

## Motivation
- What is the use case for this feature?
Having the AZURE_STORAGE_CONNECTION_STRING or AZURE_STORAGE_ACCESS_KEY variables set will give the user of that environment complete control over the whole storage account. This is not always desirable in a situation where a storage account has multiple blob containers and the users of one blob container should not have access to another user's container. 

- Why is this use case valuable to support for MLflow users in general?
It gives the user more options to choose from while connecting with their artifact repository.

- Why is this use case valuable to support for your project(s) or organization
My organizations hosts multiple tracking servers (with corresponding artifact repositories) in which groups of users should be separated. Allowing the connection via service principal credentials gives us the ability assign permissions on a container to a service principal

- Why is it currently difficult to achieve this use case? (please be as specific as possible about why related MLflow features and components are insufficient)
The AzureBlobArtifactRepository requires either the AZURE_STORAGE_CONNECTION_STRING or AZURE_STORAGE_ACCESS_KEY to be set, which gives the user full control of the whole storage account.

### What component(s), interfaces, languages, and integrations does this feature affect?
Components 
- [x] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [x] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [ ] `area/projects`: MLproject format, project running backends
- [ ] `area/scoring`: Local serving, model deployment tools, spark UDFs
- [ ] `area/server-infra`: MLflow server, JavaScript dev server
- [ ] `area/tracking`: Tracking Service, tracking client APIs, autologging

Interfaces
- [ ] `area/uiux`: Front-end, user experience, JavaScript, plotting
- [ ] `area/docker`: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
- [ ] `area/sqlalchemy`: Use of SQLAlchemy in the Tracking Service or Model Registry
- [ ] `area/windows`: Windows support

Languages 
- [ ] `language/r`: R APIs and clients
- [ ] `language/java`: Java APIs and clients
- [ ] `language/new`: Proposals for new client languages

Integrations
- [ ] `integrations/azure`: Azure and Azure ML integrations
- [ ] `integrations/sagemaker`: SageMaker integrations
- [ ] `integrations/databricks`: Databricks integrations

## Details

The solution would entail adding another elif branch in the AzureBlobArtifactRepository __init__ function, that checks for the AZURE_TENANT_ID, AZURE_CLIENT_ID and AZURE_CLIENT_SECRET environment variables, creates a [ClientSecretCredential](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity.clientsecretcredential?view=azure-python) and uses it to create the BlobServiceClient.




Discussion:

Resolving patch:
diff --git a/mlflow/store/artifact/azure_blob_artifact_repo.py b/mlflow/store/artifact/azure_blob_artifact_repo.py
--- a/mlflow/store/artifact/azure_blob_artifact_repo.py
+++ b/mlflow/store/artifact/azure_blob_artifact_repo.py
@@ -14,8 +14,10 @@ class AzureBlobArtifactRepository(ArtifactRepository):
 
     This repository is used with URIs of the form
     ``wasbs://<container-name>@<ystorage-account-name>.blob.core.windows.net/<path>``,
-    following the same URI scheme as Hadoop on Azure blob storage. It requires that your Azure
-    storage access key be available in the environment variable ``AZURE_STORAGE_ACCESS_KEY``.
+    following the same URI scheme as Hadoop on Azure blob storage. It requires either that:
+    - Azure storage connection string is in the env var ``AZURE_STORAGE_CONNECTION_STRING``
+    - Azure storage access key is in the env var ``AZURE_STORAGE_ACCESS_KEY``
+    - DefaultAzureCredential is configured
     """
 
     def __init__(self, artifact_uri, client=None):
@@ -39,9 +41,17 @@ def __init__(self, artifact_uri, client=None):
                 account_url=account_url, credential=os.environ.get("AZURE_STORAGE_ACCESS_KEY")
             )
         else:
-            raise Exception(
-                "You need to set one of AZURE_STORAGE_CONNECTION_STRING or "
-                "AZURE_STORAGE_ACCESS_KEY to access Azure storage."
+            try:
+                from azure.identity import DefaultAzureCredential
+            except ImportError as exc:
+                raise ImportError(
+                    "Using DefaultAzureCredential requires the azure-identity package. "
+                    "Please install it via: pip install azure-identity"
+                ) from exc
+
+            account_url = "https://{account}.blob.core.windows.net".format(account=account)
+            self.client = BlobServiceClient(
+                account_url=account_url, credential=DefaultAzureCredential()
             )
 
     @staticmethod


Test patch:
diff --git a/tests/store/artifact/test_azure_blob_artifact_repo.py b/tests/store/artifact/test_azure_blob_artifact_repo.py
--- a/tests/store/artifact/test_azure_blob_artifact_repo.py
+++ b/tests/store/artifact/test_azure_blob_artifact_repo.py
@@ -52,11 +52,12 @@ def test_artifact_uri_factory(mock_client):
     del os.environ["AZURE_STORAGE_ACCESS_KEY"]
 
 
-def test_exception_if_no_env_vars(mock_client):
+@mock.patch("azure.identity.DefaultAzureCredential")
+def test_default_az_cred_if_no_env_vars(mock_default_azure_credential, mock_client):
     # pylint: disable=unused-argument
     # We pass in the mock_client here to clear Azure environment variables, but we don't use it
-    with pytest.raises(Exception, match="AZURE_STORAGE_CONNECTION_STRING"):
-        AzureBlobArtifactRepository(TEST_URI)
+    AzureBlobArtifactRepository(TEST_URI)
+    assert mock_default_azure_credential.call_count == 1
 
 
 def test_parse_wasbs_uri():