Repository: apache/airflow

Issue:
WASB remote logging too verbose in task logger
**Apache Airflow version**: 2.1.0

**Environment**:
apache-airflow-providers-microsoft-azure == 2.0.0

**What happened**:

When wasb remote logger wants to create container (always), the log is part of the task logger.

```
[2021-06-02 13:55:51,619] {wasb.py:385} INFO - Attempting to create container:  xxxxx
[2021-06-02 13:55:51,479] {_universal.py:419} INFO - Request URL: 'XXXXXXX'
[2021-06-02 13:55:51,483] {_universal.py:420} INFO - Request method: 'HEAD'
[2021-06-02 13:55:51,490] {_universal.py:421} INFO - Request headers:
[2021-06-02 13:55:51,495] {_universal.py:424} INFO -     'x-ms-version': 'REDACTED'
[2021-06-02 13:55:51,499] {_universal.py:424} INFO -     'Accept': 'application/xml'
[2021-06-02 13:55:51,507] {_universal.py:424} INFO -     'User-Agent': 'azsdk-python-storage-blob/12.8.1 Python/3.7.10 (Linux-5.4.0-1046-azure-x86_64-with-debian-10.9)'
[2021-06-02 13:55:51,511] {_universal.py:424} INFO -     'x-ms-date': 'REDACTED'
[2021-06-02 13:55:51,517] {_universal.py:424} INFO -     'x-ms-client-request-id': ''
[2021-06-02 13:55:51,523] {_universal.py:424} INFO -     'Authorization': 'REDACTED'
[2021-06-02 13:55:51,529] {_universal.py:437} INFO - No body was attached to the request
[2021-06-02 13:55:51,541] {_universal.py:452} INFO - Response status: 404
[2021-06-02 13:55:51,550] {_universal.py:453} INFO - Response headers:
[2021-06-02 13:55:51,556] {_universal.py:456} INFO -     'Transfer-Encoding': 'chunked'
[2021-06-02 13:55:51,561] {_universal.py:456} INFO -     'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0'
[2021-06-02 13:55:51,567] {_universal.py:456} INFO -     'x-ms-request-id': 'xxxx'
[2021-06-02 13:55:51,573] {_universal.py:456} INFO -     'x-ms-client-request-id': 'xxxx'
[2021-06-02 13:55:51,578] {_universal.py:456} INFO -     'x-ms-version': 'REDACTED'
[2021-06-02 13:55:51,607] {_universal.py:456} INFO -     'x-ms-error-code': 'REDACTED'
[2021-06-02 13:55:51,613] {_universal.py:456} INFO -     'Date': 'Wed, 02 Jun 2021 13:55:50 GMT'
```
**What you expected to happen**:
HTTP verbose logging not showing and "Attempting to create container" maybe it's not useful in the task log


ref: https://github.com/Azure/azure-sdk-for-python/issues/9422



Discussion:
based on Azure/azure-sdk-for-python#9422

maybe a solution could be adding
```
logger = logging.getLogger("azure.core.pipeline.policies.http_logging_policy")
logger.setLevel(logging.WARNING)
```
to wasb.py file
I’m thinking mayne it’d make sense to set the entire `azure` namespace to `WARNING`. This can be better done by modifying `airflow.config_templates_airflow_local_settings.DEFAULT_LOGGING_CONFIG` (and we will likely need a way to enable these logs back on, see https://issues.apache.org/jira/browse/AIRFLOW-1455 (it’s closed as WONTFIX but there is one person wanting to see more verbose logs).

Resolving patch:
diff --git a/airflow/providers/microsoft/azure/hooks/wasb.py b/airflow/providers/microsoft/azure/hooks/wasb.py
--- a/airflow/providers/microsoft/azure/hooks/wasb.py
+++ b/airflow/providers/microsoft/azure/hooks/wasb.py
@@ -26,6 +26,8 @@
 
 """
 
+import logging
+import os
 from typing import Any, Dict, List, Optional
 
 from azure.core.exceptions import HttpResponseError, ResourceExistsError, ResourceNotFoundError
@@ -112,6 +114,12 @@ def __init__(
         self.public_read = public_read
         self.blob_service_client = self.get_conn()
 
+        logger = logging.getLogger("azure.core.pipeline.policies.http_logging_policy")
+        try:
+            logger.setLevel(os.environ.get("AZURE_HTTP_LOGGING_LEVEL", logging.WARNING))
+        except ValueError:
+            logger.setLevel(logging.WARNING)
+
     def get_conn(self) -> BlobServiceClient:
         """Return the BlobServiceClient object."""
         conn = self.get_connection(self.conn_id)


Test patch: