Repository: great-expectations/great_expectations

Issue:
Unnecessary S3 GetBucketLocation call for endpoint path building
Following code breaks some self-hosted S3 implementations by requesting GetBucketLocation from the S3 endpoint (also if a `endpoint_url` is specified).

```
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetBucketLocation operation: Access Denied
```

GetBucketLocation is only needed in the `else` branch of the if statement and is otherwise discarded (overwritten):

https://github.com/great-expectations/great_expectations/blob/ee792823df549f58cc86c649a4c66bcc30817790/great_expectations/data_context/store/tuple_store_backend.py#L650-L665

Following code would only request the bucket location if necessary:

```diff
    def get_url_for_key(self, key, protocol=None):
--      location = self._create_client().get_bucket_location(Bucket=self.bucket)[
--          "LocationConstraint"
--      ]
        if self.boto3_options.get("endpoint_url"):
            location = self.boto3_options.get("endpoint_url")
        elif location is None:
            location = "https://s3.amazonaws.com"
        else:
++          location = self._create_client().get_bucket_location(Bucket=self.bucket)[
++              "LocationConstraint"
++          ]
            location = f"https://s3-{location}.amazonaws.com"

        s3_key = self._convert_key_to_filepath(key)

        if not self.prefix:
            return f"{location}/{self.bucket}/{s3_key}"
        return f"{location}/{self.bucket}/{self.prefix}/{s3_key}"
```


Discussion:
Thanks for raising this, @error418! It looks like you've tracked down the source of the error and have a proposed fix - is this something you'd interested in contributing? Please let us know if you need any help with opening up a PR and getting this across the line.

Resolving patch:
diff --git a/great_expectations/data_context/store/tuple_store_backend.py b/great_expectations/data_context/store/tuple_store_backend.py
--- a/great_expectations/data_context/store/tuple_store_backend.py
+++ b/great_expectations/data_context/store/tuple_store_backend.py
@@ -648,15 +648,20 @@ def list_keys(self, prefix: Tuple = ()) -> List[Tuple]:
         return key_list
 
     def get_url_for_key(self, key, protocol=None):
-        location = self._create_client().get_bucket_location(Bucket=self.bucket)[
-            "LocationConstraint"
-        ]
+        location = None
         if self.boto3_options.get("endpoint_url"):
             location = self.boto3_options.get("endpoint_url")
-        elif location is None:
-            location = "https://s3.amazonaws.com"
         else:
-            location = f"https://s3-{location}.amazonaws.com"
+            # build s3 endpoint when no endpoint_url is configured
+
+            location = self._create_client().get_bucket_location(Bucket=self.bucket)[
+                "LocationConstraint"
+            ]
+
+            if location is None:
+                location = "https://s3.amazonaws.com"
+            else:
+                location = f"https://s3-{location}.amazonaws.com"
 
         s3_key = self._convert_key_to_filepath(key)
 


Test patch: