Repository: zulip/zulip

Issue:
Add a configuration flag to skip proxy with boto3, for EC2 using IAM roles to upload to S3
Attempts to upload files to the S3 file upload backend can fail if they use IAM roles on the EC2 instance to provide auth to perform the S3 upload.  This is because boto3, if using IAM instance credentials, makes a request to the metadata IP address (169.254.169.254), which is precisely the kind of address which Smokescreen does _not_ allow.

The S3 storage backend should use a configuration option to allow [passing an empty set of proxies to `boto3`](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html#specify-proxies).  Enabling this by default is not correct, since non-EC2 hosts may _require_ the proxy if it is an exit proxy.  Since we primarily document the access_key / secret_access_key authentication method, we should add an additional configuration option to support the corner case of IAM roles.

See [discussion on chat.zulip.org](https://chat.zulip.org/#narrow/stream/31-production-help/topic/Can't.20access.20AWS.20metadata.20service/near/1303045).


Discussion:
Hello @zulip/server-misc, @zulip/server-production members, this issue was labeled with the "area: production", "area: uploads" labels, so you may want to check it out!

<!-- areaLabelAddition -->

@zulipbot claim
Hello @Shubh0405!

Thanks for your interest in Zulip! You have attempted to claim an issue without the labels "help wanted", "good first issue". Since you're a new contributor, you can only claim and submit pull requests for issues with the [help wanted](https://github.com/zulip/zulip/issues?q=is%3Aopen+is%3Aissue+no%3Aassignee+label%3A%22help+wanted%22) or [good first issue](https://github.com/zulip/zulip/issues?q=is%3Aopen+is%3Aissue+no%3Aassignee+label%3A%22good+first+issue%22) labels.

If this is your first time here, we recommend reading our [guide for new contributors](https://zulip.readthedocs.io/en/latest/overview/contributing.html) before getting started.

@Shubh0405: Go ahead and give this a shot.
Hello @Shubh0405, you have been unassigned from this issue because you have not updated this issue or any referenced pull requests for over 14 days.

You can reclaim this issue or claim any other issue by commenting `@zulipbot claim` on that issue.

Thanks for your contributions, and hope to see you again soon!


Resolving patch:
diff --git a/zerver/lib/upload.py b/zerver/lib/upload.py
--- a/zerver/lib/upload.py
+++ b/zerver/lib/upload.py
@@ -85,6 +85,14 @@
 # through a sanitization function.
 
 
+# https://github.com/boto/botocore/issues/2644 means that the IMDS
+# request _always_ pulls from the environment.  Monkey-patch the
+# `should_bypass_proxies` function if we need to skip them, based
+# on S3_SKIP_PROXY.
+if settings.S3_SKIP_PROXY is True:  # nocoverage
+    botocore.utils.should_bypass_proxies = lambda url: True
+
+
 class RealmUploadQuotaError(JsonableError):
     code = ErrorCode.REALM_UPLOAD_QUOTA
 
diff --git a/zproject/default_settings.py b/zproject/default_settings.py
--- a/zproject/default_settings.py
+++ b/zproject/default_settings.py
@@ -137,6 +137,7 @@
 S3_AUTH_UPLOADS_BUCKET = ""
 S3_REGION: Optional[str] = None
 S3_ENDPOINT_URL: Optional[str] = None
+S3_SKIP_PROXY = True
 LOCAL_UPLOADS_DIR: Optional[str] = None
 MAX_FILE_UPLOAD_SIZE = 25
 
diff --git a/zproject/prod_settings_template.py b/zproject/prod_settings_template.py
--- a/zproject/prod_settings_template.py
+++ b/zproject/prod_settings_template.py
@@ -721,6 +721,7 @@
 # S3_AVATAR_BUCKET = ""
 # S3_REGION = None
 # S3_ENDPOINT_URL = None
+# S3_SKIP_PROXY = True
 
 ## Maximum allowed size of uploaded files, in megabytes.  This value is
 ## capped at 80MB in the nginx configuration, because the file upload


Test patch: