Repository: boto/boto

Issue:
When using a proxy with SSL connections redirects do not work
# What steps will reproduce the problem?
1. Create an S3 bucket in the EU region
2. Attempt to upload to the S3 bucket when using a proxy server
# What is the expected output? What do you see instead?

This should upload a file but instead returns a signature error.  When viewing the full debugging information it turns out that the wrong host header is being sent.  This is due to a base bug boto/connections.py.

Basically what is happening is this: 
1. Boto attempts the connection to the default S3 host <bucketname>.s3.aws.amazon.com
2. Amazon returns a 307 redirect error with a Location header pointed at an actual server in the EU to do the upload to
3. boto.connection._mexe sees the redirect and goes to create a new connection for the Location returned in the header (<bucketname>.external-s3.aws.amazon.com or something similar)
4. new_http_connection gets called with the new host but immediately clobbers over it with the existing proxy config because use_proxy is True ( BUG #1 ).  
5. Because is_secure is forced to true and use_proxy is true the connection type returned is proxy_ssl().  This will pick the Host and Port from the base AWSConnectionObject (e.g., aws.amazon.com & 443) ( BUG #2 )

When _mexe goes to retry using the connection it thinks it is connected to <bucketname>.external-s3.aws.amazon.com but the actual HTTPConnection returned from proxy_ssl and the proxy both think they are talking to aws.amazon.com and the calculated signature is wrong.  Note that even if the signature was right this is still the wrong host to upload too.

I think the right thing to do here is to make proxy_ssl accept a host name and port that it should open a proxy too as parameters. The new_http_connection method would also need to be changed to pass the redirected location and port on from the Location header (and not replace them if use_proxy is True).
# What version of the product are you using? On what operating system?

boto-2.0, Ubuntu Linux (lucid), boto-2.5.2, Ubuntu Linux (lucid)



Discussion:
This is a duplicate of http://code.google.com/p/boto/issues/detail?id=535 but I couldn't find a copy here.


Resolving patch:
diff --git a/boto/connection.py b/boto/connection.py
--- a/boto/connection.py
+++ b/boto/connection.py
@@ -658,7 +658,7 @@ def get_http_connection(self, host, is_secure):
             return self.new_http_connection(host, is_secure)
 
     def new_http_connection(self, host, is_secure):
-        if self.use_proxy:
+        if self.use_proxy and not is_secure:
             host = '%s:%d' % (self.proxy, int(self.proxy_port))
         if host is None:
             host = self.server_name()
@@ -667,7 +667,7 @@ def new_http_connection(self, host, is_secure):
                     'establishing HTTPS connection: host=%s, kwargs=%s',
                     host, self.http_connection_kwargs)
             if self.use_proxy:
-                connection = self.proxy_ssl()
+                connection = self.proxy_ssl(host, is_secure and 443 or 80)
             elif self.https_connection_factory:
                 connection = self.https_connection_factory(host)
             elif self.https_validate_certificates and HAVE_HTTPS_CONNECTION:
@@ -703,8 +703,11 @@ def new_http_connection(self, host, is_secure):
     def put_http_connection(self, host, is_secure, connection):
         self._pool.put_http_connection(host, is_secure, connection)
 
-    def proxy_ssl(self):
-        host = '%s:%d' % (self.host, self.port)
+    def proxy_ssl(self, host=None, port=None):
+        if host and port:
+            host = '%s:%d' % (host, port)
+        else:
+            host = '%s:%d' % (self.host, self.port)
         sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         try:
             sock.connect((self.proxy, int(self.proxy_port)))


Test patch: