Repository: boto/boto

Issue:
S3 connections fail with [Errno 11] Resource temporarily unavailable with boto >= 2.22.0
I'm trying to create anonymous connection to S3:

from boto.s3.connection import S3Connection
conn = S3Connection(anon=True, debug=10)
print conn.get_bucket('rhuiqe')

With boto-2.13.3 it works perfectly:
% python /tmp/s3repr.py  
send: 'GET /?max-keys=0 HTTP/1.1\r\nHost: rhuiqe.s3.amazonaws.com\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nUser-Agent: Boto/2.13.3 Python/2.7.5 Linux/3.11.10-200.fc19.x86_64\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: x-amz-id-2: IB19IyGLQiA5FFB3zAaW7UVRj/A8j74Rk6qf2ZHscSmNW3rtplU0hz29ZrnLsEMa
header: x-amz-request-id: 3A42EC28C62C0E99
header: Date: Mon, 13 Jan 2014 16:22:50 GMT
header: Content-Type: application/xml
header: Transfer-Encoding: chunked
header: Server: AmazonS3
<Bucket: rhuiqe>

With boto-2.22.0 and 2.22.1 it fails on the same host:
% python /tmp/s3repr.py 
send: 'GET /?max-keys=0 HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nHost: s3.amazonaws.com\r\nUser-Agent: Boto/2.22.0 Python/2.7.5 Linux/3.11.10-200.fc19.x86_64\r\n\r\n'
reply: 'HTTP/1.1 307 Temporary Redirect\r\n'
header: x-amz-id-2: O2zkCr1jF/kDq+gXnUvvJ1hFwGlCxk7m6jL5ssoXzYPClf06Q71z0DUXugyMHscS
header: x-amz-request-id: 4571B74D9FEF45EB
header: Date: Mon, 13 Jan 2014 16:18:43 GMT
header: Location: http://aws.amazon.com/s3
header: Content-Length: 0
header: Server: AmazonS3
send: 'GET /s3 HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nHost: s3.amazonaws.com\r\nUser-Agent: Boto/2.22.0 Python/2.7.5 Linux/3.11.10-200.fc19.x86_64\r\n\r\n'
send: 'GET /s3 HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nHost: s3.amazonaws.com\r\nUser-Agent: Boto/2.22.0 Python/2.7.5 Linux/3.11.10-200.fc19.x86_64\r\n\r\n'
reply: 'HTTP/1.1 301 Moved Permanently\r\n'
header: Date: Mon, 13 Jan 2014 16:18:43 GMT
header: Server: Server
header: X-Frame-Options: SAMEORIGIN
header: x-amz-id-1: 1KT06MGZ64BFBM4RF2ZV
header: x-amz-id-2: SqJJw6YHJqhlBid6oZFDCzETneRGyDQsaPXq0nznleVEJq6cyshVwoKuK7toy9vUhGxCmZzSJo76 MWOKlm0DYX0uH/v62JrW
header: Location: /s3/
header: Cache-Control: no-store, no-cache, must-revalidate
header: Content-Length: 0
header: Content-Type: text/plain
header: Vary: User-Agent
Traceback (most recent call last):
  File "/tmp/s3repr.py", line 3, in <module>
    print conn.get_bucket('rhuiqe')
  File "/usr/lib/python2.7/site-packages/boto/s3/connection.py", line 434, in get_bucket
    bucket.get_all_keys(headers, maxkeys=0)
  File "/usr/lib/python2.7/site-packages/boto/s3/bucket.py", line 449, in get_all_keys
    '', headers, **params)
  File "/usr/lib/python2.7/site-packages/boto/s3/bucket.py", line 377, in _get_all
    query_args=query_args)
  File "/usr/lib/python2.7/site-packages/boto/s3/connection.py", line 550, in make_request
    retry_handler=retry_handler
  File "/usr/lib/python2.7/site-packages/boto/connection.py", line 1017, in make_request
    retry_handler=retry_handler)
  File "/usr/lib/python2.7/site-packages/boto/connection.py", line 897, in _mexe
    request.body, request.headers)
  File "/usr/lib64/python2.7/httplib.py", line 973, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.7/httplib.py", line 1007, in _send_request
    self.endheaders(body)
  File "/usr/lib64/python2.7/httplib.py", line 969, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 829, in _send_output
    self.send(msg)
  File "/usr/lib64/python2.7/httplib.py", line 791, in send
    self.connect()
  File "/usr/lib64/python2.7/httplib.py", line 1172, in connect
    self.timeout, self.source_address)
  File "/usr/lib64/python2.7/socket.py", line 553, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.error: [Errno 11] Resource temporarily unavailable



Discussion:

Resolving patch:
diff --git a/boto/connection.py b/boto/connection.py
--- a/boto/connection.py
+++ b/boto/connection.py
@@ -900,8 +900,8 @@ def _mexe(self, request, sender=None, override_num_retries=None,
                 # the port info. All others should be now be up to date and
                 # not include the port.
                 if 's3' not in self._required_auth_capability():
-                    self.set_host_header(request)
-
+                    if not getattr(self, 'anon', False):
+                        self.set_host_header(request)
                 if callable(sender):
                     response = sender(connection, request.method, request.path,
                                       request.body, request.headers)


Test patch:
diff --git a/tests/integration/s3/test_connection.py b/tests/integration/s3/test_connection.py
--- a/tests/integration/s3/test_connection.py
+++ b/tests/integration/s3/test_connection.py
@@ -15,7 +15,7 @@
 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
 # OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
 # ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
-# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
 # WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
 # IN THE SOFTWARE.
@@ -196,7 +196,7 @@ def test_1_basic(self):
         # now try to inject a response header
         data = k.get_contents_as_string(response_headers={'response-content-type' : 'foo/bar'})
         assert k.content_type == 'foo/bar'
-        
+
         # now delete all keys in bucket
         for k in bucket:
             if k.name == 'reduced_redundancy':
@@ -224,6 +224,7 @@ def test_basic_anon(self):
 
         # give bucket anon user access and anon read again
         auth_bucket.set_acl('public-read')
+        time.sleep(5)
         try:
             iter(anon_bucket.list()).next()
             self.fail("not expecting contents")