Repository: localstack/localstack

Issue:
CORS S3 responds twice with Access-Control-Allow-Origin headers, breaking Browser based calls
Scenario:
- Given I have a S3 bucket, with or without any explicit CORS configuration
- Given I do a browser request against any s3 object (it includes an origin header)
- I receive a response with two `Access-Control-Allow-Origin` headers.
  - `Access-Control-Allow-Origin: *` and `Access-Control-Allow-Origin: https://example.com`

*Expected result:* A response with just one `Access-Control-Allow-Origin` header.

*Impact:* all browser based request fail due to a CORS violation. Sending two `Access-Control-Allow-Origin` requests is not allowed.

My setup:
- Mac OS X
- Docker-based setup
- puma-dev to create a local domain

## Example request:
```
$ curl 'https://librario-dev-assets2.library-aws-s3.test/uploads/account/1/asset/file/616af873-b2bb-4c2b-8c5d-77c9ca8026c1/file' -v \
-H 'Origin: https://example.com'
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to librario-dev-assets2.library-aws-s3.test (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-ECDSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: O=Puma-dev Signed; CN=librario-dev-assets2.library-aws-s3.test
*  start date: Oct  1 18:45:24 2019 GMT
*  expire date: Sep 30 18:45:24 2020 GMT
*  subjectAltName: host "librario-dev-assets2.library-aws-s3.test" matched cert's "librario-dev-assets2.library-aws-s3.test"
*  issuer: O=Developer Certificate; CN=Puma-dev CA
*  SSL certificate verify ok.
> GET /uploads/account/1/asset/file/616af873-b2bb-4c2b-8c5d-77c9ca8026c1/file HTTP/1.1
> Host: librario-dev-assets2.library-aws-s3.test
> User-Agent: curl/7.54.0
> Accept: */*
> Origin: https://example.com
>
< HTTP/1.1 404 Not Found
< Access-Control-Allow-Headers: ,authorization,content-type,content-md5,cache-control,x-amz-content-sha256,x-amz-date,x-amz-security-token,x-amz-user-agent,x-amz-target,x-amz-acl,x-amz-version-id,x-localstack-target
< Access-Control-Allow-Methods: HEAD,GET,PUT,POST,DELETE,OPTIONS,PATCH
< Access-Control-Allow-Origin: https://example.com
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: x-amz-version-id
< Content-Length: 244
< Content-Type: text/html; charset=utf-8
< Date: Tue, 01 Oct 2019 18:48:59 GMT
< Server: BaseHTTP/0.6 Python/3.6.8
< Vary: Origin
<
<?xml version="1.0" encoding="UTF-8"?>
<Error>
    <Code>NoSuchBucket</Code>
    <Message>The specified bucket does not exist</Message>
    <BucketName>uploads</BucketName>
    <RequestID>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</RequestID>
* Connection #0 to host librario-dev-assets2.library-aws-s3.test left intact
</Error>%
```


Discussion:
Thanks for reporting @tmaier . I wasn't able to reproduce this issue locally. Can you post the HTTP dump when you run a request directly against LocalStack, bypassing Puma (i.e., curl to `localhost:4572/...`)? Would be great if we could try to create a reproducible example that exposes this bug with LocalStack itself, without external dependencies. Thanks!
I just ran it against `localhost:4572`. Same issue.

```
$ curl 'http://localhost:4572/uploads/account/1/asset/file/616af873-b2bb-4c2b-8c5d-77c9ca8026c1/file' -v \
-H 'Origin: https://example.com'
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 4572 (#0)
> GET /uploads/account/1/asset/file/616af873-b2bb-4c2b-8c5d-77c9ca8026c1/file HTTP/1.1
> Host: localhost:4572
> User-Agent: curl/7.54.0
> Accept: */*
> Origin: https://example.com
>
< HTTP/1.1 404 Not Found
< Server: BaseHTTP/0.6 Python/3.6.8
< Date: Sat, 05 Oct 2019 22:16:10 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 244
< Access-Control-Allow-Origin: https://example.com
< Vary: Origin
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: HEAD,GET,PUT,POST,DELETE,OPTIONS,PATCH
< Access-Control-Allow-Headers: authorization,content-type,content-md5,cache-control,x-amz-content-sha256,x-amz-date,x-amz-security-token,x-amz-user-agent,x-amz-target,x-amz-acl,x-amz-version-id,x-localstack-target
< Access-Control-Expose-Headers: x-amz-version-id
<
<?xml version="1.0" encoding="UTF-8"?>
<Error>
    <Code>NoSuchBucket</Code>
    <Message>The specified bucket does not exist</Message>
    <BucketName>uploads</BucketName>
    <RequestID>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</RequestID>
* Connection #0 to host localhost left intact
</Error>%
```
Different stack, same issue

```
$ pip3 install localstack
$ localstack start
$ localstack --version                                                                                                                           1 ↵
0.10.4.1
$ curl 'http://localhost:4572/test' -v -H 'Origin: https://example.com'
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 4572 (#0)
> GET /test HTTP/1.1
> Host: localhost:4572
> User-Agent: curl/7.54.0
> Accept: */*
> Origin: https://example.com
>
< HTTP/1.1 404 Not Found
< Server: BaseHTTP/0.6 Python/3.6.8
< Date: Sat, 05 Oct 2019 22:35:30 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 241
< Access-Control-Allow-Origin: https://example.com
< Vary: Origin
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: HEAD,GET,PUT,POST,DELETE,OPTIONS,PATCH
< Access-Control-Allow-Headers: authorization,content-type,content-md5,cache-control,x-amz-content-sha256,x-amz-date,x-amz-security-token,x-amz-user-agent,x-amz-target,x-amz-acl,x-amz-version-id,x-localstack-target
< Access-Control-Expose-Headers: x-amz-version-id
<
<?xml version="1.0" encoding="UTF-8"?>
<Error>
    <Code>NoSuchBucket</Code>
    <Message>The specified bucket does not exist</Message>
    <BucketName>test</BucketName>
    <RequestID>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</RequestID>
* Connection #0 to host localhost left intact
</Error>%
```
I downgraded now to `localstack/localstack:0.10.3` and I just see one `Access-Control-Allow-Origin`.

Could this be related to the latest changes in this commit? https://github.com/localstack/localstack/commit/3ea1ca3ebbeb9d42f56de737c3a4a8e7b815bd36#diff-fa47b177ffc75756ce3d7baf0b01db1dR259-R351 (#1586)
I also set up a new linux (Ubuntu 18), just to ensure that it is nothing specific to my setup.
I get the same error.

```
parallels@parallels-Parallels-Virtual-Platform:~$ curl 'http://localhost:4572/test' -v -H 'Origin: https://example.com'
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 4572 (#0)
> GET /test HTTP/1.1
> Host: localhost:4572
> User-Agent: curl/7.58.0
> Accept: */*
> Origin: https://example.com
> 
< HTTP/1.1 404 Not Found
< Server: BaseHTTP/0.6 Python/3.6.8
< Date: Sun, 06 Oct 2019 08:55:30 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 241
< Access-Control-Allow-Origin: https://example.com
< Vary: Origin
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: HEAD,GET,PUT,POST,DELETE,OPTIONS,PATCH
< Access-Control-Allow-Headers: authorization,content-type,content-md5,cache-control,x-amz-content-sha256,x-amz-date,x-amz-security-token,x-amz-user-agent,x-amz-target,x-amz-acl,x-amz-version-id,x-localstack-target
< Access-Control-Expose-Headers: x-amz-version-id
< 
<?xml version="1.0" encoding="UTF-8"?>
<Error>
    <Code>NoSuchBucket</Code>
    <Message>The specified bucket does not exist</Message>
    <BucketName>test</BucketName>
    <RequestID>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</RequestID>
* Connection #0 to host localhost left intact
```
Had same issue with an API Gateway & Lambda backend. When returning a 400 status code, and with Lambda returning a specific domain in "Access-Control-Allow-Origin" header, localstack is adding another "Access-Control-Allow-Origin : *" header which breaks CORS.
Downgrading to 0.10.3 resolved issue in this case too

Resolving patch:
diff --git a/localstack/services/generic_proxy.py b/localstack/services/generic_proxy.py
--- a/localstack/services/generic_proxy.py
+++ b/localstack/services/generic_proxy.py
@@ -337,7 +337,8 @@ def forward(self, method):
                 LOG.warning('Unable to flush write file: %s' % e)
 
     def _send_cors_headers(self, response=None):
-        headers = response and response.headers or {}
+        # Note: Use "response is not None" here instead of "not response"!
+        headers = response is not None and response.headers or {}
         if 'Access-Control-Allow-Origin' not in headers:
             self.send_header('Access-Control-Allow-Origin', '*')
         if 'Access-Control-Allow-Methods' not in headers:
diff --git a/localstack/utils/common.py b/localstack/utils/common.py
--- a/localstack/utils/common.py
+++ b/localstack/utils/common.py
@@ -447,12 +447,12 @@ def recurse_object(obj, func, path=''):
     obj = func(obj, path=path)
     if isinstance(obj, list):
         for i in range(len(obj)):
-            path = '%s[%s]' % (path or '.', i)
-            obj[i] = recurse_object(obj[i], func, path)
+            tmp_path = '%s[%s]' % (path or '.', i)
+            obj[i] = recurse_object(obj[i], func, tmp_path)
     elif isinstance(obj, dict):
         for k, v in obj.items():
-            path = '%s%s' % ((path + '.') if path else '', k)
-            obj[k] = recurse_object(v, func, path)
+            tmp_path = '%s%s' % ((path + '.') if path else '', k)
+            obj[k] = recurse_object(v, func, tmp_path)
     return obj
 
 


Test patch: