Repository: localstack/localstack

Issue:
S3: POST/PUT to bucket URLs don't route correctly on port 4566
<!-- Love localstack? Please consider supporting our collective:
👉  https://opencollective.com/localstack/donate -->

# Type of request: This is a ...

[x] bug report
[ ] feature request

# Detailed description

We noticed this while converting a service to use Localstack for tests. The service generates S3 presigned post URLs. We're able to create and use presigned S3 URLs on port 4572 (deprecated S3 port), but not 4566, the new shared one. The same issue happens with PUT requests, which is the simplest to repro.

While just using 4572 works, this does force us to use the deprecated port and I figured it was worth opening an issue because of the discrepancy.

## Expected behavior

POST http://localhost:4566/hello (with appropriate form params) should return a 204, in the same way that POST http://localhost:4572/hello does.

PUT http://localhost:4566/hello should create a bucket and return a 200, in the same way that PUT http://localhost:4572/hello does.

## Actual behavior

Both PUT and POST http://localhost:4566/hello return a 404.

In the localstack logs:
2020-05-20T13:37:41:INFO:localstack.services.edge: Unable to find forwarding rule for host "localhost:4566", path "/hello", target header "", auth header ""

# Steps to reproduce

```bash
$ curl -i -XPUT http://localhost:4572/hello
HTTP/1.1 200 OK
Server: BaseHTTP/0.6 Python/3.8.2
Date: Wed, 20 May 2020 13:43:17 GMT
Content-Type: application/xml; charset=utf-8
content-length: 159
Access-Control-Allow-Origin: *
Last-Modified: Wed, 20 May 2020 13:43:17 GMT
x-amz-request-id: 0ABD347D7A4E0697
x-amz-id-2: MzRISOwyjmnup0ABD347D7A4E06977/JypPGXLh0OVFGcJaaO3KW/hRAqKOpIEEp
Access-Control-Allow-Methods: HEAD,GET,PUT,POST,DELETE,OPTIONS,PATCH
Access-Control-Allow-Headers: authorization,content-type,content-md5,cache-control,x-amz-content-sha256,x-amz-date,x-amz-security-token,x-amz-user-agent,x-amz-target,x-amz-acl,x-amz-version-id,x-localstack-target,x-amz-tagging
Access-Control-Expose-Headers: x-amz-version-id

<CreateBucketResponse xmlns="http://s3.amazonaws.com/doc/2006-03-01"><CreateBucketResponse><Bucket>hello</Bucket></CreateBucketResponse></CreateBucketResponse>%                                                                                                                

$ curl -i -XPUT http://localhost:4566/hello
HTTP/1.1 404 Not Found
Server: BaseHTTP/0.6 Python/3.8.2
Date: Wed, 20 May 2020 13:43:22 GMT
Content-Length: 21
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: HEAD,GET,PUT,POST,DELETE,OPTIONS,PATCH
Access-Control-Allow-Headers: authorization,content-type,content-md5,cache-control,x-amz-content-sha256,x-amz-date,x-amz-security-token,x-amz-user-agent,x-amz-target,x-amz-acl,x-amz-version-id,x-localstack-target,x-amz-tagging
Access-Control-Expose-Headers: x-amz-version-id

{"status": "running"}%
```

## Command used to start LocalStack

`localstack start`


Discussion:
is this issue and #2430 the same?

Resolving patch:
diff --git a/localstack/services/edge.py b/localstack/services/edge.py
--- a/localstack/services/edge.py
+++ b/localstack/services/edge.py
@@ -6,7 +6,7 @@
 from requests.models import Response
 from localstack import config
 from localstack.constants import HEADER_LOCALSTACK_TARGET, HEADER_LOCALSTACK_EDGE_URL, LOCALSTACK_ROOT_FOLDER
-from localstack.utils.common import run, is_root, TMP_THREADS
+from localstack.utils.common import run, is_root, TMP_THREADS, to_bytes
 from localstack.utils.common import safe_requests as requests
 from localstack.services.generic_proxy import ProxyListener, GenericProxy
 
@@ -41,10 +41,18 @@ def forward_request(self, method, path, data, headers):
             # detect S3 presigned URLs
             if 'AWSAccessKeyId=' in path or 'Signature=' in path:
                 port = config.PORT_S3
-            # assume that this is an S3 GET request with URL path `/<bucket>/<key ...>`
             # TODO: move S3 public URLs to a separate port/endpoint, OR check ACLs here first
-            if method == 'GET' and '/' in path.strip('/'):
+            stripped = path.strip('/')
+            if method == 'GET' and '/' in stripped:
+                # assume that this is an S3 GET request with URL path `/<bucket>/<key ...>`
                 port = config.PORT_S3
+            if stripped and '/' not in stripped:
+                if method == 'PUT':
+                    # assume that this is an S3 PUT bucket request with URL path `/<bucket>`
+                    port = config.PORT_S3
+                elif method == 'POST' and to_bytes('key=') in to_bytes(data or ''):
+                    # assume that this is an S3 POST request with form parameters in the body
+                    port = config.PORT_S3
 
         if not port:
             if api in ['', None, '_unknown_']:


Test patch:
diff --git a/tests/integration/test_edge.py b/tests/integration/test_edge.py
--- a/tests/integration/test_edge.py
+++ b/tests/integration/test_edge.py
@@ -1,14 +1,17 @@
+import io
 import unittest
+import requests
 from localstack import config
 from localstack.utils.aws import aws_stack
-from localstack.utils.common import short_uid, get_service_protocol
+from localstack.utils.common import short_uid, get_service_protocol, to_str
 from localstack.utils.bootstrap import is_api_enabled
 
 
 class TestEdgeAPI(unittest.TestCase):
 
     def test_invoke_apis_via_edge(self):
-        edge_url = '%s://localhost:%s' % (get_service_protocol(), config.EDGE_PORT)
+        edge_port = config.EDGE_PORT_HTTP or config.EDGE_PORT
+        edge_url = '%s://localhost:%s' % (get_service_protocol(), edge_port)
 
         if is_api_enabled('s3'):
             self._invoke_s3_via_edge(edge_url)
@@ -49,3 +52,17 @@ def _invoke_s3_via_edge(self, edge_url):
         result = client.head_bucket(Bucket=bucket_name)
         self.assertEqual(result['ResponseMetadata']['HTTPStatusCode'], 200)
         client.delete_bucket(Bucket=bucket_name)
+
+        bucket_name = 'edge-%s' % short_uid()
+        object_name = 'testobject'
+        bucket_url = '%s/%s' % (edge_url, bucket_name)
+        result = requests.put(bucket_url)
+        self.assertEqual(result.status_code, 200)
+        result = client.head_bucket(Bucket=bucket_name)
+        self.assertEqual(result['ResponseMetadata']['HTTPStatusCode'], 200)
+        headers = {'Content-Type': 'application/x-www-form-urlencoded'}
+        result = requests.post(bucket_url, data='key=%s&file=file_content_123' % object_name, headers=headers)
+        self.assertEqual(result.status_code, 204)
+        result = io.BytesIO()
+        client.download_fileobj(bucket_name, object_name, result)
+        self.assertEqual('file_content_123', to_str(result.getvalue()))