Repository: getmoto/moto

Issue:
moto_server S3 HTTP PUT/GET
using moto 1.3.16

Not sure if moto actually supports this and I'm just being daft, or if this isn't supported to begin with.
I've tried searching through documentation and other issues/PRs with no luck.

I am running a standalone `moto_server s3` as I need to proxy NGINX HTTP GET/PUT requests to it, but for the purposes of getting-it-working I have cut out the NGINX middleman for now.
I also have some pytests which use `@mock_s3` for creating a test bucket.

I have a couple of issues:
- My pytest tests *seem* to work, successfully creating a bucket and `PUT`ing a file in the correct bucket, however with no contents according to `http://localhost:5000/moto-api`, failing the last stage of one of my tests:
```python
    data_input = b'some data 123 321'
    req = urllib.request.Request(url='localhost:5000/mybucket/test.txt', data=data_input, method='PUT')

    with urllib.request.urlopen(req) as f:
        assert f.status == 200  # PASSED
    
    req = urllib.request.Request(url='localhost:5000/mybucket/test.txt', method='GET')
    with urllib.request.urlopen(req) as f:
        assert f.status == 200   # PASSED

        data_response = f.read()
        assert data_response == data_input  # <- FAILURE: b'' != b'some data 123 321'
```
- If I manually PUT a file using `curl` I again get no contents when using `curl -X PUT localhost:5000/mybucket/test.txt -d 'test123'`, however when using `curl -X PUT localhost:5000/mybucket/test.txt --upload-file ./test.txt` it works, creating a file with *actual* contents for a change. The problem is if I try to access the file via HTTP GET (either through `curl` or my browser) I get a 403 error. As part of the bucket creation (using `@mock_s3`) I have tried attaching a bucket policy allowing [read-only permission for anonymous users](https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html#example-bucket-policies-use-case-2), however this has not made any difference.


TL;DR: I want to be able to PUT and GET files (and their contents) unauthenticated with REST/without boto3.
Just to clarify, all my tests **work** when proxying NGINX to an actual live S3 bucket (with v4 authentication).

Apologies if I haven't been too clear, I'm writing this after a day of tinkering not having achieved much.

On a completely unrelated note, I thought I'd use this opportunity to ask if `moto_server` has a daemon/background mode.


## Update
Turns out adding `req.add_header("Content-Type", "text/plain")` to the PUT request makes it all work.

In which case, this issue could now be a question about whether or not `moto_server` should accept/store data without a content-type. S3 REST allows it therefore I believe (granted, as a new user of moto in general) moto_server should accommodate this.




Discussion:
Hi @notmeta, thanks for raising this, and providing a workaround! You're right, if it works in S3 it should work in moto as well.
Going to mark this as a bug.

Validating/parsing a bucket policy is a feature request, but hasn't been implemented yet: https://github.com/spulec/moto/issues/2892

`moto_server` does not have a daemon mode build in - I typically just use `moto_server &`

Some notes for whoever picks this up: 
This if-statement needs an additional elif:
https://github.com/spulec/moto/blob/master/moto/s3/responses.py#L1101
The data comes in as a form, so we'd need to parse `request.form` as well.
(Similar to how this is done in a POST request: https://github.com/spulec/moto/blob/master/moto/s3/responses.py#L837)
Thanks for the response, glad I found something valid!

> `moto_server` does not have a daemon mode build in - I typically just use `moto_server &`

Ah okay - that's what I was going to do anyway in the event that such a thing didn't exist

Resolving patch:
diff --git a/moto/s3/responses.py b/moto/s3/responses.py
--- a/moto/s3/responses.py
+++ b/moto/s3/responses.py
@@ -1187,10 +1187,17 @@ def _key_response(self, request, full_url, headers):
         else:
             # Flask server
             body = request.data
-            # when the data is being passed as a file
-            if request.files and not body:
-                for _, value in request.files.items():
-                    body = value.stream.read()
+            if not body:
+                # when the data is being passed as a file
+                if request.files:
+                    for _, value in request.files.items():
+                        body = value.stream.read()
+                elif hasattr(request, "form"):
+                    # Body comes through as part of the form, if no content-type is set on the PUT-request
+                    # form = ImmutableMultiDict([('some data 123 321', '')])
+                    form = request.form
+                    for k, _ in form.items():
+                        body = k
 
         if body is None:
             body = b""


Test patch:
diff --git a/tests/test_s3/test_s3.py b/tests/test_s3/test_s3.py
--- a/tests/test_s3/test_s3.py
+++ b/tests/test_s3/test_s3.py
@@ -5186,6 +5186,22 @@ def test_object_headers():
     res.should.have.key("BucketKeyEnabled")
 
 
+if settings.TEST_SERVER_MODE:
+
+    @mock_s3
+    def test_upload_data_without_content_type():
+        bucket = "mybucket"
+        s3 = boto3.client("s3")
+        s3.create_bucket(Bucket=bucket)
+        data_input = b"some data 123 321"
+        req = requests.put("http://localhost:5000/mybucket/test.txt", data=data_input)
+        req.status_code.should.equal(200)
+
+        res = s3.get_object(Bucket=bucket, Key="test.txt")
+        data = res["Body"].read()
+        assert data == data_input
+
+
 @mock_s3
 def test_get_object_versions_with_prefix():
     bucket_name = "testbucket-3113"