Repository: marshmallow-code/webargs

Issue:
Non UTF-8 json payload produce a 500 and a stack trace
Hi,

First of all thanks for thie library, while doing some testing with random payload we stumbled upon a corner case:

## Actual Behavior:
If non utf-8 data is sent with header Application/Json the `core`'s `parse_json` will raise a UnicodeDecodeError, which generates a 500.

## Expected Behavior
A `400 BAD REQUEST`

## Notes
- `Python 3.6`
- `webargs==5.5.1`

I would handle the problem my self but as you can see in the stack trace, my code is never reached so i would need a wrapper above `@use_args`

## Reproduce

CODE (From README example with `methods=` added):
```
from flask import Flask
from webargs import fields
from webargs.flaskparser import use_args

app = Flask(__name__)


@app.route("/", methods=['GET', 'POST'])
@use_args({"name": fields.Str(required=True)})
def index(args):
    return "Hello " + args["name"]


if __name__ == "__main__":
    app.run()
 
```

Curl invocation 
```
$ curl http://localhost:5000/ -H 'Content-Type: Application/Json' -d $(python -c "import os; import sys;  os.write(sys.stdout.fileno(), b'\xfe')")
```
Output
```
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>
```

Full Stack trace
```
.../.tox/py36/bin/python3.6 .../tests/webargs_poc.py
 * Serving Flask app "webargs_poc" (lazy loading)
 * Environment: production
   WARNING: Do not use the development server in a production environment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
[2019-09-23 09:07:54,383] ERROR in app: Exception on / [POST]
Traceback (most recent call last):
  File ".../.tox/py36/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File ".../.tox/py36/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File ".../.tox/py36/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File ".../.tox/py36/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File ".../.tox/py36/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File ".../.tox/py36/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/core.py", line 444, in wrapper
    error_headers=error_headers,
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/core.py", line 347, in parse
    schema=schema, req=req, locations=locations or self.locations
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/core.py", line 263, in _parse_request
    parsed_value = self.parse_arg(argname, field_obj, req, locations)
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/core.py", line 226, in parse_arg
    value = self._get_value(name, field, req=req, location=location)
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/core.py", line 202, in _get_value
    return function(req, name, argobj)
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/flaskparser.py", line 69, in parse_json
    self._cache["json"] = json_data = core.parse_json(data)
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/core.py", line 115, in parse_json
    s = s.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position 0: invalid start byte
127.0.0.1 - - [23/Sep/2019 09:07:54] "POST / HTTP/1.1" 500 -
```

Discussion:

Patch:
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 1b98a37..3eb6b9c 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -15,7 +15,7 @@ repos:
   - id: blacken-docs
     additional_dependencies: [black==19.3b0]
 - repo: https://github.com/pre-commit/mirrors-mypy
-  rev: v0.720
+  rev: v0.730
   hooks:
   - id: mypy
     language_version: python3
diff --git a/AUTHORS.rst b/AUTHORS.rst
index 363afc7..b1c8efd 100644
--- a/AUTHORS.rst
+++ b/AUTHORS.rst
@@ -40,3 +40,4 @@ Contributors (chronological)
 * Xiaoyu Lee <https://github.com/lee3164>
 * Jonathan Angelo <https://github.com/jangelo>
 * @zhenhua32 <https://github.com/zhenhua32>
+* Martin Roy <https://github.com/lindycoder>
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index d038031..224bb84 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -1,6 +1,14 @@
 Changelog
 ---------
 
+5.5.2 (unreleased)
+******************
+
+Bug fixes:
+
+* Handle ``UnicodeDecodeError`` when parsing JSON payloads (:issue:`427`).
+  Thanks :user:`lindycoder` for the catch and patch.
+
 5.5.1 (2019-09-15)
 ******************
 
diff --git a/src/webargs/aiohttpparser.py b/src/webargs/aiohttpparser.py
index b36428c..9e76e05 100644
--- a/src/webargs/aiohttpparser.py
+++ b/src/webargs/aiohttpparser.py
@@ -102,6 +102,9 @@ class AIOHTTPParser(AsyncParser):
                     return core.missing
                 else:
                     return self.handle_invalid_json_error(e, req)
+            except UnicodeDecodeError as e:
+                return self.handle_invalid_json_error(e, req)
+
             self._cache["json"] = json_data
         return core.get_value(json_data, name, field, allow_many_nested=True)
 
@@ -164,7 +167,11 @@ class AIOHTTPParser(AsyncParser):
         )
 
     def handle_invalid_json_error(
-        self, error: json.JSONDecodeError, req: Request, *args, **kwargs
+        self,
+        error: typing.Union[json.JSONDecodeError, UnicodeDecodeError],
+        req: Request,
+        *args,
+        **kwargs
     ) -> "typing.NoReturn":
         error_class = exception_map[400]
         messages = {"json": ["Invalid JSON body."]}
diff --git a/src/webargs/bottleparser.py b/src/webargs/bottleparser.py
index a03fb11..568dc65 100644
--- a/src/webargs/bottleparser.py
+++ b/src/webargs/bottleparser.py
@@ -47,6 +47,9 @@ class BottleParser(core.Parser):
                     return core.missing
                 else:
                     return self.handle_invalid_json_error(e, req)
+            except UnicodeDecodeError as e:
+                return self.handle_invalid_json_error(e, req)
+
             if json_data is None:
                 return core.missing
         return core.get_value(json_data, name, field, allow_many_nested=True)
diff --git a/src/webargs/core.py b/src/webargs/core.py
index b3eec6b..fe2f39b 100644
--- a/src/webargs/core.py
+++ b/src/webargs/core.py
@@ -112,7 +112,14 @@ def get_value(data, name, field, allow_many_nested=False):
 
 def parse_json(s, encoding="utf-8"):
     if isinstance(s, bytes):
-        s = s.decode(encoding)
+        try:
+            s = s.decode(encoding)
+        except UnicodeDecodeError as e:
+            raise json.JSONDecodeError(
+                "Bytes decoding error : {}".format(e.reason),
+                doc=str(e.object),
+                pos=e.start,
+            )
     return json.loads(s)
 
 


Test patch:
diff --git a/src/webargs/testing.py b/src/webargs/testing.py
index 2d07684..922bc47 100644
--- a/src/webargs/testing.py
+++ b/src/webargs/testing.py
@@ -117,6 +117,18 @@ class CommonTestCase(object):
         text = u"øˆƒ£ºº∆ƒˆ∆"
         assert testapp.post_json("/echo", {"name": text}).json == {"name": text}
 
+    # https://github.com/marshmallow-code/webargs/issues/427
+    def test_parse_json_with_nonutf8_chars(self, testapp):
+        res = testapp.post(
+            "/echo",
+            b"\xfe",
+            headers={"Accept": "application/json", "Content-Type": "application/json"},
+            expect_errors=True,
+        )
+
+        assert res.status_code == 400
+        assert res.json == {"json": ["Invalid JSON body."]}
+
     def test_validation_error_returns_422_response(self, testapp):
         res = testapp.post("/echo", {"name": "b"}, expect_errors=True)
         assert res.status_code == 422
diff --git a/tests/test_falconparser.py b/tests/test_falconparser.py
index 7162a0d..d6092c7 100644
--- a/tests/test_falconparser.py
+++ b/tests/test_falconparser.py
@@ -16,6 +16,18 @@ class TestFalconParser(CommonTestCase):
     def test_use_args_hook(self, testapp):
         assert testapp.get("/echo_use_args_hook?name=Fred").json == {"name": "Fred"}
 
+    # https://github.com/marshmallow-code/webargs/issues/427
+    def test_parse_json_with_nonutf8_chars(self, testapp):
+        res = testapp.post(
+            "/echo",
+            b"\xfe",
+            headers={"Accept": "application/json", "Content-Type": "application/json"},
+            expect_errors=True,
+        )
+
+        assert res.status_code == 400
+        assert res.json["errors"] == {"json": ["Invalid JSON body."]}
+
     # https://github.com/sloria/webargs/issues/329
     def test_invalid_json(self, testapp):
         res = testapp.post(