Repository: marshmallow-code/webargs

Issue:
Automatically trim leading/trailing whitespace from argument values
Does webargs provide any clean way to do this? I guess leading/trailing whitespace are almost never something you want (especially when having required fields that must not be empty)...


Discussion:
I don't think we'll be adding this into webargs core. It's simple enough to override `parse_arg` for this use case.

```python
from webargs.flaskparser import FlaskParser

class CustomParser(FlaskParser):
    def parse_arg(self, name, field, req, locations=None):
        ret = super().parse_arg(name, field, req, locations=locations)
        if hasattr(ret, 'strip'):
            return ret.strip()
        return ret

parser = CustomParser()
use_args = parser.use_args
use_kwargs = parser.use_kwargs
```
This doesn't work anymore in the latest version, since `parse_arg` is gone. I could override `_load_location_data` and do it in there, but that's an internal API. Is there a better way for it?

```python
def strip_whitespace(value):
    if isinstance(value, str):
        value = value.strip()
    elif isinstance(value, dict):
        return {k: strip_whitespace(v) for k, v in value.items()}
    elif isinstance(value, (list, set)):
        return type(value)(map(strip_whitespace, value))
    return value


class StrippingFlaskParser(FlaskParser):
    def _load_location_data(self, **kwargs):
        data = super()._load_location_data(**kwargs)
        return strip_whitespace(data)
```
@sloria is the workaround im my previous comment here still the way to go in v6/v7?
Actually, this no longer works due to the MultiDictProxy and it does not seem straightforward at all to get this to work without ugly hacks. Maybe it'd be a good time to revisit supporting this in the core?
```python
def _strip_whitespace(value):
    if isinstance(value, str):
        value = value.strip()
    elif isinstance(value, MultiDict):
        return type(value)((k, _strip_whitespace(v)) for k, vals in value.lists() for v in vals)
    elif isinstance(value, dict):
        return {k: strip_whitespace(v) for k, v in value.items()}
    elif isinstance(value, (list, set)):
        return type(value)(map(strip_whitespace, value))
    return value



class StrippingFlaskParser(FlaskParser):
    def load_querystring(self, req, schema):
        return MultiDictProxy(_strip_whitespace(req.args), schema)

    def load_form(self, req, schema):
        return MultiDictProxy(_strip_whitespace(req.form), schema)

    def load_json(self, req, schema):
        return _strip_whitespace(super().load_json(req, schema))
```

this seems to work. but feel much more fragile than the old solution with `parse_arg`
Reopening, but can't give this the attention it requires at this very moment. Perhaps @lafrech or @sirosen can give this some thought
My instinct when reading this is that stripping whitespace would be the job of the fields in use in your schemas.

Something like
```python
class StrippedString(fields.String):
    def _deserialize(self, value, attr, data, **kwargs):
        value = super()._deserialize(value, attr, data, **kwargs)
        return value.strip()
```

That works for some simple cases, but obviously is not as easy to globally apply as the 5.x solution of changing the way that arg parsing works.

*If* you're only using dict-schemas, you can set a custom schema class on your parser, and then you should be able to do it like so:
```python
# modified slightly from @ThiefMaster's example, but untested
def _strip_whitespace(value):
    if isinstance(value, str):
        value = value.strip()
    # you'll be getting a MultiDictProxy here potentially, but it should work
    elif isinstance(value, Mapping):
        return {k: strip_whitespace(value[k]) for k in value}
    elif isinstance(value, (list, set)):
        return type(value)(map(strip_whitespace, value))
    return value

class StrippingSchema(Schema):
    @pre_load
    def strip_whitespace(self, data, **kwargs):
        return _strip_whitespace(data)

class StrippingFlaskParser(FlaskParser):
    # or use `schema_class=...` when instantiating a parser
    DEFAULT_SCHEMA_CLASS = StrippingSchema
```

I'm pretty sure that works if you look at [`MultiDictProxy.__getitem__`](https://github.com/marshmallow-code/webargs/blob/4cb3b19cb16a7717eeb0efb0a8a6000d5c63c5b5/src/webargs/multidictproxy.py#L29-L41).
But it only works for dict-schemas. For any cases where a full Schema class is used, it won't apply.

So I'm wondering what the right approach is here.

This "feels" to me like a pre-load hook, but applied on a per-parser basis, rather than a per-schema basis.
Should we allow users to define a "parser pre-load hook"?

I'm imagining something in the Parser like
```python
location_data = self._load_location_data(
    schema=schema, req=req, location=location
)
if self.pre_load_hook:  # <---- New!
    location_data = self.pre_load_hook(location_data, schema=schema, req=req, location=location)
data = schema.load(location_data, **load_kwargs)
```

I'm still thinking about this, but sharing my thoughts thusfar. I don't think this leads towards parser-post-load hooks or anything like that -- the problem is unique to pre-load because what's wanted is something which goes between `_load_location_data` and `schema.load()`, and you can't do it by just subclassing and wrapping `parse()`.
> My instinct when reading this is that stripping whitespace would be the job of the fields in use in your schemas.

While that would work, it requires being much more explicit all the time - so not amazing. And in almost all cases there's no interest point in having surrounding whitespace (the only case that comes to my mind where it may be useful is for a markdown field where someone wants to put an indented code block at the very beginning).

> But it only works for dict-schemas. For any cases where a full Schema class is used, it won't apply.

Hm, that may not be such a big problem since one could easily use a custom base schema and inherit all their schemas from that one. Drawback; AFAIK the execution order when having multiple `pre_load` hooks is undefined. So having such a hook in the base schema would not guarantee that it doesn't run before some other more specific `pre_load` hook..
I assume that solving this using custom fields is simply too laborious and too error prone for your use-case.

And I've run into exactly that issue with pre-load hooks in the past: having a base schema with `pre_load` or `post_load`  is usually safe only if no child schema defines hooks because marshmallow doesn't guarantee hook order.

How would you feel about the possibility of this being a "parser pre-load hook"? I'm imagining some usage like
```python
@parser.pre_load
def strip_whitespace(data, schema, req, location):
    if location == "json":  # as an example of the possibilities this opens up, skip json
        return data
    return _strip_whitespace(data)
```


I'm more keen on this + documentation covering `strip_whitespace` as an example than I am on something like adding `Parser.strip_whitespace = True` and making it do this. It's more flexible and doesn't bake in as many details into webargs.

Maybe it needs a better/less confusing name than "pre_load", and maybe I'm over-engineering here. Hard for me to see! 😅 
Looks like a good solution :) And it's more flexible than a hardcoded "strip whitespace" feature.

Resolving patch:
diff --git a/src/webargs/core.py b/src/webargs/core.py
--- a/src/webargs/core.py
+++ b/src/webargs/core.py
@@ -322,7 +322,10 @@ def parse(
             location_data = self._load_location_data(
                 schema=schema, req=req, location=location
             )
-            data = schema.load(location_data, **load_kwargs)
+            preprocessed_data = self.pre_load(
+                location_data, schema=schema, req=req, location=location
+            )
+            data = schema.load(preprocessed_data, **load_kwargs)
             self._validate_arguments(data, validators)
         except ma.exceptions.ValidationError as error:
             self._on_validation_error(
@@ -523,6 +526,15 @@ def handle_error(error, req, schema, *, error_status_code, error_headers):
         self.error_callback = func
         return func
 
+    def pre_load(
+        self, location_data: Mapping, *, schema: ma.Schema, req: Request, location: str
+    ) -> Mapping:
+        """A method of the parser which can transform data after location
+        loading is done. By default it does nothing, but users can subclass
+        parsers and override this method.
+        """
+        return location_data
+
     def _handle_invalid_json_error(
         self,
         error: typing.Union[json.JSONDecodeError, UnicodeDecodeError],


Test patch:
diff --git a/tests/test_core.py b/tests/test_core.py
--- a/tests/test_core.py
+++ b/tests/test_core.py
@@ -1,4 +1,5 @@
 import datetime
+import typing
 from unittest import mock
 
 import pytest
@@ -37,6 +38,9 @@ class MockRequestParser(Parser):
     def load_querystring(self, req, schema):
         return self._makeproxy(req.query, schema)
 
+    def load_form(self, req, schema):
+        return MultiDictProxy(req.form, schema)
+
     def load_json(self, req, schema):
         return req.json
 
@@ -1224,3 +1228,84 @@ class CustomParser(Parser):
     p = CustomParser()
     ret = p.parse(argmap, web_request)
     assert ret == {"value": "hello world"}
+
+
+def test_parser_pre_load(web_request):
+    class CustomParser(MockRequestParser):
+        # pre-load hook to strip whitespace from query params
+        def pre_load(self, data, *, schema, req, location):
+            if location == "query":
+                return {k: v.strip() for k, v in data.items()}
+            return data
+
+    parser = CustomParser()
+
+    # mock data for both query and json
+    web_request.query = web_request.json = {"value": " hello "}
+    argmap = {"value": fields.Str()}
+
+    # data gets through for 'json' just fine
+    ret = parser.parse(argmap, web_request)
+    assert ret == {"value": " hello "}
+
+    # but for 'query', the pre_load hook changes things
+    ret = parser.parse(argmap, web_request, location="query")
+    assert ret == {"value": "hello"}
+
+
+# this test is meant to be a run of the WhitspaceStrippingFlaskParser we give
+# in the docs/advanced.rst examples for how to use pre_load
+# this helps ensure that the example code is correct
+# rather than a FlaskParser, we're working with the mock parser, but it's
+# otherwise the same
+def test_whitespace_stripping_parser_example(web_request):
+    def _strip_whitespace(value):
+        if isinstance(value, str):
+            value = value.strip()
+        elif isinstance(value, typing.Mapping):
+            return {k: _strip_whitespace(value[k]) for k in value}
+        elif isinstance(value, (list, tuple)):
+            return type(value)(map(_strip_whitespace, value))
+        return value
+
+    class WhitspaceStrippingParser(MockRequestParser):
+        def pre_load(self, location_data, *, schema, req, location):
+            if location in ("query", "form"):
+                ret = _strip_whitespace(location_data)
+                return ret
+            return location_data
+
+    parser = WhitspaceStrippingParser()
+
+    # mock data for query, form, and json
+    web_request.form = web_request.query = web_request.json = {"value": " hello "}
+    argmap = {"value": fields.Str()}
+
+    # data gets through for 'json' just fine
+    ret = parser.parse(argmap, web_request)
+    assert ret == {"value": " hello "}
+
+    # but for 'query' and 'form', the pre_load hook changes things
+    for loc in ("query", "form"):
+        ret = parser.parse(argmap, web_request, location=loc)
+        assert ret == {"value": "hello"}
+
+    # check that it applies in the case where the field is a list type
+    # applied to an argument (logic for `tuple` is effectively the same)
+    web_request.form = web_request.query = web_request.json = {
+        "ids": [" 1", "3", " 4"],
+        "values": [" foo  ", " bar"],
+    }
+    schema = Schema.from_dict(
+        {"ids": fields.List(fields.Int), "values": fields.List(fields.Str)}
+    )
+    for loc in ("query", "form"):
+        ret = parser.parse(schema, web_request, location=loc)
+        assert ret == {"ids": [1, 3, 4], "values": ["foo", "bar"]}
+
+    # json loading should also work even though the pre_load hook above
+    # doesn't strip whitespace from JSON data
+    #   - values=[" foo  ", ...]  will have whitespace preserved
+    #   - ids=[" 1", ...]  will still parse okay because "  1" is valid for fields.Int
+    ret = parser.parse(schema, web_request, location="json")
+    assert ret == {"ids": [1, 3, 4], "values": [" foo  ", " bar"]}