Repository: marshmallow-code/marshmallow

Issue:
RFC: Change the way we store metadata?
Users are often bit by the fact that fields store arbitrary keyword arguments as metadata. See https://github.com/marshmallow-code/marshmallow/issues/683.

> ...The reasons we use **kwargs instead of e.g. `metadata=` are mostly historical. The original decision was that storing kwargs 1) was more concise and 2) saved us from having to come up with an appropriate name... "metadata" didn't seem right because there are use cases where the things your storing aren't really metadata. At this point, it's not worth breaking the API.

> Not the best reasons, but I think it's not terrible. We've discussed adding a [whitelist of metadata keys](https://github.com/marshmallow-code/marshmallow/issues/683#issuecomment-385113845) in the past, but we decided it wasn't worth the added API surface.

_Originally posted by @sloria in https://github.com/marshmallow-code/marshmallow/issues/779#issuecomment-522283135_

Possible solutions:

1. Use `metadata=`.
2. Specify a whitelist of allowed metadata arguments.

Feedback welcome!


Discussion:
Solution 1. is preferable to 2., I think. That said, there are some use cases where it's awkward to call additional kwargs "metadata". `location` in webargs is one that comes to mind.

```python
# current API
"some_query_param": fields.Bool(location="query")
```

though we could probably wrap all the fields in webargs to take the additional `location` argument. 🤔 
I wanted to note that even webargs' `location` doesn't necessarily make the case against `metadata=...`. I was surprised/confused at first when I went looking for `location` in marshmallow and found no mention of it. At the cost of a little bit of verbosity, it would make it easier to understand how marshmallow is functioning.

Relatedly, the plan in https://github.com/marshmallow-code/webargs/issues/419 includes making `location=...` for webargs a thing of the past.
cc @jtrakk . This was your suggestion in https://github.com/marshmallow-code/marshmallow/issues/779#issuecomment-522282845 . I'm leaning towards this more and more.
+1 on this, IMHO biggest problem of `self.metadata=kwargs` is not that it's unexpected, but that it's not generating errors on wrong keyword arguments, which is pretty annoying due frequent api changes :-) so - you can find mistakes only later, all your typos in metadata field...
One one hand, I think it is better to specify `metadata=`. More explicit.

OTOH, this will make my models a bit more verbose:

```py
class MyModel(ma.Schema:
    some_int = ma.fields.Int(
        required=True,
        validate=ma.validate.OneOf([1, 2, 3]),
        metadata={"description": "This string explains what this is all about"}
    )
```

For such a use case, the shortcut of using extra kwargs as metadata is nice.

If we went with solution 2, users would be able to extend the whitelist with their own stuff. Apispec could extend it with the keyword arguments it expects (valid OpenAPI attributes) and we could even catch typos inside metadata, while solution 1 blindly accepts anything in metadata.

However, this would prevent accepting arbitrary attributes in metadata, which sucks. E.g. in apispec, we also accept any `"x-..."` attribute. So we'd need to genericize the whitelist to a callable mechanism. And we end up with a gas factory feature while we wanted to make thing simple.

Overall, perhaps the downside of 1 (model verbosity) is not that bad.
Agreed. Consider this accepted.

My plan is to deprecate `metadata=kwargs` in a later 3.x release. Let's let the dust settle on v3 before bombarding users with DeprecationWarnings 😅 

Patch:
diff --git a/src/marshmallow/fields.py b/src/marshmallow/fields.py
--- a/src/marshmallow/fields.py
+++ b/src/marshmallow/fields.py
@@ -101,7 +101,7 @@ class Field(FieldABC):
         its value will be present in the deserialized object. In the context of an
         HTTP API, this effectively marks the field as "read-only".
     :param dict error_messages: Overrides for `Field.default_error_messages`.
-    :param metadata: Extra arguments to be stored as metadata.
+    :param metadata: Extra information to be stored as field metadata.
 
     .. versionchanged:: 2.0.0
         Removed `error` parameter. Use ``error_messages`` instead.
@@ -160,7 +160,8 @@ def __init__(
         load_only: bool = False,
         dump_only: bool = False,
         error_messages: typing.Optional[typing.Dict[str, str]] = None,
-        **metadata
+        metadata: typing.Optional[typing.Mapping[str, typing.Any]] = None,
+        **additional_metadata
     ) -> None:
         self.default = default
         self.attribute = attribute
@@ -187,7 +188,16 @@ def __init__(
             raise ValueError("'missing' must not be set for required fields.")
         self.required = required
         self.missing = missing
-        self.metadata = metadata
+
+        metadata = metadata or {}
+        self.metadata = {**metadata, **additional_metadata}
+        if additional_metadata:
+            warnings.warn(
+                "Passing field metadata as a keyword arg is deprecated. Use the "
+                "explicit `metadata=...` argument instead.",
+                RemovedInMarshmallow4Warning,
+            )
+
         self._creation_index = Field._creation_index
         Field._creation_index += 1
 


Test patch:
diff --git a/tests/test_fields.py b/tests/test_fields.py
--- a/tests/test_fields.py
+++ b/tests/test_fields.py
@@ -197,11 +197,27 @@ def test_extra_metadata_may_be_added_to_field(self, FieldClass):  # noqa
             required=True,
             default=None,
             validate=lambda v: True,
-            description="foo",
-            widget="select",
+            metadata={"description": "foo", "widget": "select"},
         )
         assert field.metadata == {"description": "foo", "widget": "select"}
 
+    @pytest.mark.parametrize("FieldClass", ALL_FIELDS)
+    def test_field_metadata_added_in_deprecated_style_warns(self, FieldClass):  # noqa
+        # just the old style
+        with pytest.warns(DeprecationWarning):
+            field = FieldClass(description="Just a normal field.")
+            assert field.metadata["description"] == "Just a normal field."
+        # mixed styles
+        with pytest.warns(DeprecationWarning):
+            field = FieldClass(
+                required=True,
+                default=None,
+                validate=lambda v: True,
+                description="foo",
+                metadata={"widget": "select"},
+            )
+        assert field.metadata == {"description": "foo", "widget": "select"}
+
 
 class TestErrorMessages:
     class MyField(fields.Field):