Repository: marcosschroh/dataclasses-avroschema

Issue:
Avro schema field with key 'doc': ... should be rendered as pydantic.Field(description=...) when generating pydantic models from avro schemas
Hi, 

first of all many thanks for this great package, really happy I discovered it for my project!
I'm not sure if I missed an option here, but if not...

**Describe the bug**
When generating a pydantic model with`ModelGenerator().render_module` with argument `model_type=ModelType.AVRODANTIC.value`, if the avro schema has a field with the key ` "doc": "Add description here"`, then, in the pydantic model inheriting from `AvroBaseModel `, this field is rendered as ` pydantic.Field(metadata={"doc": "Add description here"})`. However, `metadata `isn't a valid parameter for `pydantic.Field`. Ideally, this should be rendered as `pydantic.Field(description="Add description here")`, as the argument `description` already exists in pydantic.

**To Reproduce**
See above - happens all the time with `doc` key in avro schemas

**Expected behavior**
A field in an avro schema with ` "doc": "Add description here"` should be rendered as `pydantic.Field(description="Add description here")` when using `ModelGenerator().render_module` with argument `model_type=ModelType.AVRODANTIC.value`.

Discussion:
@marcosschroh: Hi @chrisssch 

Thanks for your comment. You are right that `metadata` is not a valid `kwarg` in the `Field` function, however you can pass extra ones as it defines `extras/json_schema_extra` (**extra: Unpack[_EmptyKwargs])(https://docs.pydantic.dev/latest/api/fields/#pydantic.fields.Field). I made in this way because it cover all uses cases, `dataclasses, pydantic and faust`. 

I am not sure whether `docs hould be rendered as pydantic.Field(description=...)` as it does not cause any error and it works except the `mypy` warning

What do you expect to be the behavior when extra data needs to be supplied? It is possible to provide extra data to `avro fields` that it is used in the `schema`, for [example](https://marcosschroh.github.io/dataclasses-avroschema/fields_specification/#adding-custom-field-level-attributes)@chrisssch: Hi @marcosschroh ,

Thanks a lot for the clarification!
To your question: To be honest, I don't have any expectation re: extra data, as no such need arised in my use case. I was just wondering about the generated "metadata{doc: ...}" parameter, as there would be a corresponding argument with "description" in pydantic.Field. 
For my use case I just added a function now that removes "metadata" substring from the string generated by render_module to get rid of the mypy complaints - the field explanations are already in the avro schemas anyway if someone needs that documentation. I don't know if there's a built-in way in your awesome package for that that I just missed though.@marcosschroh: You're welcome. There is not a way to put the `avro doc` into the `pydantic Field description property` however it is possible ti implemented. From `pydantic class` -> `avro` works fine, so the other wat should work as well. I will try to come up with a PR soon@marcosschroh: @chrisssch 

The last version has the fixed that you wanted.@chrisssch: Oh wow, thanks a lot, I'll check it out asap!

Solution patch:
diff --git a/dataclasses_avroschema/model_generator/lang/python/avrodantic/avrodantic_model_generator.py b/dataclasses_avroschema/model_generator/lang/python/avrodantic/avrodantic_model_generator.py
index 0e7fa73..5ce7da8 100644
--- a/dataclasses_avroschema/model_generator/lang/python/avrodantic/avrodantic_model_generator.py
+++ b/dataclasses_avroschema/model_generator/lang/python/avrodantic/avrodantic_model_generator.py
@@ -3,6 +3,9 @@ from dataclasses import dataclass
 
 from dataclasses_avroschema.model_generator.lang.python import templates
 from dataclasses_avroschema.model_generator.lang.python.base import BaseGenerator
+from dataclasses_avroschema.model_generator.lang.python.pydantic.pydantic_model_generator import (
+    PydanticFieldRepresentation,
+)
 from dataclasses_avroschema.types import JsonDict
 
 
@@ -18,6 +21,7 @@ class AvroDanticModelGenerator(BaseGenerator):
 
         # Templates
         self.field_template = templates.pydantic_field_template
+        self.field_representation_class = PydanticFieldRepresentation
 
     def _resolve_type_from_metadata(self, *, field: JsonDict) -> typing.Optional[str]:
         """
diff --git a/dataclasses_avroschema/model_generator/lang/python/base.py b/dataclasses_avroschema/model_generator/lang/python/base.py
index b33a4dc..7df142f 100644
--- a/dataclasses_avroschema/model_generator/lang/python/base.py
+++ b/dataclasses_avroschema/model_generator/lang/python/base.py
@@ -79,6 +79,34 @@ class FieldRepresentation:
 
         return result
 
+    def get_field_metadata(self) -> typing.Optional[str]:
+        if self.metadata:
+            return f"metadata={self.metadata}"
+        return None
+
+    def add_field_properties(self, default_repr: str) -> str:
+        field_metadata_repr = self.get_field_metadata()
+
+        if field_metadata_repr or isinstance(self.default, (dict, list)):
+            dataclass_field_properties = [field_metadata_repr]
+
+            if isinstance(self.default, (dict, list)):
+                if self.default:
+                    dataclass_prop = f"default_factory=lambda: {default_repr}"
+                else:
+                    dataclass_prop = f"default_factory={default_repr}"
+
+                dataclass_field_properties.append(dataclass_prop)
+            else:
+                if self.default is not dataclasses.MISSING:
+                    dataclass_field_properties.append(f"default={default_repr}")
+
+            default_repr = self.render_dataclass_field(
+                properties=", ".join([prop for prop in dataclass_field_properties if prop])
+            )
+
+        return default_repr
+
     def get_field_default(self) -> str:
         """
         Returns the default value according to the field type
@@ -89,8 +117,6 @@ class FieldRepresentation:
             If the default is "bond" the method should return '"bond"' so the double quotes
             won't be scaped during the field render
         """
-        field_metadata = self.metadata or {}
-        field_metadata_repr = None
         default_repr = ""
 
         if self.default is dataclasses.MISSING:
@@ -139,27 +165,7 @@ class FieldRepresentation:
         else:
             default_repr = str(self.default)
 
-        dataclass_field_properties = []
-        if field_metadata:
-            field_metadata_repr = f"metadata={field_metadata}"
-
-        if field_metadata_repr or isinstance(self.default, (dict, list)):
-            dataclass_field_properties = [field_metadata_repr]
-
-            if isinstance(self.default, (dict, list)):
-                if self.default:
-                    dataclass_prop = f"default_factory=lambda: {default_repr}"
-                else:
-                    dataclass_prop = f"default_factory={default_repr}"
-
-                dataclass_field_properties.append(dataclass_prop)
-            else:
-                if self.default is not dataclasses.MISSING:
-                    dataclass_field_properties.append(f"default={default_repr}")
-
-            default_repr = self.render_dataclass_field(
-                properties=", ".join([prop for prop in dataclass_field_properties if prop])
-            )
+        default_repr = self.add_field_properties(default_repr)
 
         return default_repr
 
@@ -355,6 +361,7 @@ class BaseGenerator:
     field_identation: str = "\n    "
     imports: typing.Set[str] = field(default_factory=set)
     imports_dict: typing.Dict[str, str] = field(default_factory=dict)
+    field_representation_class: typing.Type[FieldRepresentation] = FieldRepresentation
 
     # extras is used fot extra code that is generated, for example Enums
     extras: typing.List[str] = field(default_factory=list)
@@ -507,7 +514,7 @@ class BaseGenerator:
             if name:
                 self.type_hint_clashes[name] = type_hint
 
-        return FieldRepresentation(
+        return self.field_representation_class(
             name=name,
             avro_type=avro_type,
             type_hint=type_hint,
diff --git a/dataclasses_avroschema/model_generator/lang/python/pydantic/pydantic_model_generator.py b/dataclasses_avroschema/model_generator/lang/python/pydantic/pydantic_model_generator.py
index c747a74..d952d4f 100644
--- a/dataclasses_avroschema/model_generator/lang/python/pydantic/pydantic_model_generator.py
+++ b/dataclasses_avroschema/model_generator/lang/python/pydantic/pydantic_model_generator.py
@@ -1,11 +1,52 @@
 import typing
-from dataclasses import dataclass
+from dataclasses import MISSING, dataclass
 
 from dataclasses_avroschema.model_generator.lang.python import templates
-from dataclasses_avroschema.model_generator.lang.python.base import BaseGenerator
+from dataclasses_avroschema.model_generator.lang.python.base import BaseGenerator, FieldRepresentation
 from dataclasses_avroschema.types import JsonDict
 
 
+@dataclass
+class PydanticFieldRepresentation(FieldRepresentation):
+    def get_field_metadata(self) -> typing.Optional[str]:
+        metadata = {k: v for k, v in self.metadata.items() if k != "doc"}
+        if metadata:
+            return f"metadata={metadata}"
+        return None
+
+    def add_field_properties(self, default_repr: str) -> str:
+        field_metadata_repr = self.get_field_metadata()
+
+        if "doc" in self.metadata:
+            doc = self.metadata["doc"]
+            description = f'description="{doc}"'
+
+            if field_metadata_repr:
+                field_metadata_repr = f"{field_metadata_repr}, {description}"
+            else:
+                field_metadata_repr = description
+
+        if field_metadata_repr or isinstance(self.default, (dict, list)):
+            dataclass_field_properties = [field_metadata_repr]
+
+            if isinstance(self.default, (dict, list)):
+                if self.default:
+                    dataclass_prop = f"default_factory=lambda: {default_repr}"
+                else:
+                    dataclass_prop = f"default_factory={default_repr}"
+
+                dataclass_field_properties.append(dataclass_prop)
+            else:
+                if self.default is not MISSING:
+                    dataclass_field_properties.append(f"default={default_repr}")
+
+            default_repr = self.render_dataclass_field(
+                properties=", ".join([prop for prop in dataclass_field_properties if prop])
+            )
+
+        return default_repr
+
+
 @dataclass
 class PydanticModelGenerator(BaseGenerator):
     def __post_init__(self) -> None:
@@ -18,6 +59,7 @@ class PydanticModelGenerator(BaseGenerator):
 
         # Templates
         self.field_template = templates.pydantic_field_template
+        self.field_representation_class = PydanticFieldRepresentation
 
     def _resolve_type_from_metadata(self, *, field: JsonDict) -> typing.Optional[str]:
         """


Test patch:
diff --git a/tests/model_generator/conftest.py b/tests/model_generator/conftest.py
index 8265394..f512d62 100644
--- a/tests/model_generator/conftest.py
+++ b/tests/model_generator/conftest.py
@@ -1339,6 +1339,11 @@ def with_fields_with_metadata() -> JsonDict:
                 "doc": "test",
                 "avro.java.string": "String",
             },
+            {
+                "name": "fieldwithdoc",
+                "type": "long",
+                "doc": "test",
+            },
         ],
     }
 
diff --git a/tests/model_generator/test_model_generator.py b/tests/model_generator/test_model_generator.py
index 103a7b6..fe65939 100644
--- a/tests/model_generator/test_model_generator.py
+++ b/tests/model_generator/test_model_generator.py
@@ -982,11 +982,12 @@ import dataclasses
 @dataclasses.dataclass
 class Message(AvroModel):
     someotherfield: int = dataclasses.field(metadata={'aliases': ['oldname'], 'doc': 'test', 'avro.java.string': 'String'})
+    fieldwithdoc: int = dataclasses.field(metadata={'doc': 'test'})
     fieldwithdefault: str = dataclasses.field(metadata={'avro.java.string': 'String'}, default="some default value")
 
     
     class Meta:
-        field_order = ['fieldwithdefault', 'someotherfield']
+        field_order = ['fieldwithdefault', 'someotherfield', 'fieldwithdoc']
 
 """
     model_generator = ModelGenerator()
diff --git a/tests/model_generator/test_model_pydantic_generator.py b/tests/model_generator/test_model_pydantic_generator.py
index 8af1067..088ec2e 100644
--- a/tests/model_generator/test_model_pydantic_generator.py
+++ b/tests/model_generator/test_model_pydantic_generator.py
@@ -137,6 +137,29 @@ class User(AvroBaseModel):
     assert result.strip() == expected_result.strip()
 
 
+def test_pydantic_with_fields_with_metadata(
+    with_fields_with_metadata: types.JsonDict,
+) -> None:
+    expected_result = """
+import pydantic
+
+
+
+class Message(pydantic.BaseModel):
+    someotherfield: int = pydantic.Field(metadata={'aliases': ['oldname'], 'avro.java.string': 'String'}, description="test")
+    fieldwithdoc: int = pydantic.Field(description="test")
+    fieldwithdefault: str = pydantic.Field(metadata={'avro.java.string': 'String'}, default="some default value")
+
+    
+    class Meta:
+        field_order = ['fieldwithdefault', 'someotherfield', 'fieldwithdoc']
+
+"""  # noqa: E501
+    model_generator = ModelGenerator()
+    result = model_generator.render(schema=with_fields_with_metadata, model_type=ModelType.PYDANTIC.value)
+    assert result.strip() == expected_result.strip()
+
+
 def test_avro_pydantic_primitive_types_as_defined_types(
     schema_primitive_types_as_defined_types: types.JsonDict,
 ) -> None:
@@ -175,14 +198,15 @@ import pydantic
 
 
 class Message(AvroBaseModel):
-    someotherfield: int = pydantic.Field(metadata={'aliases': ['oldname'], 'doc': 'test', 'avro.java.string': 'String'})
+    someotherfield: int = pydantic.Field(metadata={'aliases': ['oldname'], 'avro.java.string': 'String'}, description="test")
+    fieldwithdoc: int = pydantic.Field(description="test")
     fieldwithdefault: str = pydantic.Field(metadata={'avro.java.string': 'String'}, default="some default value")
 
     
     class Meta:
-        field_order = ['fieldwithdefault', 'someotherfield']
+        field_order = ['fieldwithdefault', 'someotherfield', 'fieldwithdoc']
 
-"""
+"""  # noqa: E501
     model_generator = ModelGenerator()
     result = model_generator.render(schema=with_fields_with_metadata, model_type=ModelType.AVRODANTIC.value)
     assert result.strip() == expected_result.strip()