Repository: pydantic/pydantic

Issue:
Subclasses of known types are not JSON serializable
# Feature Request/(Bug?)

Output of `python -c "import pydantic.utils; print(pydantic.utils.version_info())"`:
```
        pydantic version: 1.4
       pydantic compiled: False
            install path: /Users/step7212/.pyenv/versions/3.8.1/envs/fastdate/lib/python3.8/site-packages/pydantic
          python version: 3.8.1 (default, Jan 10 2020, 09:36:37)  [Clang 11.0.0 (clang-1100.0.33.16)]
                platform: macOS-10.14.6-x86_64-i386-64bit
optional deps. installed: []
```

I would like to use a package that subclasses a standard python library class, adding convenience features but maintaining the interface of it's inheritance. Currently, pydantic will error when attempting to serialize the subclass, as it is not known in `ENCODERS_BY_TYPE`: https://github.com/samuelcolvin/pydantic/blob/e3243d267b06bda2d7b2213a8bad70f82171033c/pydantic/json.py#L20-L40

Example:
```py
>>> import pendulum
>>> import pydantic

>>> class MyModel(pydantic.BaseModel):
...     date_field: pendulum.DateTime
... 
>>> m = MyModel(date_field=pendulum.now('UTC'))
>>> m
MyModel(date_field=DateTime(2020, 3, 2, 16, 44, 42, 977836, tzinfo=Timezone('UTC')))
>>> m.dict()
{'date_field': DateTime(2020, 3, 2, 16, 44, 42, 977836, tzinfo=Timezone('UTC'))}
>>> m.json()
Traceback (most recent call last):
  File "/Users/step7212/git/hub/pydantic/pydantic/json.py", line 57, in pydantic_encoder
    encoder = ENCODERS_BY_TYPE[type(obj)]
KeyError: <class 'pendulum.datetime.DateTime'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/step7212/git/hub/pydantic/pydantic/main.py", line 419, in json
    return self.__config__.json_dumps(data, default=encoder, **dumps_kwargs)
  File "/Users/step7212/.pyenv/versions/3.8.1/lib/python3.8/json/__init__.py", line 234, in dumps
    return cls(
  File "/Users/step7212/.pyenv/versions/3.8.1/lib/python3.8/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/Users/step7212/.pyenv/versions/3.8.1/lib/python3.8/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/Users/step7212/git/hub/pydantic/pydantic/json.py", line 59, in pydantic_encoder
    raise TypeError(f"Object of type '{obj.__class__.__name__}' is not JSON serializable")
TypeError: Object of type 'DateTime' is not JSON serializable
```

Since all pendulum types are children of the std `datetime` types, they should be serializable in the same way:
```py
>>> import pendulum
>>> import pydantic
>>> 
>>> class MyModel(pydantic.BaseModel):
...     date_field: pendulum.DateTime
... 
>>> m = MyModel(date_field=pendulum.now('UTC'))
>>> m
MyModel(date_field=DateTime(2020, 3, 2, 16, 42, 22, 403965, tzinfo=Timezone('UTC')))
>>> m.dict()
{'date_field': DateTime(2020, 3, 2, 16, 42, 22, 403965, tzinfo=Timezone('UTC'))}
>>> m.json()
'{"date_field": "2020-03-02T16:42:22.403965+00:00"}'
```
This is doable by iterating over the [method resolution order](https://docs.python.org/3/library/stdtypes.html#class.__mro__) attribute, and I've created a branch to demonstrate, which was used for the desired output above: https://github.com/samuelcolvin/pydantic/compare/master...StephenBrown2:encode_known_subclasses
<details>
<summary>Benchmarks for funsies, this actually turns out to be a little faster than before:</summary>

```
❯ git checkout master
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
❯ make benchmark-pydantic
python benchmarks/run.py pydantic-only
generating test cases...
    pydantic time=1.043s, success=50.10%
    pydantic time=0.982s, success=50.10%
    pydantic time=0.788s, success=50.10%
    pydantic time=0.732s, success=50.10%
    pydantic time=0.741s, success=50.10%
    pydantic best=0.732s, avg=0.857s, stdev=0.145s

    pydantic best=121.947μs/iter avg=142.857μs/iter stdev=24.164μs/iter version=1.4a1
❯ git checkout encode_known_subclasses
Switched to branch 'encode_known_subclasses'
Your branch is up to date with 'mine/encode_known_subclasses'.
❯ make benchmark-pydantic
python benchmarks/run.py pydantic-only
    pydantic time=0.973s, success=50.10%
    pydantic time=0.742s, success=50.10%
    pydantic time=0.731s, success=50.10%
    pydantic time=0.764s, success=50.10%
    pydantic time=0.734s, success=50.10%
    pydantic best=0.731s, avg=0.789s, stdev=0.104s

    pydantic best=121.800μs/iter avg=131.485μs/iter stdev=17.306μs/iter version=1.4a1
```

Perhaps this could also encourage the merge of using the `__class__` attribute instead of `type()` as well: https://github.com/samuelcolvin/pydantic/compare/samuelcolvin:master...use-class-attribute EDITED Rebased on current master: https://github.com/samuelcolvin/pydantic/compare/master...StephenBrown2:use-class-attribute

</details>

<details>
<summary>Aside on usage of __mro__:</summary>

I noticed that [`__mro__`](https://docs.python.org/3/library/stdtypes.html?highlight=__mro__#class.__mro__) is not used in many places, though it is referenced in a few places, mostly in mypy.py:

https://github.com/samuelcolvin/pydantic/blob/e3243d267b06bda2d7b2213a8bad70f82171033c/pydantic/class_validators.py#L331-L332
https://github.com/samuelcolvin/pydantic/blob/e3243d267b06bda2d7b2213a8bad70f82171033c/pydantic/main.py#L153
https://github.com/samuelcolvin/pydantic/blob/e3243d267b06bda2d7b2213a8bad70f82171033c/pydantic/mypy.py#L84
https://github.com/samuelcolvin/pydantic/blob/e3243d267b06bda2d7b2213a8bad70f82171033c/pydantic/mypy.py#L174
https://github.com/samuelcolvin/pydantic/blob/e3243d267b06bda2d7b2213a8bad70f82171033c/pydantic/mypy.py#L204
https://github.com/samuelcolvin/pydantic/blob/e3243d267b06bda2d7b2213a8bad70f82171033c/pydantic/mypy.py#L275

Rather, [typing's `__supertype__`](https://github.com/python/cpython/blob/ab6423fe2de0ed5f8a0dc86a9c7070229326b0f0/Lib/typing.py#L1929) is used, which I assumed was custom only because I couldn't find it in the [Python docs](https://docs.python.org/3/search.html?q=__supertype__) or anywhere outside of https://github.com/samuelcolvin/pydantic/blob/e3243d267b06bda2d7b2213a8bad70f82171033c/pydantic/typing.py#L180-L187

Would it be prudent/performant to replace the usage of `__supertype__` with a fast-breaking iteration of `__mro__` or perhaps more common usage of `lenient_subclass` in `typing.py`? and subsequently:
https://github.com/samuelcolvin/pydantic/blob/e3243d267b06bda2d7b2213a8bad70f82171033c/pydantic/fields.py#L361-L362
https://github.com/samuelcolvin/pydantic/blob/e3243d267b06bda2d7b2213a8bad70f82171033c/pydantic/schema.py#L641-L642
In particular, other checks for inheritance:
https://github.com/samuelcolvin/pydantic/blob/e3243d267b06bda2d7b2213a8bad70f82171033c/pydantic/validators.py#L554-L565
https://github.com/samuelcolvin/pydantic/blob/e3243d267b06bda2d7b2213a8bad70f82171033c/pydantic/utils.py#L103-L104
and of course, the tests:
https://github.com/samuelcolvin/pydantic/blob/e3243d267b06bda2d7b2213a8bad70f82171033c/tests/test_utils.py#L141-L153

</details>


Comments:

Patch:
diff --git a/docs/examples/exporting_models_json.py b/docs/examples/exporting_models_json.py
--- a/docs/examples/exporting_models_json.py
+++ b/docs/examples/exporting_models_json.py
@@ -1,6 +1,5 @@
-from datetime import datetime, timedelta
+from datetime import datetime
 from pydantic import BaseModel
-from pydantic.json import timedelta_isoformat
 
 class BarModel(BaseModel):
     whatever: int
@@ -11,16 +10,3 @@ class FooBarModel(BaseModel):
 
 m = FooBarModel(foo=datetime(2032, 6, 1, 12, 13, 14), bar={'whatever': 123})
 print(m.json())
-# (returns a str)
-class WithCustomEncoders(BaseModel):
-    dt: datetime
-    diff: timedelta
-
-    class Config:
-        json_encoders = {
-            datetime: lambda v: v.timestamp(),
-            timedelta: timedelta_isoformat,
-        }
-
-m = WithCustomEncoders(dt=datetime(2032, 6, 1), diff=timedelta(hours=100))
-print(m.json())
diff --git a/docs/examples/exporting_models_json_encoders.py b/docs/examples/exporting_models_json_encoders.py
new file mode 100644
--- /dev/null
+++ b/docs/examples/exporting_models_json_encoders.py
@@ -0,0 +1,16 @@
+from datetime import datetime, timedelta
+from pydantic import BaseModel
+from pydantic.json import timedelta_isoformat
+
+class WithCustomEncoders(BaseModel):
+    dt: datetime
+    diff: timedelta
+
+    class Config:
+        json_encoders = {
+            datetime: lambda v: v.timestamp(),
+            timedelta: timedelta_isoformat,
+        }
+
+m = WithCustomEncoders(dt=datetime(2032, 6, 1), diff=timedelta(hours=100))
+print(m.json())
diff --git a/docs/examples/exporting_models_json_subclass.py b/docs/examples/exporting_models_json_subclass.py
new file mode 100644
--- /dev/null
+++ b/docs/examples/exporting_models_json_subclass.py
@@ -0,0 +1,23 @@
+from datetime import date, timedelta
+from pydantic import BaseModel
+from pydantic.validators import int_validator
+
+class DayThisYear(date):
+    """
+    Contrived example of a special type of date that
+    takes an int and interprets it as a day in the current year
+    """
+    @classmethod
+    def __get_validators__(cls):
+        yield int_validator
+        yield cls.validate
+
+    @classmethod
+    def validate(cls, v: int):
+        return date.today().replace(month=1, day=1) + timedelta(days=v)
+
+class FooModel(BaseModel):
+    date: DayThisYear
+
+m = FooModel(date=300)
+print(m.json())
diff --git a/pydantic/json.py b/pydantic/json.py
--- a/pydantic/json.py
+++ b/pydantic/json.py
@@ -18,25 +18,27 @@ def isoformat(o: Union[datetime.date, datetime.time]) -> str:
 
 
 ENCODERS_BY_TYPE: Dict[Type[Any], Callable[[Any], Any]] = {
+    bytes: lambda o: o.decode(),
     Color: str,
+    datetime.date: isoformat,
+    datetime.datetime: isoformat,
+    datetime.time: isoformat,
+    datetime.timedelta: lambda td: td.total_seconds(),
+    Decimal: float,
+    Enum: lambda o: o.value,
+    frozenset: list,
+    GeneratorType: list,
     IPv4Address: str,
-    IPv6Address: str,
     IPv4Interface: str,
-    IPv6Interface: str,
     IPv4Network: str,
+    IPv6Address: str,
+    IPv6Interface: str,
     IPv6Network: str,
-    SecretStr: str,
+    Path: str,
     SecretBytes: str,
-    UUID: str,
-    datetime.datetime: isoformat,
-    datetime.date: isoformat,
-    datetime.time: isoformat,
-    datetime.timedelta: lambda td: td.total_seconds(),
+    SecretStr: str,
     set: list,
-    frozenset: list,
-    GeneratorType: list,
-    bytes: lambda o: o.decode(),
-    Decimal: float,
+    UUID: str,
 }
 
 
@@ -46,26 +48,29 @@ def pydantic_encoder(obj: Any) -> Any:
 
     if isinstance(obj, BaseModel):
         return obj.dict()
-    elif isinstance(obj, Enum):
-        return obj.value
-    elif isinstance(obj, Path):
-        return str(obj)
     elif is_dataclass(obj):
         return asdict(obj)
 
-    try:
-        encoder = ENCODERS_BY_TYPE[obj.__class__]
-    except KeyError:
-        raise TypeError(f"Object of type '{obj.__class__.__name__}' is not JSON serializable")
-    else:
+    # Check the class type and its superclasses for a matching encoder
+    for base in obj.__class__.__mro__[:-1]:
+        try:
+            encoder = ENCODERS_BY_TYPE[base]
+        except KeyError:
+            continue
         return encoder(obj)
+    else:  # We have exited the for loop without finding a suitable encoder
+        raise TypeError(f"Object of type '{obj.__class__.__name__}' is not JSON serializable")
 
 
 def custom_pydantic_encoder(type_encoders: Dict[Any, Callable[[Type[Any]], Any]], obj: Any) -> Any:
-    encoder = type_encoders.get(obj.__class__)
-    if encoder:
+    # Check the class type and its superclasses for a matching encoder
+    for base in obj.__class__.__mro__[:-1]:
+        try:
+            encoder = type_encoders[base]
+        except KeyError:
+            continue
         return encoder(obj)
-    else:
+    else:  # We have exited the for loop without finding a suitable encoder
         return pydantic_encoder(obj)
 
 


Test patch:
diff --git a/tests/test_json.py b/tests/test_json.py
--- a/tests/test_json.py
+++ b/tests/test_json.py
@@ -91,6 +91,41 @@ class Model(BaseModel):
     assert m.json(exclude={'b'}) == '{"a": 10.2, "c": 10.2, "d": {"x": 123, "y": "123"}}'
 
 
+def test_subclass_encoding():
+    class SubDate(datetime.datetime):
+        pass
+
+    class Model(BaseModel):
+        a: datetime.datetime
+        b: SubDate
+
+    m = Model(a=datetime.datetime(2032, 1, 1, 1, 1), b=SubDate(2020, 2, 29, 12, 30))
+    assert m.dict() == {'a': datetime.datetime(2032, 1, 1, 1, 1), 'b': SubDate(2020, 2, 29, 12, 30)}
+    assert m.json() == '{"a": "2032-01-01T01:01:00", "b": "2020-02-29T12:30:00"}'
+
+
+def test_subclass_custom_encoding():
+    class SubDate(datetime.datetime):
+        pass
+
+    class SubDelta(datetime.timedelta):
+        pass
+
+    class Model(BaseModel):
+        a: SubDate
+        b: SubDelta
+
+        class Config:
+            json_encoders = {
+                datetime.datetime: lambda v: v.strftime('%a, %d %b %C %H:%M:%S'),
+                datetime.timedelta: timedelta_isoformat,
+            }
+
+    m = Model(a=SubDate(2032, 1, 1, 1, 1), b=SubDelta(hours=100))
+    assert m.dict() == {'a': SubDate(2032, 1, 1, 1, 1), 'b': SubDelta(days=4, seconds=14400)}
+    assert m.json() == '{"a": "Thu, 01 Jan 20 01:01:00", "b": "P4DT4H0M0.000000S"}'
+
+
 def test_invalid_model():
     class Foo:
         pass