Repository: pydantic/pydantic

Issue:
Cannot parse nested object with custom root type
### Checks

* [x] I added a descriptive title to this issue
* [x] I have searched (google, github) for similar issues and couldn't find anything
* [x] I have read and followed [the docs](https://pydantic-docs.helpmanual.io/) and couldn't find an answer
* [x] After submitting this, I commit to one of:
    * Look through open issues and helped at least one other person
    * Hit the "watch" button on this repo to receive notifications and I commit to help at least 2 people that ask questions in the future
    * Implement a Pull Request for a confirmed bug

<!-- Sorry to sound so draconian, but every second saved replying to issues is time spend improving pydantic :-) -->

# Question

Output of `python -c "import pydantic.utils; print(pydantic.utils.version_info())"`:
```
python -c "import pydantic.utils; print(pydantic.utils.version_info())"
             pydantic version: 1.7.2
            pydantic compiled: True
                 install path: <path>/venv/lib/python3.7/site-packages/pydantic
               python version: 3.7.7 (default, Nov  3 2020, 00:02:20)  [Clang 12.0.0 (clang-1200.0.32.21)]
                     platform: Darwin-19.6.0-x86_64-i386-64bit
     optional deps. installed: []
```
Hello, thank you for the library - I have used it in production across multiple applications now! I am trying to parse a JSON with a nested JSON needing a custom root type of `Dict` - but get an error as shown in the snippet. I tried to check thru old bugs and the fixes for them like https://github.com/samuelcolvin/pydantic/pull/1253 and https://github.com/samuelcolvin/pydantic/issues/1773 and verified that all fixes are already in the version of the code I have pulled, but still don't see the behavior fixed. Can someone help point what I'm doing wrong?

Thanks!

```py
import pydantic
import json

class Player(BaseModel):
    player: str
    number: int
    pos: str

class Lineup(BaseModel):
    coach: str
    startXI: List[Player]
    substitutes: List[Player]


class Lineups(BaseModel):
    __root__: Dict[str, Lineup]


class LineupsResponse(BaseModel):
    results: int
    lineUps: Lineups


class LineupsApiResponse(BaseModel):
    api: LineupsResponse

if __name__ == "__main__":
    text = '{"api":{"results":2,"lineUps":{"ManchesterCity":{"coach":"Guardiola","coach_id":4,"formation":"4-3-3","startXI":[{"team_id":50,"player_id":617,"player":"Ederson","number":31,"pos":"G"}],"substitutes":[{"team_id":50,"player_id":50828,"player":"ZackSteffen","number":13,"pos":"G"}]},"Liverpool":{"coach":"J.Klopp","coach_id":1,"formation":"4-2-2-2","startXI":[{"team_id":40,"player_id":280,"player":"Alisson","number":1,"pos":"G"}],"substitutes":[{"team_id":40,"player_id":18812,"player":"Adrián","number":13,"pos":"G"},{"team_id":40,"player_id":127472,"player":"NathanielPhillips","number":47,"pos":"D"},{"team_id":40,"player_id":293,"player":"CurtisJones","number":17,"pos":"M"}]}}}}'
    try:
        response: LineupsApiResponse = LineupsApiResponse.parse_obj(json.loads(text))
    except Exception as e:
        print(e)
    try:
        response2: LineupsApiResponse = LineupsApiResponse.parse_raw(text)
    except Exception as e:
        print(e)
```
Output:
```
1 validation error for LineupsApiResponse
api -> lineUps -> __root__
  field required (type=value_error.missing)
1 validation error for LineupsApiResponse
api -> lineUps -> __root__
  field required (type=value_error.missing)
```



Discussion:
Hello @sohama4 
You're right `__root__` is only supported [at parent level](https://github.com/samuelcolvin/pydantic/blob/master/pydantic/main.py#L505) and is not handled properly for nested models.

To support this we could add this logic directly
- in `BaseModel.validate` but it wouldn't work with `construct` for example
- in `BaseModel.__init__` but it would cost a bit of running time

I run the benchmark and it doesn't seem to have any effect on the performance so the [latter solution](https://github.com/PrettyWood/pydantic/commit/ba6d3b4d0cfe5227223c12ae2336fb1f2566bcf2) could be a good solution. @samuelcolvin WDYT?

If you want to try it out already @sohama4 I would go with something like this

```python
import json
from typing import Any, Dict, List

from pydantic import BaseModel as PydanticBaseModel
from pydantic.utils import ROOT_KEY


class BaseModel(PydanticBaseModel):
    def __init__(__pydantic_self__, **data: Any) -> None:
        if __pydantic_self__.__custom_root_type__ and data.keys() != {ROOT_KEY}:
            data = {ROOT_KEY: data}
        super().__init__(**data)

...
```

Hope it helps!
@PrettyWood - thank you for the response, I will try it out soon, and will close this out ASAP. I think the docs must mention this behavior, so I left a comment on the PR you linked.
Similar issue with respect to `.from_orm()`. Solution advised by @PrettyWood doesn't work for this case since `validate_model` is called before constructor.

Worked this around like this:
```python
class BaseCustomRootModel(BaseModel):

    @classmethod
    def from_orm(cls, obj: Any):
        if cls.__custom_root_type__:
            class Wrapper:
                __slots__ = ('__root__',)

                def __init__(self, obj):
                    self.__root__ = obj

            return super().from_orm(Wrapper(obj))

        return super().from_orm(obj)
```

```
             pydantic version: 1.7.3
            pydantic compiled: False
                 install path: /home/user/.local/share/virtualenvs/project-hash/lib/python3.6/site-packages/pydantic
               python version: 3.6.8 (default, Feb 11 2019, 08:59:55)  [GCC 8.2.1 20181127]
                     platform: Linux-5.9.11-zen2-1-zen-x86_64-with-arch
     optional deps. installed: ['email-validator']
```
@scorpp What about [this solution](https://github.com/samuelcolvin/pydantic/issues/1193#issuecomment-753652412)? It should work with everything above + orm if I'm not mistaken
So something like this
```python
from pydantic import BaseModel as PydanticBaseModel

class BaseModel(PydanticBaseModel):
    @classmethod
    def validate(cls, value):
        if cls.__custom_root_type__:
            return cls.parse_obj(value)
        return super().validate(value)
```
@PrettyWood no, this doesn't work. in `BaseModel.from_orm` in `main.py:570` `m = cls.__new__(cls)` returns a model with no `__root__` set. and then the call to `validate_model` on next line fails.

Also your option with overriding `validate` turn execution to different path - `parse_obj` instead of `from_orm`. Meaning that nested ORM models won't work anyway, since `parse_obj` will not accept regular objects.

I think that my solution (or alike) should be incorporated into `from_orm` of the `BaseModel`. Similarly to `parse_obj`.
Ah yes sorry I wrote that on my phone yesterday night and should have paid more attention on the `from_orm` implementation.
I guess the fix is trivial if you can directly change the code like https://github.com/samuelcolvin/pydantic/pull/2237
WDYT @scorpp ?

Resolving patch:
diff --git a/pydantic/main.py b/pydantic/main.py
--- a/pydantic/main.py
+++ b/pydantic/main.py
@@ -509,12 +509,18 @@ def json(
         return self.__config__.json_dumps(data, default=encoder, **dumps_kwargs)
 
     @classmethod
-    def parse_obj(cls: Type['Model'], obj: Any) -> 'Model':
+    def _enforce_dict_if_root(cls, obj: Any) -> Any:
         if cls.__custom_root_type__ and (
             not (isinstance(obj, dict) and obj.keys() == {ROOT_KEY}) or cls.__fields__[ROOT_KEY].shape == SHAPE_MAPPING
         ):
-            obj = {ROOT_KEY: obj}
-        elif not isinstance(obj, dict):
+            return {ROOT_KEY: obj}
+        else:
+            return obj
+
+    @classmethod
+    def parse_obj(cls: Type['Model'], obj: Any) -> 'Model':
+        obj = cls._enforce_dict_if_root(obj)
+        if not isinstance(obj, dict):
             try:
                 obj = dict(obj)
             except (TypeError, ValueError) as e:
@@ -662,14 +668,13 @@ def __get_validators__(cls) -> 'CallableGenerator':
 
     @classmethod
     def validate(cls: Type['Model'], value: Any) -> 'Model':
+        value = cls._enforce_dict_if_root(value)
         if isinstance(value, dict):
             return cls(**value)
         elif isinstance(value, cls):
             return value.copy()
         elif cls.__config__.orm_mode:
             return cls.from_orm(value)
-        elif cls.__custom_root_type__:
-            return cls.parse_obj(value)
         else:
             try:
                 value_as_dict = dict(value)


Test patch:
diff --git a/tests/test_main.py b/tests/test_main.py
--- a/tests/test_main.py
+++ b/tests/test_main.py
@@ -1120,6 +1120,60 @@ class MyModel(BaseModel):
     ]
 
 
+def test_parse_obj_nested_root():
+    class Pokemon(BaseModel):
+        name: str
+        level: int
+
+    class Pokemons(BaseModel):
+        __root__: List[Pokemon]
+
+    class Player(BaseModel):
+        rank: int
+        pokemons: Pokemons
+
+    class Players(BaseModel):
+        __root__: Dict[str, Player]
+
+    class Tournament(BaseModel):
+        players: Players
+        city: str
+
+    payload = {
+        'players': {
+            'Jane': {
+                'rank': 1,
+                'pokemons': [
+                    {
+                        'name': 'Pikachu',
+                        'level': 100,
+                    },
+                    {
+                        'name': 'Bulbasaur',
+                        'level': 13,
+                    },
+                ],
+            },
+            'Tarzan': {
+                'rank': 2,
+                'pokemons': [
+                    {
+                        'name': 'Jigglypuff',
+                        'level': 7,
+                    },
+                ],
+            },
+        },
+        'city': 'Qwerty',
+    }
+
+    tournament = Tournament.parse_obj(payload)
+    assert tournament.city == 'Qwerty'
+    assert len(tournament.players.__root__) == 2
+    assert len(tournament.players.__root__['Jane'].pokemons.__root__) == 2
+    assert tournament.players.__root__['Jane'].pokemons.__root__[0].name == 'Pikachu'
+
+
 def test_untouched_types():
     from pydantic import BaseModel