Repository: pydantic/pydantic

Issue:
Nested Models with optional members fail on 1.6.1
# Bug

Defining a model with Optional nested recursive submodels is throwing a required value error when validating the model.  The nested models also contain nested submodels that container other recursive models potentially.  This works on 1.6.0 and all versions prior that we have been using the last year. It only breaks when we move to version 1.6.1

Here is a sample of the model that we are using with basic one letter named attributes.
```
class Spec(BaseModel):
    spec_fields: List[str] = Field(..., alias="fields")
    filter: Optional[str]
    sort: Optional[str]


class PSpec(Spec):
    g: Optional[GSpec]

class GSpec(Spec):
    p: Optional[PSpec]

class Filter(BaseModel):
    g: Optional[GSpec]
    p: Optional[PSpec]
```

Trying with the following values will generate an error
```
        data = {"p": {"sort": "some_field:asc", "fields": []}}
        result = Filter(**data)
```
The error generated is:
```
E   pydantic.error_wrappers.ValidationError: 1 validation error for Filter
E   p -> g
E     field required (type=value_error.missing)
```

It seems to be disregarding the Optional attribute for GSpec field in the nested PSpec model.
Same code on 1.6.0 works as expected.

Thanks!





Output of `python -c "import pydantic.utils; print(pydantic.utils.version_info())"`:
```
pydantic version: 1.6.1
            pydantic compiled: True
                 install path: /Users/..../.local/share/virtualenvs/..../lib/python3.7/site-packages/pydantic
               python version: 3.7.3 (default, Mar 27 2019, 09:23:15)  [Clang 10.0.1 (clang-1001.0.46.3)]
                     platform: Darwin-19.5.0-x86_64-i386-64bit
     optional deps. installed: ['typing-extensions', 'email-validator']
```
<!-- or if you're using pydantic prior to v1.3, manually include: OS, python version and pydantic version -->

<!-- Please read the [docs](https://pydantic-docs.helpmanual.io/) and search through issues to
confirm your bug hasn't already been reported. -->

<!-- Where possible please include a self-contained code snippet describing your bug: -->

```py
import pydantic

...
```



Discussion:
Hi @roycollins 
I can't reproduce your issue with the code sample. I reproduced the (almost) same environment

```
             pydantic version: 1.6.1
            pydantic compiled: True
                 install path: /Users/.../.pyenv/versions/3.7.7/envs/.../lib/python3.7/site-packages/pydantic
               python version: 3.7.7 (default, Mar 18 2020, 12:01:21)  [Clang 10.0.1 (clang-1001.0.46.4)]
                     platform: Darwin-18.7.0-x86_64-i386-64bit
     optional deps. installed: ['typing-extensions', 'email-validator']
```

and just added `PSpec.update_forward_refs()` as it is impossible to run without it.

```python
from pydantic import BaseModel, Field
from typing import List, Optional


class Spec(BaseModel):
    spec_fields: List[str] = Field(..., alias="fields")
    filter: Optional[str]
    sort: Optional[str]


class PSpec(Spec):
    g: Optional['GSpec']

class GSpec(Spec):
    p: Optional[PSpec]

PSpec.update_forward_refs()

class Filter(BaseModel):
    g: Optional[GSpec]
    p: Optional[PSpec]


data = {"p": {"sort": "some_field:asc", "fields": []}}
print(repr(Filter(**data)))
```
output
```
Filter(g=None, p=PSpec(spec_fields=[], filter=None, sort='some_field:asc', g=None))
```
Here is some more details to make it work. It seems if I import the models from another file then I get the error. If I run the models in the same file then I dont have any issues.  Here are the 2 files that I am using.

If I run the files with pydantic 1.6.1 it throws the error above. If I downgrade to the 1.6.0 version then it works.

[Archive.zip](https://github.com/samuelcolvin/pydantic/files/4956170/Archive.zip)



Hi @roycollins 
I just tried quickly and it indeed crashes with v1.6.1 and not with v1.6. Moving some code of `ModelField.prepare` into a dedicated method breaks the code. But I'm not convinced your code is supposed to work when you use `GSpec` without quotes to make as it is a forward ref.

```python
from __future__ import annotations
from pydantic import BaseModel, Field
from typing import List, Optional


class Spec(BaseModel):
    spec_fields: List[str] = Field(..., alias="fields")
    filter: Optional[str]
    sort: Optional[str]


class PSpec(Spec):
    g: Optional[GSpec]


class GSpec(Spec):
    p: Optional[PSpec]

PSpec.update_forward_refs()

class Filter(BaseModel):
    g: Optional[GSpec]
    p: Optional[PSpec]


data = {"p": {"sort": "some_field:asc", "fields": []}}
print(repr(Filter(**data)))
```

this crashes indeed!
And in fact when I remove the `from __future__ import annotations`, the error changes to be
```
NameError: name 'GSpec' is not defined
```
If you change `class PSpec` into

```python
class PSpec(Spec):
    g: Optional['GSpec']
```

can you confirm it works as expected please?
Yes removing the __future__ for delayed annotations does indeed work if I define the types as a string. I am perfectly fine converting the type annotations to strings to make it work.

I was more concerned that it stopped working on 1.6.1 when it worked on 1.6.0.  It does seem to be a change with the way nested models are instantiated when they are optional only.  If they are not optional it seems to work.

There is supposed to be support for python 3.7 delayed annotations per the last block of the docs:
https://docs.google.com/document/d/1jbHNCcAyMC29-2ooBcLz0ROaA0TOfPX-JL2e9klZJEE/edit#
I would be willing to update that part of the docs if it is not meant to support Optional.

Thanks for taking the time to help get to the bottom of this issue. 
You can keep the `__future__` import. Just have to change the type into string as it doesn't exist yet.
I'm more concerned on "why it used to work before?" personally x)

**EDIT**: I understood the "regression"!

Resolving patch:
diff --git a/pydantic/fields.py b/pydantic/fields.py
--- a/pydantic/fields.py
+++ b/pydantic/fields.py
@@ -342,6 +342,11 @@ def prepare(self) -> None:
         """
 
         self._set_default_and_type()
+        if self.type_.__class__ == ForwardRef:
+            # self.type_ is currently a ForwardRef and there's nothing we can do now,
+            # user will need to call model.update_forward_refs()
+            return
+
         self._type_analysis()
         if self.required is Undefined:
             self.required = True
@@ -374,11 +379,6 @@ def _set_default_and_type(self) -> None:
         if self.type_ is None:
             raise errors_.ConfigError(f'unable to infer type for attribute "{self.name}"')
 
-        if self.type_.__class__ == ForwardRef:
-            # self.type_ is currently a ForwardRef and there's nothing we can do now,
-            # user will need to call model.update_forward_refs()
-            return
-
         if self.required is False and default_value is None:
             self.allow_none = True
 


Test patch:
diff --git a/tests/test_forward_ref.py b/tests/test_forward_ref.py
--- a/tests/test_forward_ref.py
+++ b/tests/test_forward_ref.py
@@ -439,3 +439,36 @@ class Foo(BaseModel):
     raise AssertionError('error not raised')
     """
     )
+
+
+@skip_pre_37
+def test_forward_ref_optional(create_module):
+    module = create_module(
+        """
+from __future__ import annotations
+from pydantic import BaseModel, Field
+from typing import List, Optional
+
+
+class Spec(BaseModel):
+    spec_fields: List[str] = Field(..., alias="fields")
+    filter: Optional[str]
+    sort: Optional[str]
+
+
+class PSpec(Spec):
+    g: Optional[GSpec]
+
+
+class GSpec(Spec):
+    p: Optional[PSpec]
+
+PSpec.update_forward_refs()
+
+class Filter(BaseModel):
+    g: Optional[GSpec]
+    p: Optional[PSpec]
+    """
+    )
+    Filter = module.Filter
+    assert isinstance(Filter(p={'sort': 'some_field:asc', 'fields': []}), Filter)