Repository: samuelcolvin/pydantic
Pull request #3159 (2021-09-01 17:47:24)

Nested env
<!-- Thank you for your contribution! -->
<!-- Unless your change is trivial, please create an issue to discuss the change before creating a PR -->
<!-- See https://pydantic-docs.helpmanual.io/contributing/ for help on Contributing -->

## Change Summary
- new ```env_nested_delimiter option``` in Config class
- optional ```_env_nested_delimiter``` kwarg in init ```BaseSettings```
- patched ```EnvSettingsSource```
Depending on ```env_nested_delimiter``` trying to find env variables with that delimiter and split those variable into nested dicts. Merge those dicts if they have same nesting structure.
So lets say we have 
```
class A(BaseSettings):
    foo: str


class B(BaseSettings):
    bar: A


class C(BaseSettings):
    baz: B
```
now we are able to fill the ```A.foo``` with env variable like
```BAZ__BAR__FOO='hello' ```

## Related issue number

#2304 #2232

## Checklist

* [x] Unit tests for the changes exist
* [x] Tests pass on CI and coverage remains at 100%
* [x] Documentation reflects the changes where applicable
* [x] `changes/<pull request or issue id>-<github username>.md` file added describing change
  (see [changes/README.md](https://github.com/samuelcolvin/pydantic/blob/master/changes/README.md) for details)
* [x] My PR is ready to review, **please add a comment including the phrase "please review" to assign reviewers**


MadJlzz: Exactly what I was looking for! I believe that `pydantic` is already doing an excellent job ; removing boiler plate code thanks to this PR will help us centralize settings configuration.

I can help if needed (doc, testing, ...)

cc @daviskirk, because https://github.com/daviskirk/climatecontrol#pydantic seems to have this already implemented.


Air-Mark: @MadJlzz it is nice to hear that it was useful for you. Sorry for disappearing for almost a moth... I cleaned up code and logic a bit. Added some docs and tests. I think this PR is now ready for submission. And of course It would be great to have some help if needed.
Air-Mark: please review
samuelcolvin: please update
Air-Mark: please review
samuelcolvin: please update.
Air-Mark: Thank you for review. Did changes that you been asking for. Improved and redesigned `explode_env_vars` a bit. 
Please review
samuelcolvin: Thanks so much for this, I'm really looking forward to using it.

I've made some changes to the logic to avoid a few errors I found. Please let me know if you have any and feedback or want to make any more changes, otherwie I'll merge over the next few days.

Patch:
diff --git a/changes/3159-Air-Mark.md b/changes/3159-Air-Mark.md
new file mode 100644
index 000000000..1677fd0f5
--- /dev/null
+++ b/changes/3159-Air-Mark.md
@@ -0,0 +1,1 @@
+Nested env variables can now be configured regarding the `env_nested_delimiter`
diff --git a/changes/3175-christianbundy.md b/changes/3175-christianbundy.md
new file mode 100644
index 000000000..3faf22eaf
--- /dev/null
+++ b/changes/3175-christianbundy.md
@@ -0,0 +1,1 @@
+Add support for Mypy 0.920
diff --git a/changes/3254-kludex.md b/changes/3254-kludex.md
new file mode 100644
index 000000000..d86e6889e
--- /dev/null
+++ b/changes/3254-kludex.md
@@ -0,0 +1,1 @@
+Add `AmqpDsn` class
diff --git a/changes/3434-jasujm.md b/changes/3434-jasujm.md
new file mode 100644
index 000000000..bd93d666f
--- /dev/null
+++ b/changes/3434-jasujm.md
@@ -0,0 +1,2 @@
+When generating field schema, pass optional `field` argument (of type
+`pydantic.fields.ModelField`) to `__modify_schema__()` if present
diff --git a/docs/build/exec_examples.py b/docs/build/exec_examples.py
index 50f9efee3..70b41425d 100755
--- a/docs/build/exec_examples.py
+++ b/docs/build/exec_examples.py
@@ -148,6 +148,11 @@ def exec_examples():
         'my_auth_key': 'xxx',
         'my_api_key': 'xxx',
         'database_dsn': 'postgres://postgres@localhost:5432/env_db',
+        'v0': '0',
+        'sub_model': '{"v1": "json-1", "v2": "json-2"}',
+        'sub_model__v2': 'nested-2',
+        'sub_model__v3': '3',
+        'sub_model__deep__v4': 'v4',
     })
 
     sys.path.append(str(EXAMPLES_DIR))
diff --git a/docs/examples/schema_with_field.py b/docs/examples/schema_with_field.py
new file mode 100644
index 000000000..9288efd8c
--- /dev/null
+++ b/docs/examples/schema_with_field.py
@@ -0,0 +1,31 @@
+# output-json
+from typing import Optional
+
+from pydantic import BaseModel, Field
+from pydantic.fields import ModelField
+
+
+class RestrictedAlphabetStr(str):
+    @classmethod
+    def __get_validators__(cls):
+        yield cls.validate
+
+    @classmethod
+    def validate(cls, value, field: ModelField):
+        alphabet = field.field_info.extra['alphabet']
+        if any(c not in alphabet for c in value):
+            raise ValueError(f'{value!r} is not restricted to {alphabet!r}')
+        return cls(value)
+
+    @classmethod
+    def __modify_schema__(cls, field_schema, field: Optional[ModelField]):
+        if field:
+            alphabet = field.field_info.extra['alphabet']
+            field_schema['examples'] = [c * 3 for c in alphabet]
+
+
+class MyModel(BaseModel):
+    value: RestrictedAlphabetStr = Field(alphabet='ABC')
+
+
+print(MyModel.schema_json(indent=2))
diff --git a/docs/examples/settings_main.py b/docs/examples/settings_main.py
index c2beadc77..2ef6e4609 100644
--- a/docs/examples/settings_main.py
+++ b/docs/examples/settings_main.py
@@ -6,6 +6,7 @@ from pydantic import (
     PyObject,
     RedisDsn,
     PostgresDsn,
+    AmqpDsn,
     Field,
 )
 
@@ -21,6 +22,7 @@ class Settings(BaseSettings):
 
     redis_dsn: RedisDsn = 'redis://user:pass@localhost:6379/1'
     pg_dsn: PostgresDsn = 'postgres://user:pass@localhost:5432/foobar'
+    amqp_dsn: AmqpDsn = 'amqp://user:pass@localhost:5672/'
 
     special_function: PyObject = 'math.cos'
 
diff --git a/docs/examples/settings_nested_env.py b/docs/examples/settings_nested_env.py
new file mode 100644
index 000000000..8e414d232
--- /dev/null
+++ b/docs/examples/settings_nested_env.py
@@ -0,0 +1,23 @@
+from pydantic import BaseModel, BaseSettings
+
+
+class DeepSubModel(BaseModel):
+    v4: str
+
+
+class SubModel(BaseModel):
+    v1: str
+    v2: bytes
+    v3: int
+    deep: DeepSubModel
+
+
+class Settings(BaseSettings):
+    v0: str
+    sub_model: SubModel
+
+    class Config:
+        env_nested_delimiter = '__'
+
+
+print(Settings().dict())
diff --git a/docs/requirements.txt b/docs/requirements.txt
index dee12cff6..2bc1b7dc1 100644
--- a/docs/requirements.txt
+++ b/docs/requirements.txt
@@ -1,12 +1,12 @@
 ansi2html==1.6.0
 flake8==4.0.1
 flake8-quotes==3.3.1
-hypothesis==6.30.1
+hypothesis==6.31.4
 markdown-include==0.6.0
 mdx-truly-sane-lists==1.2
 mkdocs==1.2.3
 mkdocs-exclude==1.0.2
-mkdocs-material==8.0.5
+mkdocs-material==8.1.1
 sqlalchemy
 orjson
 ujson
diff --git a/docs/usage/schema.md b/docs/usage/schema.md
index 2ee1a634d..9928b2782 100644
--- a/docs/usage/schema.md
+++ b/docs/usage/schema.md
@@ -150,6 +150,21 @@ For versions of Python prior to 3.9, `typing_extensions.Annotated` can be used.
 Custom field types can customise the schema generated for them using the `__modify_schema__` class method;
 see [Custom Data Types](types.md#custom-data-types) for more details.
 
+`__modify_schema__` can also take a `field` argument which will have type `Optional[ModelField]`.
+*pydantic* will inspect the signature of `__modify_schema__` to determine whether the `field` argument should be
+included.
+
+```py
+{!.tmp_examples/schema_with_field.py!}
+```
+_(This script is complete, it should run "as is")_
+
+Outputs:
+
+```json
+{!.tmp_examples/schema_with_field.json!}
+```
+
 ## JSON Schema Types
 
 Types, custom field types, and constraints (like `max_length`) are mapped to the corresponding spec formats in the
diff --git a/docs/usage/settings.md b/docs/usage/settings.md
index b4a47ea06..1de49c0a5 100644
--- a/docs/usage/settings.md
+++ b/docs/usage/settings.md
@@ -69,6 +69,36 @@ be if passed directly to the initialiser (as a string).
 Complex types like `list`, `set`, `dict`, and sub-models are populated from the environment
 by treating the environment variable's value as a JSON-encoded string.
 
+Another way to populate nested complex variables is to configure your model with the `env_nested_delimiter`
+config setting, then use an env variable with a name pointing to the nested module fields.
+What it does is simply explodes yor variable into nested models or dicts.
+So if you define a variable `FOO__BAR__BAZ=123` it will convert it into `FOO={'BAR': {'BAZ': 123}}`
+If you have multiple variables with the same structure they will be merged.
+
+With the following environment variables:
+```bash
+# your environment
+export V0=0
+export SUB_MODEL='{"v1": "json-1", "v2": "json-2"}'
+export SUB_MODEL__V2=nested-2
+export SUB_MODEL__V3=3
+export SUB_MODEL__DEEP__V4=v4
+```
+
+You could load a settings module thus:
+```py
+{!.tmp_examples/settings_nested_env.py!}
+```
+
+`env_nested_delimiter` can be configured via the `Config` class as shown above, or via the 
+`_env_nested_delimiter` keyword argument on instantiation.
+
+JSON is only parsed in top-level fields, if you need to parse JSON in sub-models, you will need to implement
+validators on those models.
+
+Nested environment variables take precedence over the top-level environment variable JSON
+(e.g. in the example above, `SUB_MODEL__V2` trumps `SUB_MODEL`).
+
 ## Dotenv (.env) support
 
 !!! note
diff --git a/docs/usage/types.md b/docs/usage/types.md
index 1a852169f..3fcd61d31 100644
--- a/docs/usage/types.md
+++ b/docs/usage/types.md
@@ -537,6 +537,9 @@ _(This script is complete, it should run "as is")_
 `PostgresDsn`
 : a postgres DSN style URL; see [URLs](#urls)
 
+`RabbitMqDsn`
+: an `AMQP` DSN style URL as used by RabbitMQ, StormMQ, ActiveMQ etc.; see [URLs](#urls)
+
 `RedisDsn`
 : a redis DSN style URL; see [URLs](#urls)
 
@@ -633,6 +636,7 @@ For URI/URL validation the following types are available:
   - `postgresql+psycopg2cffi`
   - `postgresql+py-postgresql`
   - `postgresql+pygresql`
+- `AmqpDsn`: schema `amqp` or `amqps`, user info not required, TLD not required, host not required
 - `RedisDsn`: scheme `redis` or `rediss`, user info not required, tld not required, host not required (CHANGED: user info
   not required from **v1.6** onwards), user info may be passed without user part (e.g., `rediss://:pass@localhost`)
 - `stricturl`: method with the following keyword arguments:
diff --git a/pydantic/__init__.py b/pydantic/__init__.py
index 6b6c56095..0443acef6 100644
--- a/pydantic/__init__.py
+++ b/pydantic/__init__.py
@@ -57,6 +57,7 @@ __all__ = [
     'IPvAnyInterface',
     'IPvAnyNetwork',
     'PostgresDsn',
+    'AmqpDsn',
     'RedisDsn',
     'KafkaDsn',
     'validate_email',
diff --git a/pydantic/env_settings.py b/pydantic/env_settings.py
index b68aba8d1..35b96970f 100644
--- a/pydantic/env_settings.py
+++ b/pydantic/env_settings.py
@@ -30,13 +30,18 @@ class BaseSettings(BaseModel):
         __pydantic_self__,
         _env_file: Optional[StrPath] = env_file_sentinel,
         _env_file_encoding: Optional[str] = None,
+        _env_nested_delimiter: Optional[str] = None,
         _secrets_dir: Optional[StrPath] = None,
         **values: Any,
     ) -> None:
         # Uses something other than `self` the first arg to allow "self" as a settable attribute
         super().__init__(
             **__pydantic_self__._build_values(
-                values, _env_file=_env_file, _env_file_encoding=_env_file_encoding, _secrets_dir=_secrets_dir
+                values,
+                _env_file=_env_file,
+                _env_file_encoding=_env_file_encoding,
+                _env_nested_delimiter=_env_nested_delimiter,
+                _secrets_dir=_secrets_dir,
             )
         )
 
@@ -45,6 +50,7 @@ class BaseSettings(BaseModel):
         init_kwargs: Dict[str, Any],
         _env_file: Optional[StrPath] = None,
         _env_file_encoding: Optional[str] = None,
+        _env_nested_delimiter: Optional[str] = None,
         _secrets_dir: Optional[StrPath] = None,
     ) -> Dict[str, Any]:
         # Configure built-in sources
@@ -54,6 +60,9 @@ class BaseSettings(BaseModel):
             env_file_encoding=(
                 _env_file_encoding if _env_file_encoding is not None else self.__config__.env_file_encoding
             ),
+            env_nested_delimiter=(
+                _env_nested_delimiter if _env_nested_delimiter is not None else self.__config__.env_nested_delimiter
+            ),
         )
         file_secret_settings = SecretsSettingsSource(secrets_dir=_secrets_dir or self.__config__.secrets_dir)
         # Provide a hook to set built-in sources priority and add / remove sources
@@ -71,6 +80,7 @@ class BaseSettings(BaseModel):
         env_prefix = ''
         env_file = None
         env_file_encoding = None
+        env_nested_delimiter = None
         secrets_dir = None
         validate_all = True
         extra = Extra.forbid
@@ -132,17 +142,20 @@ class InitSettingsSource:
 
 
 class EnvSettingsSource:
-    __slots__ = ('env_file', 'env_file_encoding')
+    __slots__ = ('env_file', 'env_file_encoding', 'env_nested_delimiter')
 
-    def __init__(self, env_file: Optional[StrPath], env_file_encoding: Optional[str]):
+    def __init__(
+        self, env_file: Optional[StrPath], env_file_encoding: Optional[str], env_nested_delimiter: Optional[str] = None
+    ):
         self.env_file: Optional[StrPath] = env_file
         self.env_file_encoding: Optional[str] = env_file_encoding
+        self.env_nested_delimiter: Optional[str] = env_nested_delimiter
 
-    def __call__(self, settings: BaseSettings) -> Dict[str, Any]:
+    def __call__(self, settings: BaseSettings) -> Dict[str, Any]:  # noqa C901
         """
         Build environment variables suitable for passing to the Model.
         """
-        d: Dict[str, Optional[str]] = {}
+        d: Dict[str, Any] = {}
 
         if settings.__config__.case_sensitive:
             env_vars: Mapping[str, Optional[str]] = os.environ
@@ -166,26 +179,68 @@ class EnvSettingsSource:
                 if env_val is not None:
                     break
 
-            if env_val is None:
-                continue
+            is_complex, allow_json_failure = self.field_is_complex(field)
+            if is_complex:
+                if env_val is None:
+                    # field is complex but no value found so far, try explode_env_vars
+                    env_val_built = self.explode_env_vars(field, env_vars)
+                    if env_val_built:
+                        d[field.alias] = env_val_built
+                else:
+                    # field is complex and there's a value, decode that as JSON, then add explode_env_vars
+                    try:
+                        env_val = settings.__config__.json_loads(env_val)
+                    except ValueError as e:
+                        if not allow_json_failure:
+                            raise SettingsError(f'error parsing JSON for "{env_name}"') from e
+
+                    if isinstance(env_val, dict):
+                        d[field.alias] = deep_update(env_val, self.explode_env_vars(field, env_vars))
+                    else:
+                        d[field.alias] = env_val
+            elif env_val is not None:
+                # simplest case, field is not complex, we only need to add the value if it was found
+                d[field.alias] = env_val
 
-            if field.is_complex():
-                try:
-                    env_val = settings.__config__.json_loads(env_val)
-                except ValueError as e:
-                    raise SettingsError(f'error parsing JSON for "{env_name}"') from e
-            elif (
-                is_union(get_origin(field.type_)) and field.sub_fields and any(f.is_complex() for f in field.sub_fields)
-            ):
-                try:
-                    env_val = settings.__config__.json_loads(env_val)
-                except ValueError:
-                    pass
-            d[field.alias] = env_val
         return d
 
+    def field_is_complex(self, field: ModelField) -> Tuple[bool, bool]:
+        """
+        Find out if a field is complex, and if so whether JSON errors should be ignored
+        """
+        if field.is_complex():
+            allow_json_failure = False
+        elif is_union(get_origin(field.type_)) and field.sub_fields and any(f.is_complex() for f in field.sub_fields):
+            allow_json_failure = True
+        else:
+            return False, False
+
+        return True, allow_json_failure
+
+    def explode_env_vars(self, field: ModelField, env_vars: Mapping[str, Optional[str]]) -> Dict[str, Any]:
+        """
+        Process env_vars and extract the values of keys containing env_nested_delimiter into nested dictionaries.
+
+        This is applied to a single field, hence filtering by env_var prefix.
+        """
+        prefixes = [f'{env_name}{self.env_nested_delimiter}' for env_name in field.field_info.extra['env_names']]
+        result: Dict[str, Any] = {}
+        for env_name, env_val in env_vars.items():
+            if not any(env_name.startswith(prefix) for prefix in prefixes):
+                continue
+            _, *keys, last_key = env_name.split(self.env_nested_delimiter)
+            env_var = result
+            for key in keys:
+                env_var = env_var.setdefault(key, {})
+            env_var[last_key] = env_val
+
+        return result
+
     def __repr__(self) -> str:
-        return f'EnvSettingsSource(env_file={self.env_file!r}, env_file_encoding={self.env_file_encoding!r})'
+        return (
+            f'EnvSettingsSource(env_file={self.env_file!r}, env_file_encoding={self.env_file_encoding!r}, '
+            f'env_nested_delimiter={self.env_nested_delimiter!r})'
+        )
 
 
 class SecretsSettingsSource:
diff --git a/pydantic/mypy.py b/pydantic/mypy.py
index 119c1a07c..c2120992f 100644
--- a/pydantic/mypy.py
+++ b/pydantic/mypy.py
@@ -14,6 +14,14 @@ except ImportError:  # pragma: no cover
 
         warnings.warn('No TOML parser installed, cannot read configuration from `pyproject.toml`.')
         toml = None  # type: ignore
+
+try:
+    from mypy.types import TypeVarDef
+except ImportError:  # pragma: no cover
+    # Backward-compatible with TypeVarDef from Mypy 0.910.
+    from mypy.types import TypeVarType as TypeVarDef  # type: ignore[misc]
+
+
 from mypy.errorcodes import ErrorCode
 from mypy.nodes import (
     ARG_NAMED,
@@ -59,7 +67,6 @@ from mypy.types import (
     Type,
     TypeOfAny,
     TypeType,
-    TypeVarDef,
     TypeVarType,
     UnionType,
     get_proper_type,
@@ -359,7 +366,13 @@ class PydanticModelTransformer:
         tvd = TypeVarDef(self_tvar_name, tvar_fullname, -1, [], obj_type)
         self_tvar_expr = TypeVarExpr(self_tvar_name, tvar_fullname, [], obj_type)
         ctx.cls.info.names[self_tvar_name] = SymbolTableNode(MDEF, self_tvar_expr)
-        self_type = TypeVarType(tvd)
+
+        # Backward-compatible with TypeVarDef from Mypy 0.910.
+        if isinstance(tvd, TypeVarType):  # pragma: no cover
+            self_type = tvd
+        else:
+            self_type = TypeVarType(tvd)
+
         add_method(
             ctx,
             'construct',
diff --git a/pydantic/networks.py b/pydantic/networks.py
index 2be1ac80d..53b4643b0 100644
--- a/pydantic/networks.py
+++ b/pydantic/networks.py
@@ -68,6 +68,7 @@ __all__ = [
     'IPvAnyInterface',
     'IPvAnyNetwork',
     'PostgresDsn',
+    'AmqpDsn',
     'RedisDsn',
     'KafkaDsn',
     'validate_email',
@@ -351,6 +352,11 @@ class PostgresDsn(AnyUrl):
     user_required = True
 
 
+class AmqpDsn(AnyUrl):
+    allowed_schemes = {'amqp', 'amqps'}
+    host_required = False
+
+
 class RedisDsn(AnyUrl):
     allowed_schemes = {'redis', 'rediss'}
     host_required = False
diff --git a/pydantic/schema.py b/pydantic/schema.py
index ff6f8af1b..e979678c2 100644
--- a/pydantic/schema.py
+++ b/pydantic/schema.py
@@ -90,6 +90,19 @@ TypeModelOrEnum = Union[Type['BaseModel'], Type[Enum]]
 TypeModelSet = Set[TypeModelOrEnum]
 
 
+def _apply_modify_schema(
+    modify_schema: Callable[..., None], field: Optional[ModelField], field_schema: Dict[str, Any]
+) -> None:
+    from inspect import signature
+
+    sig = signature(modify_schema)
+    args = set(sig.parameters.keys())
+    if 'field' in args or 'kwargs' in args:
+        modify_schema(field_schema, field=field)
+    else:
+        modify_schema(field_schema)
+
+
 def schema(
     models: Sequence[Union[Type['BaseModel'], Type['Dataclass']]],
     *,
@@ -335,7 +348,7 @@ def get_field_schema_validations(field: ModelField) -> Dict[str, Any]:
         f_schema.update(field.field_info.extra)
     modify_schema = getattr(field.outer_type_, '__modify_schema__', None)
     if modify_schema:
-        modify_schema(f_schema)
+        _apply_modify_schema(modify_schema, field, f_schema)
     return f_schema
 
 
@@ -567,7 +580,7 @@ def field_type_schema(
             field_type = field.outer_type_
         modify_schema = getattr(field_type, '__modify_schema__', None)
         if modify_schema:
-            modify_schema(f_schema)
+            _apply_modify_schema(modify_schema, field, f_schema)
     return f_schema, definitions, nested_models
 
 
@@ -579,6 +592,7 @@ def model_process_schema(
     ref_prefix: Optional[str] = None,
     ref_template: str = default_ref_template,
     known_models: TypeModelSet = None,
+    field: Optional[ModelField] = None,
 ) -> Tuple[Dict[str, Any], Dict[str, Any], Set[str]]:
     """
     Used by ``model_schema()``, you probably should be using that function.
@@ -592,7 +606,7 @@ def model_process_schema(
     known_models = known_models or set()
     if lenient_issubclass(model, Enum):
         model = cast(Type[Enum], model)
-        s = enum_process_schema(model)
+        s = enum_process_schema(model, field=field)
         return s, {}, set()
     model = cast(Type['BaseModel'], model)
     s = {'title': model.__config__.title or model.__name__}
@@ -674,7 +688,7 @@ def model_type_schema(
     return out_schema, definitions, nested_models
 
 
-def enum_process_schema(enum: Type[Enum]) -> Dict[str, Any]:
+def enum_process_schema(enum: Type[Enum], *, field: Optional[ModelField] = None) -> Dict[str, Any]:
     """
     Take a single `enum` and generate its schema.
 
@@ -695,7 +709,7 @@ def enum_process_schema(enum: Type[Enum]) -> Dict[str, Any]:
 
     modify_schema = getattr(enum, '__modify_schema__', None)
     if modify_schema:
-        modify_schema(schema_)
+        _apply_modify_schema(modify_schema, field, schema_)
 
     return schema_
 
@@ -871,7 +885,7 @@ def field_singleton_schema(  # noqa: C901 (ignore complexity)
         enum_name = model_name_map[field_type]
         f_schema, schema_overrides = get_field_info_schema(field, schema_overrides)
         f_schema.update(get_schema_ref(enum_name, ref_prefix, ref_template, schema_overrides))
-        definitions[enum_name] = enum_process_schema(field_type)
+        definitions[enum_name] = enum_process_schema(field_type, field=field)
     elif is_namedtuple(field_type):
         sub_schema, *_ = model_process_schema(
             field_type.__pydantic_model__,
@@ -880,6 +894,7 @@ def field_singleton_schema(  # noqa: C901 (ignore complexity)
             ref_prefix=ref_prefix,
             ref_template=ref_template,
             known_models=known_models,
+            field=field,
         )
         items_schemas = list(sub_schema['properties'].values())
         f_schema.update(
@@ -895,7 +910,7 @@ def field_singleton_schema(  # noqa: C901 (ignore complexity)
 
         modify_schema = getattr(field_type, '__modify_schema__', None)
         if modify_schema:
-            modify_schema(f_schema)
+            _apply_modify_schema(modify_schema, field, f_schema)
 
     if f_schema:
         return f_schema, definitions, nested_models
@@ -914,6 +929,7 @@ def field_singleton_schema(  # noqa: C901 (ignore complexity)
                 ref_prefix=ref_prefix,
                 ref_template=ref_template,
                 known_models=known_models,
+                field=field,
             )
             definitions.update(sub_definitions)
             definitions[model_name] = sub_schema


Test patch:
diff --git a/tests/requirements-linting.txt b/tests/requirements-linting.txt
index 1a8712743..81614a46e 100644
--- a/tests/requirements-linting.txt
+++ b/tests/requirements-linting.txt
@@ -1,11 +1,11 @@
 black==21.12b0
 flake8==4.0.1
 flake8-quotes==3.3.1
-hypothesis==6.30.1
+hypothesis==6.31.4
 isort==5.10.1
 mypy==0.910
 pre-commit==2.16.0
 pycodestyle==2.8.0
 pyflakes==2.4.0
-twine==3.7.0
+twine==3.7.1
 types-toml==0.10.1
diff --git a/tests/requirements-testing.txt b/tests/requirements-testing.txt
index cb76137e7..faa291edf 100644
--- a/tests/requirements-testing.txt
+++ b/tests/requirements-testing.txt
@@ -1,5 +1,5 @@
 coverage==6.2
-hypothesis==6.30.1
+hypothesis==6.31.4
 # pin importlib-metadata as upper versions need typing-extensions to work if on python < 3.8
 importlib-metadata==3.1.0;python_version<"3.8"
 mypy==0.910
diff --git a/tests/test_networks.py b/tests/test_networks.py
index 4e7fe29fd..5ced4ac4c 100644
--- a/tests/test_networks.py
+++ b/tests/test_networks.py
@@ -1,6 +1,7 @@
 import pytest
 
 from pydantic import (
+    AmqpDsn,
     AnyUrl,
     BaseModel,
     EmailStr,
@@ -443,6 +444,37 @@ def test_postgres_dsns():
     assert error == {'loc': ('a',), 'msg': 'URL host invalid', 'type': 'value_error.url.host'}
 
 
+def test_amqp_dsns():
+    class Model(BaseModel):
+        a: AmqpDsn
+
+    m = Model(a='amqp://user:pass@localhost:1234/app')
+    assert m.a == 'amqp://user:pass@localhost:1234/app'
+    assert m.a.user == 'user'
+    assert m.a.password == 'pass'
+
+    m = Model(a='amqps://user:pass@localhost:5432//')
+    assert m.a == 'amqps://user:pass@localhost:5432//'
+
+    with pytest.raises(ValidationError) as exc_info:
+        Model(a='http://example.org')
+    assert exc_info.value.errors()[0]['type'] == 'value_error.url.scheme'
+
+    # Password is not required for AMQP protocol
+    m = Model(a='amqp://localhost:1234/app')
+    assert m.a == 'amqp://localhost:1234/app'
+    assert m.a.user is None
+    assert m.a.password is None
+
+    # Only schema is required for AMQP protocol.
+    # https://www.rabbitmq.com/uri-spec.html
+    m = Model(a='amqps://')
+    assert m.a.scheme == 'amqps'
+    assert m.a.host is None
+    assert m.a.port is None
+    assert m.a.path is None
+
+
 def test_redis_dsns():
     class Model(BaseModel):
         a: RedisDsn
diff --git a/tests/test_schema.py b/tests/test_schema.py
index c1e1f01b0..eb1f081e8 100644
--- a/tests/test_schema.py
+++ b/tests/test_schema.py
@@ -32,6 +32,7 @@ from typing_extensions import Annotated, Literal
 from pydantic import BaseModel, Extra, Field, ValidationError, confrozenset, conlist, conset, validator
 from pydantic.color import Color
 from pydantic.dataclasses import dataclass
+from pydantic.fields import ModelField
 from pydantic.generics import GenericModel
 from pydantic.networks import AnyUrl, EmailStr, IPvAnyAddress, IPvAnyInterface, IPvAnyNetwork, NameEmail, stricturl
 from pydantic.schema import (
@@ -2628,6 +2629,27 @@ def test_complex_nested_generic():
     }
 
 
+def test_schema_with_field_parameter():
+    class RestrictedAlphabetStr(str):
+        @classmethod
+        def __modify_schema__(cls, field_schema, field: Optional[ModelField]):
+            assert isinstance(field, ModelField)
+            alphabet = field.field_info.extra['alphabet']
+            field_schema['examples'] = [c * 3 for c in alphabet]
+
+    class MyModel(BaseModel):
+        value: RestrictedAlphabetStr = Field(alphabet='ABC')
+
+    assert MyModel.schema() == {
+        'title': 'MyModel',
+        'type': 'object',
+        'properties': {
+            'value': {'title': 'Value', 'alphabet': 'ABC', 'examples': ['AAA', 'BBB', 'CCC'], 'type': 'string'}
+        },
+        'required': ['value'],
+    }
+
+
 def test_discriminated_union():
     class BlackCat(BaseModel):
         pet_type: Literal['cat']
diff --git a/tests/test_settings.py b/tests/test_settings.py
index 11f051993..c11cdbb73 100644
--- a/tests/test_settings.py
+++ b/tests/test_settings.py
@@ -79,7 +79,7 @@ def test_nested_env_with_basemodel(env):
     assert s.top == {'apple': 'value', 'banana': 'secret_value'}
 
 
-def test_nested_env_with_dict(env):
+def test_merge_dict(env):
     class Settings(BaseSettings):
         top: Dict[str, str]
 
@@ -90,6 +90,80 @@ def test_nested_env_with_dict(env):
     assert s.top == {'apple': 'value', 'banana': 'secret_value'}
 
 
+def test_nested_env_delimiter(env):
+    class SubSubValue(BaseSettings):
+        v6: str
+
+    class SubValue(BaseSettings):
+        v4: str
+        v5: int
+        sub_sub: SubSubValue
+
+    class TopValue(BaseSettings):
+        v1: str
+        v2: str
+        v3: str
+        sub: SubValue
+
+    class Cfg(BaseSettings):
+        v0: str
+        v0_union: Union[SubValue, int]
+        top: TopValue
+
+        class Config:
+            env_nested_delimiter = '__'
+
+    env.set('top', '{"v1": "json-1", "v2": "json-2", "sub": {"v5": "xx"}}')
+    env.set('top__sub__v5', '5')
+    env.set('v0', '0')
+    env.set('top__v2', '2')
+    env.set('top__v3', '3')
+    env.set('v0_union', '0')
+    env.set('top__sub__sub_sub__v6', '6')
+    env.set('top__sub__v4', '4')
+    cfg = Cfg()
+    assert cfg.dict() == {
+        'v0': '0',
+        'v0_union': 0,
+        'top': {
+            'v1': 'json-1',
+            'v2': '2',
+            'v3': '3',
+            'sub': {'v4': '4', 'v5': 5, 'sub_sub': {'v6': '6'}},
+        },
+    }
+
+
+def test_nested_env_delimiter_complex_required(env):
+    class Cfg(BaseSettings):
+        v: str = 'default'
+
+        class Config:
+            env_nested_delimiter = '__'
+
+    env.set('v__x', 'x')
+    env.set('v__y', 'y')
+    cfg = Cfg()
+    assert cfg.dict() == {'v': 'default'}
+
+
+def test_nested_env_delimiter_aliases(env):
+    class SubModel(BaseSettings):
+        v1: str
+        v2: str
+
+    class Cfg(BaseSettings):
+        sub_model: SubModel
+
+        class Config:
+            fields = {'sub_model': {'env': ['foo', 'bar']}}
+            env_nested_delimiter = '__'
+
+    env.set('foo__v1', '-1-')
+    env.set('bar__v2', '-2-')
+    assert Cfg().dict() == {'sub_model': {'v1': '-1-', 'v2': '-2-'}}
+
+
 class DateModel(BaseModel):
     pips: bool = False
 
@@ -1010,6 +1084,6 @@ def test_builtins_settings_source_repr():
     )
     assert (
         repr(EnvSettingsSource(env_file='.env', env_file_encoding='utf-8'))
-        == "EnvSettingsSource(env_file='.env', env_file_encoding='utf-8')"
+        == "EnvSettingsSource(env_file='.env', env_file_encoding='utf-8', env_nested_delimiter=None)"
     )
     assert repr(SecretsSettingsSource(secrets_dir='/secrets')) == "SecretsSettingsSource(secrets_dir='/secrets')"