Repository: samuelcolvin/pydantic
Pull request #2035 (2020-10-26 01:48:04)

add pytest assert rewrite to create_module, rewrite existing tests
<!-- Thank you for your contribution! -->
<!-- Unless your change is trivial, please create an issue to discuss the change before creating a PR -->
<!-- See https://pydantic-docs.helpmanual.io/contributing/ for help on Contributing -->

## Change Summary
- Rewrite `assert` statements in modules created with `create_module` fixture
- Rewrite some test with `@create_module` decorator
- Add `# language=Python` comments before str module declarations to tell PyCharm to inject syntax highlighting 

Now such tests will have informative output:
```py
def test_with_create_module(create_module):
    @create_module
    def module():
        assert list(range(10)) == [1, 2, 3]
```
```py
=================================== FAILURES ===================================
___________________________ test_with_create_module ____________________________

create_module = <function create_module.<locals>.run at 0x7fa3a45d4e50>

    def test_with_create_module(create_module):
        @create_module
>       def module():

test.py:4: 

... traceback

>   assert list(range(10)) == [1, 2, 3]
E   assert [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] == [1, 2, 3]
/tmp_folder/test_with_create_module_5e2731ff5c.py:1: AssertionError
=========================== short test summary info ============================
FAILED test.py::test_with_create_module - assert [0, 1, 2, 3, 4, 5, 6, 7, 8, ...
============================== 1 failed in 0.11s ===============================
```


<!-- Please give a short summary of the changes. -->

## Related issue number

<!-- Are there any issues opened that will be resolved by merging this change? -->

## Checklist

* [ ] Unit tests for the changes exist
* [x] Tests pass on CI and coverage remains at 100%
* [ ] Documentation reflects the changes where applicable
* [ ] `changes/<pull request or issue id>-<github username>.md` file added describing change
  (see [changes/README.md](https://github.com/samuelcolvin/pydantic/blob/master/changes/README.md) for details)


codecov[bot]: # [Codecov](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035?src=pr&el=h1) Report
> Merging [#2035](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035?src=pr&el=desc) into [master](https://codecov.io/gh/samuelcolvin/pydantic/commit/5bfee873c8ab26b453cde59eaee4eca5b5a9eb7b?el=desc) will **decrease** coverage by `0.99%`.
> The diff coverage is `n/a`.


```diff
@@            Coverage Diff             @@
##           master    #2035      +/-   ##
==========================================
- Coverage   99.90%   98.90%   -1.00%     
==========================================
  Files          21       21              
  Lines        4035     4034       -1     
  Branches      805      805              
==========================================
- Hits         4031     3990      -41     
- Misses          3       34      +31     
- Partials        1       10       +9     
```


| [Impacted Files](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pydantic/typing.py](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035/diff?src=pr&el=tree#diff-cHlkYW50aWMvdHlwaW5nLnB5) | `74.80% <0.00%> (-22.05%)` | :arrow_down: |
| [pydantic/version.py](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035/diff?src=pr&el=tree#diff-cHlkYW50aWMvdmVyc2lvbi5weQ==) | `88.23% <0.00%> (-11.77%)` | :arrow_down: |
| [pydantic/env\_settings.py](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035/diff?src=pr&el=tree#diff-cHlkYW50aWMvZW52X3NldHRpbmdzLnB5) | `97.43% <0.00%> (-2.57%)` | :arrow_down: |
| [pydantic/networks.py](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035/diff?src=pr&el=tree#diff-cHlkYW50aWMvbmV0d29ya3MucHk=) | `99.22% <0.00%> (-0.78%)` | :arrow_down: |
| [pydantic/utils.py](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035/diff?src=pr&el=tree#diff-cHlkYW50aWMvdXRpbHMucHk=) | `99.25% <0.00%> (-0.75%)` | :arrow_down: |
| [pydantic/validators.py](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035/diff?src=pr&el=tree#diff-cHlkYW50aWMvdmFsaWRhdG9ycy5weQ==) | `99.45% <0.00%> (-0.55%)` | :arrow_down: |
| [pydantic/types.py](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035/diff?src=pr&el=tree#diff-cHlkYW50aWMvdHlwZXMucHk=) | `99.58% <0.00%> (-0.42%)` | :arrow_down: |
| [pydantic/generics.py](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035/diff?src=pr&el=tree#diff-cHlkYW50aWMvZ2VuZXJpY3MucHk=) | `100.00% <0.00%> (ø)` | |
| [pydantic/decorator.py](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035/diff?src=pr&el=tree#diff-cHlkYW50aWMvZGVjb3JhdG9yLnB5) | `100.00% <0.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035?src=pr&el=footer). Last update [5bfee87...d474c02](https://codecov.io/gh/samuelcolvin/pydantic/pull/2035?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

samuelcolvin: Looks good, is this ready to merge? I see some places still use `module = create_module(` and a string, is that required?
MrMrRobat: > I see some places still use module = create_module( and a string, is that required?

Yes, in these cases using string is required either because of new syntax, supported only in newer python versions, or because of `__future__` imports, which may exist only on top of the file

the `module = create_module` however is not required and all assertions can be moved inside of module. If that would look better, I can change it, but I thought it's unnecessary.

Patch:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 67d4ad8d9..b411771f7 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -38,7 +38,7 @@ jobs:
         run: make check-dist
 
   test-linux:
-    name: test py${{ matrix.python-version }} on ubuntu
+    name: test py${{ matrix.python-version }} on linux
     runs-on: ubuntu-latest
     strategy:
       fail-fast: false
@@ -195,15 +195,8 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os:
-        - ubuntu
-        - macos
-        - windows
-        python-version:
-        - '6'
-        - '7'
-        - '8'
-        - '9'
+        os: [ubuntu , macos , windows]
+        python-version: ['6', '7', '8', '9']
         include:
         - os: ubuntu
           platform: linux
diff --git a/.gitignore b/.gitignore
index b653170b2..262563382 100644
--- a/.gitignore
+++ b/.gitignore
@@ -4,6 +4,7 @@ venv/
 env36/
 env37/
 env38/
+env39/
 Pipfile
 *.lock
 *.py[cod]
@@ -32,4 +33,4 @@ pydantic/*.so
 /sandbox/
 /.ghtopdep_cache/
 /fastapi/
-/codecov.sh
\ No newline at end of file
+/codecov.sh
diff --git a/Makefile b/Makefile
index 69cad5f9a..8aa686f4b 100644
--- a/Makefile
+++ b/Makefile
@@ -25,7 +25,7 @@ install-benchmarks: install-pydantic
 	pip install -U -r benchmarks/requirements.txt
 
 .PHONY: install
-install: install-linting install-testing
+install: install-testing install-linting
 	@echo 'installed development requirements'
 
 .PHONY: build-trace
diff --git a/benchmarks/requirements.txt b/benchmarks/requirements.txt
index 9033b1762..6f3bf9a09 100644
--- a/benchmarks/requirements.txt
+++ b/benchmarks/requirements.txt
@@ -1,5 +1,5 @@
 # no versions since benchmarks should always run with the most recent version of each package
-devtools
+devtools==0.6.1
 
 python-dateutil
 trafaret
diff --git a/changes/1679-MrMrRobat.txt b/changes/1679-MrMrRobat.txt
new file mode 100644
index 000000000..8c6a2dc2e
--- /dev/null
+++ b/changes/1679-MrMrRobat.txt
@@ -0,0 +1,1 @@
+Add private attributes support
\ No newline at end of file
diff --git a/changes/1820-mdgilene.md b/changes/1820-mdgilene.md
new file mode 100644
index 000000000..068ef7e51
--- /dev/null
+++ b/changes/1820-mdgilene.md
@@ -0,0 +1,1 @@
+Added ability for `BaseSettings` to load `SecretStr` and `SecretBytes` from files.
diff --git a/docs/examples/private_attributes.py b/docs/examples/private_attributes.py
new file mode 100644
index 000000000..51da4456e
--- /dev/null
+++ b/docs/examples/private_attributes.py
@@ -0,0 +1,19 @@
+from datetime import datetime
+from random import randint
+
+from pydantic import BaseModel, PrivateAttr
+
+
+class TimeAwareModel(BaseModel):
+    _processed_at: datetime = PrivateAttr(default_factory=datetime.now)
+    _secret_value: str = PrivateAttr()
+
+    def __init__(self, **data):
+        super().__init__(**data)
+        # this could also be done with default_factory
+        self._secret_value = randint(1, 5)
+
+
+m = TimeAwareModel()
+print(m._processed_at)
+print(m._secret_value)
diff --git a/docs/examples/private_attributes_underscore_attrs_are_private.py b/docs/examples/private_attributes_underscore_attrs_are_private.py
new file mode 100644
index 000000000..f24db5f1c
--- /dev/null
+++ b/docs/examples/private_attributes_underscore_attrs_are_private.py
@@ -0,0 +1,16 @@
+from typing import ClassVar
+
+from pydantic import BaseModel
+
+
+class Model(BaseModel):
+    _class_var: ClassVar[str] = 'class var value'
+    _private_attr: str = 'private attr value'
+
+    class Config:
+        underscore_attrs_are_private = True
+
+
+print(Model._class_var)
+print(Model._private_attr)
+print(Model()._private_attr)
diff --git a/docs/requirements.txt b/docs/requirements.txt
index 1ce891d0b..8f3b0d73d 100644
--- a/docs/requirements.txt
+++ b/docs/requirements.txt
@@ -2,11 +2,9 @@ ansi2html==1.5.2
 flake8==3.8.4
 flake8-quotes==3.2.0
 mkdocs==1.1.2
-markdown==3.3.1
 mkdocs-exclude==1.0.2
 mkdocs-material==6.1.0
 markdown-include==0.6.0
-pygments==2.7.1
-sqlalchemy   # pyup: ignore
-orjson       # pyup: ignore
-ujson        # pyup: ignore
+sqlalchemy
+orjson
+ujson
diff --git a/docs/usage/model_config.md b/docs/usage/model_config.md
index ade95eac1..f847ad8c0 100644
--- a/docs/usage/model_config.md
+++ b/docs/usage/model_config.md
@@ -93,6 +93,11 @@ Similarly, if using the `@dataclass` decorator:
 ```
 _(This script is complete, it should run "as is")_
 
+
+**`underscore_attrs_are_private`**
+: whether to treat any underscore non-class var attrs as private, or leave them as is; See [Private model attributes](models.md#private-model-attributes)
+
+
 ## Alias Generator
 
 If data source field names do not match your code style (e. g. CamelCase fields),
diff --git a/docs/usage/models.md b/docs/usage/models.md
index 013a05e6e..87312469d 100644
--- a/docs/usage/models.md
+++ b/docs/usage/models.md
@@ -531,6 +531,26 @@ Where `Field` refers to the [field function](schema.md#field-customisation).
     Moreover if you want to validate default values with `validate_all`,
     *pydantic* will need to call the `default_factory`, which could lead to side effects!
 
+## Private model attributes
+
+If you need to use internal attributes excluded from model fields, you can declare them using `PrivateAttr`:
+
+```py
+{!.tmp_examples/private_attributes.py!}
+```
+_(This script is complete, it should run "as is")_
+
+Private attribute names must start with underscore to prevent conflicts with model fields: both `_attr` and `__attr__` 
+are supported.
+
+If `Config.underscore_attrs_are_private` is `True`, any non-ClassVar underscore attribute will be treated as private:
+```py
+{!.tmp_examples/private_attributes_underscore_attrs_are_private.py!}
+```
+_(This script is complete, it should run "as is")_
+
+Upon class creation pydantic constructs `__slots__` filled with private attributes.
+
 ## Parsing data into a specified type
 
 Pydantic includes a standalone utility function `parse_obj_as` that can be used to apply the parsing
diff --git a/docs/usage/settings.md b/docs/usage/settings.md
index 1068d42d0..a06ea52d4 100644
--- a/docs/usage/settings.md
+++ b/docs/usage/settings.md
@@ -128,6 +128,76 @@ Because python-dotenv is used to parse the file, bash-like semantics such as `ex
 (depending on your OS and environment) may allow your dotenv file to also be used with `source`,
 see [python-dotenv's documentation](https://saurabh-kumar.com/python-dotenv/#usages) for more details.
 
+## Secret Support
+
+Placing secret values in files is a common pattern to provide sensitive configuration to an application.
+
+A secret file follows the same principal as a dotenv file except it only contains a single value and the file name 
+is used as the key. A secret file will look like the following:
+
+`/var/run/db_password`:
+```
+super_secret_database_password
+```
+
+Once you have your secret files, *pydantic* supports loading it in two ways:
+
+**1.** setting `secrets_dir` on `Config` in a `BaseSettings` class to the directory where your secret files are stored:
+
+```py
+class Settings(BaseSettings):
+    ...
+    database_password: str
+
+    class Config:
+        secrets_dir = '/var/run'
+```
+
+**2.** instantiating a `BaseSettings` derived class with the `_secrets_dir` keyword argument:
+
+```py
+settings = Settings(_secrets_dir='/var/run')
+```
+
+In either case, the value of the passed argument can be any valid directory, either absolute or relative to the
+current working directory. From there, *pydantic* will handle everything for you by loading in your variables and
+validating them.
+
+Even when using a secrets directory, *pydantic* will still read environment variables from a dotenv file or the environment,
+**a dotenv file and environment variables will always take priority over values loaded from the secrets directory**.
+
+Passing a file path via the `_secrets_dir` keyword argument on instantiation (method 2) will override
+the value (if any) set on the `Config` class.
+
+### Use Case: Docker Secrets
+
+Docker Secrets can be used to provide sensitive configuration to an application running in a Docker container.
+To use these secrets in a *pydantic* application the process is simple. More information regarding creating, managing
+and using secrets in Docker see the official
+[Docker documentation](https://docs.docker.com/engine/reference/commandline/secret/).
+
+First, define your Settings
+```py
+class Settings(BaseSettings):
+    my_secret_data: str
+
+    class Config:
+        secrets_dir = '/run/secrets'
+```
+!!! note
+    By default Docker uses `/run/secrets` as the target mount point. If you want to use a different location, change 
+    `Config.secrets_dir` accordingly.
+
+Then, Create your secret via the Docker CLI
+```bash
+printf "This is a secret" | docker secret create my_secret_data -
+```
+
+Last, run your application inside a Docker container and supply your newly created secret
+```bash
+docker service create --name pydantic-with-secrets --secret my_secret_data pydantic-app:latest
+```
+
 ## Field value priority
 
 In the case where a value is specified for the same `Settings` field in multiple ways,
@@ -136,4 +206,5 @@ the selected value is determined as follows (in descending order of priority):
 1. Arguments passed to the `Settings` class initialiser.
 2. Environment variables, e.g. `my_prefix_special_function` as described above.
 3. Variables loaded from a dotenv (`.env`) file.
-4. The default field values for the `Settings` model.
+4. Variables loaded from the secrets directory.
+5. The default field values for the `Settings` model.
diff --git a/pydantic/__init__.py b/pydantic/__init__.py
index 1a3cabb8c..5cfc0583f 100644
--- a/pydantic/__init__.py
+++ b/pydantic/__init__.py
@@ -5,7 +5,7 @@ from .decorator import validate_arguments
 from .env_settings import BaseSettings
 from .error_wrappers import ValidationError
 from .errors import *
-from .fields import Field, Required, Schema
+from .fields import Field, PrivateAttr, Required, Schema
 from .main import *
 from .networks import *
 from .parse import Protocol
@@ -96,6 +96,7 @@ __all__ = [
     'StrictInt',
     'StrictFloat',
     'PaymentCardNumber',
+    'PrivateAttr',
     'ByteSize',
     # version
     'VERSION',
diff --git a/pydantic/class_validators.py b/pydantic/class_validators.py
index c697ff52d..9cd951a58 100644
--- a/pydantic/class_validators.py
+++ b/pydantic/class_validators.py
@@ -7,7 +7,7 @@ from typing import TYPE_CHECKING, Any, Callable, Dict, Iterable, List, Optional,
 
 from .errors import ConfigError
 from .typing import AnyCallable
-from .utils import in_ipython
+from .utils import ROOT_KEY, in_ipython
 
 
 class Validator:
@@ -42,7 +42,6 @@ if TYPE_CHECKING:
     ValidatorListDict = Dict[str, List[Validator]]
 
 _FUNCS: Set[str] = set()
-ROOT_KEY = '__root__'
 VALIDATOR_CONFIG_KEY = '__validator_config__'
 ROOT_VALIDATOR_CONFIG_KEY = '__root_validator_config__'
 
diff --git a/pydantic/env_settings.py b/pydantic/env_settings.py
index 6846e1020..28022d19c 100644
--- a/pydantic/env_settings.py
+++ b/pydantic/env_settings.py
@@ -6,7 +6,7 @@ from typing import AbstractSet, Any, Dict, List, Mapping, Optional, Union
 from .fields import ModelField
 from .main import BaseConfig, BaseModel, Extra
 from .typing import display_as_type
-from .utils import deep_update, sequence_like
+from .utils import deep_update, path_type, sequence_like
 
 env_file_sentinel = str(object())
 
@@ -27,11 +27,14 @@ class BaseSettings(BaseModel):
         __pydantic_self__,
         _env_file: Union[Path, str, None] = env_file_sentinel,
         _env_file_encoding: Optional[str] = None,
+        _secrets_dir: Union[Path, str, None] = None,
         **values: Any,
     ) -> None:
         # Uses something other than `self` the first arg to allow "self" as a settable attribute
         super().__init__(
-            **__pydantic_self__._build_values(values, _env_file=_env_file, _env_file_encoding=_env_file_encoding)
+            **__pydantic_self__._build_values(
+                values, _env_file=_env_file, _env_file_encoding=_env_file_encoding, _secrets_dir=_secrets_dir
+            )
         )
 
     def _build_values(
@@ -39,8 +42,41 @@ class BaseSettings(BaseModel):
         init_kwargs: Dict[str, Any],
         _env_file: Union[Path, str, None] = None,
         _env_file_encoding: Optional[str] = None,
+        _secrets_dir: Union[Path, str, None] = None,
     ) -> Dict[str, Any]:
-        return deep_update(self._build_environ(_env_file, _env_file_encoding), init_kwargs)
+        return deep_update(
+            self._build_secrets_files(_secrets_dir), self._build_environ(_env_file, _env_file_encoding), init_kwargs
+        )
+
+    def _build_secrets_files(self, _secrets_dir: Union[Path, str, None] = None) -> Dict[str, Optional[str]]:
+        """
+        Build fields from "secrets" files.
+        """
+        secrets: Dict[str, Optional[str]] = {}
+
+        secrets_dir = _secrets_dir or self.__config__.secrets_dir
+        if secrets_dir is None:
+            return secrets
+
+        secrets_path = Path(secrets_dir).expanduser()
+
+        if not secrets_path.exists():
+            raise SettingsError(f'directory "{secrets_path}" does not exist')
+        if not secrets_path.is_dir():
+            raise SettingsError(f'secrets_dir must reference a directory, not a {path_type(secrets_path)}')
+
+        for field in self.__fields__.values():
+            for env_name in field.field_info.extra['env_names']:
+                path = secrets_path / env_name
+                if path.is_file():
+                    secrets[field.alias] = path.read_text().strip()
+                elif path.exists():
+                    warnings.warn(
+                        f'attempted to load secret file "{path}" but found a {path_type(path)} instead.',
+                        stacklevel=4,
+                    )
+
+        return secrets
 
     def _build_environ(
         self, _env_file: Union[Path, str, None] = None, _env_file_encoding: Optional[str] = None
@@ -89,6 +125,7 @@ class BaseSettings(BaseModel):
         env_prefix = ''
         env_file = None
         env_file_encoding = None
+        secrets_dir = None
         validate_all = True
         extra = Extra.forbid
         arbitrary_types_allowed = True
diff --git a/pydantic/fields.py b/pydantic/fields.py
index f2c9ba050..c451770c0 100644
--- a/pydantic/fields.py
+++ b/pydantic/fields.py
@@ -797,3 +797,31 @@ class ModelField(Representation):
         if self.alt_alias:
             args.append(('alias', self.alias))
         return args
+
+
+class PrivateAttr(Representation):
+    """
+    Indicates that attribute is only used internally and never mixed with regular fields.
+
+    Types or values of private attrs are not checked by pydantic and it's up to you to keep them relevant.
+
+    Private attrs are stored in model __slots__.
+    """
+
+    __slots__ = ('default', 'default_factory')
+
+    def __init__(self, default: Any = Undefined, *, default_factory: Optional[NoArgAnyCallable] = None) -> None:
+        if default is not Undefined and default_factory is not None:
+            raise TypeError('default and default_factory args can not be used together')
+
+        self.default = default
+        self.default_factory = default_factory
+
+    def get_default(self) -> Any:
+        return smart_deepcopy(self.default) if self.default_factory is None else self.default_factory()
+
+    def __eq__(self, other: Any) -> bool:
+        return isinstance(other, self.__class__) and (self.default, self.default_factory) == (
+            other.default,
+            other.default_factory,
+        )
diff --git a/pydantic/main.py b/pydantic/main.py
index 02e27d106..bc5444a70 100644
--- a/pydantic/main.py
+++ b/pydantic/main.py
@@ -16,6 +16,7 @@ from typing import (
     List,
     Mapping,
     Optional,
+    Set,
     Tuple,
     Type,
     TypeVar,
@@ -25,21 +26,24 @@ from typing import (
     overload,
 )
 
-from .class_validators import ROOT_KEY, ValidatorGroup, extract_root_validators, extract_validators, inherit_validators
+from .class_validators import ValidatorGroup, extract_root_validators, extract_validators, inherit_validators
 from .error_wrappers import ErrorWrapper, ValidationError
 from .errors import ConfigError, DictError, ExtraError, MissingError
-from .fields import SHAPE_MAPPING, ModelField, Undefined
+from .fields import SHAPE_MAPPING, ModelField, PrivateAttr, Undefined
 from .json import custom_pydantic_encoder, pydantic_encoder
 from .parse import Protocol, load_file, load_str_bytes
 from .schema import default_ref_template, model_schema
 from .types import PyObject, StrBytes
 from .typing import AnyCallable, ForwardRef, get_origin, is_classvar, resolve_annotations, update_field_forward_refs
 from .utils import (
+    ROOT_KEY,
     ClassAttribute,
     GetterDict,
     Representation,
     ValueItems,
     generate_model_signature,
+    is_valid_field,
+    is_valid_private_name,
     lenient_issubclass,
     sequence_like,
     smart_deepcopy,
@@ -81,7 +85,6 @@ if TYPE_CHECKING:
 else:
     SchemaExtraCallable = Callable[..., None]
 
-
 try:
     import cython  # type: ignore
 except ImportError:
@@ -123,6 +126,7 @@ class BaseConfig:
     json_loads: Callable[[str], Any] = json.loads
     json_dumps: Callable[..., str] = json.dumps
     json_encoders: Dict[Type[Any], AnyCallable] = {}
+    underscore_attrs_are_private: bool = False
 
     @classmethod
     def get_field_info(cls, name: str) -> Dict[str, Any]:
@@ -189,12 +193,6 @@ def prepare_config(config: Type[BaseConfig], cls_name: str) -> None:
         config.case_sensitive = not config.case_insensitive  # type: ignore
 
 
-def is_valid_field(name: str) -> bool:
-    if not name.startswith('_'):
-        return True
-    return ROOT_KEY == name
-
-
 def validate_custom_root_type(fields: Dict[str, ModelField]) -> None:
     if len(fields) > 1:
         raise ValueError('__root__ cannot be mixed with other fields')
@@ -217,6 +215,10 @@ class ModelMetaclass(ABCMeta):
         validators: 'ValidatorListDict' = {}
 
         pre_root_validators, post_root_validators = [], []
+        private_attributes: Dict[str, PrivateAttr] = {}
+        slots: Set[str] = namespace.get('__slots__', ())
+        slots = {slots} if isinstance(slots, str) else set(slots)
+
         for base in reversed(bases):
             if _is_base_model_class_defined and issubclass(base, BaseModel) and base != BaseModel:
                 fields.update(smart_deepcopy(base.__fields__))
@@ -224,6 +226,7 @@ class ModelMetaclass(ABCMeta):
                 validators = inherit_validators(base.__validators__, validators)
                 pre_root_validators += base.__pre_root_validators__
                 post_root_validators += base.__post_root_validators__
+                private_attributes.update(base.__private_attributes__)
 
         config = inherit_config(namespace.get('Config'), config)
         validators = inherit_validators(extract_validators(namespace), validators)
@@ -263,14 +266,21 @@ class ModelMetaclass(ABCMeta):
                         class_validators=vg.get_validators(ann_name),
                         config=config,
                     )
+                elif ann_name not in namespace and config.underscore_attrs_are_private:
+                    private_attributes[ann_name] = PrivateAttr()
 
             for var_name, value in namespace.items():
-                if (
-                    var_name not in annotations
-                    and is_valid_field(var_name)
-                    and not isinstance(value, untouched_types)
-                    and var_name not in class_vars
-                ):
+                can_be_changed = var_name not in class_vars and not isinstance(value, untouched_types)
+                if isinstance(value, PrivateAttr):
+                    if not is_valid_private_name(var_name):
+                        raise NameError(
+                            f'Private attributes "{var_name}" must not be a valid field name; '
+                            f'Use sunder or dunder names, e. g. "_{var_name}" or "__{var_name}__"'
+                        )
+                    private_attributes[var_name] = value
+                elif config.underscore_attrs_are_private and is_valid_private_name(var_name) and can_be_changed:
+                    private_attributes[var_name] = PrivateAttr(default=value)
+                elif is_valid_field(var_name) and var_name not in annotations and can_be_changed:
                     validate_field_name(bases, var_name)
                     inferred = ModelField.infer(
                         name=var_name,
@@ -296,6 +306,7 @@ class ModelMetaclass(ABCMeta):
             json_encoder = pydantic_encoder
         pre_rv_new, post_rv_new = extract_root_validators(namespace)
 
+        exclude_from_namespace = fields | private_attributes.keys() | {'__slots__'}
         new_namespace = {
             '__config__': config,
             '__fields__': fields,
@@ -305,7 +316,9 @@ class ModelMetaclass(ABCMeta):
             '__schema_cache__': {},
             '__json_encoder__': staticmethod(json_encoder),
             '__custom_root_type__': _custom_root_type,
-            **{n: v for n, v in namespace.items() if n not in fields},
+            '__private_attributes__': private_attributes,
+            '__slots__': slots | private_attributes.keys(),
+            **{n: v for n, v in namespace.items() if n not in exclude_from_namespace},
         }
 
         cls = super().__new__(mcs, name, bases, new_namespace, **kwargs)
@@ -314,6 +327,9 @@ class ModelMetaclass(ABCMeta):
         return cls
 
 
+object_setattr = object.__setattr__
+
+
 class BaseModel(Representation, metaclass=ModelMetaclass):
     if TYPE_CHECKING:
         # populated by the metaclass, defined here to help IDEs only
@@ -327,6 +343,8 @@ class BaseModel(Representation, metaclass=ModelMetaclass):
         __schema_cache__: 'DictAny' = {}
         __custom_root_type__: bool = False
         __signature__: 'Signature'
+        __private_attributes__: Dict[str, Any]
+        __fields_set__: SetStr = set()
 
     Config = BaseConfig
     __slots__ = ('__dict__', '__fields_set__')
@@ -339,17 +357,18 @@ class BaseModel(Representation, metaclass=ModelMetaclass):
         Raises ValidationError if the input data cannot be parsed to form a valid model.
         """
         # Uses something other than `self` the first arg to allow "self" as a settable attribute
-        if TYPE_CHECKING:
-            __pydantic_self__.__dict__: Dict[str, Any] = {}
-            __pydantic_self__.__fields_set__: 'SetStr' = set()
         values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
         if validation_error:
             raise validation_error
-        object.__setattr__(__pydantic_self__, '__dict__', values)
-        object.__setattr__(__pydantic_self__, '__fields_set__', fields_set)
+        object_setattr(__pydantic_self__, '__dict__', values)
+        object_setattr(__pydantic_self__, '__fields_set__', fields_set)
+        __pydantic_self__._init_private_attributes()
 
     @no_type_check
-    def __setattr__(self, name, value):
+    def __setattr__(self, name, value):  # noqa: C901 (ignore complexity)
+        if name in self.__private_attributes__:
+            return object_setattr(self, name, value)
+
         if self.__config__.extra is not Extra.allow and name not in self.__fields__:
             raise ValueError(f'"{self.__class__.__name__}" object has no field "{name}"')
         elif not self.__config__.allow_mutation:
@@ -387,11 +406,24 @@ class BaseModel(Representation, metaclass=ModelMetaclass):
         self.__fields_set__.add(name)
 
     def __getstate__(self) -> 'DictAny':
-        return {'__dict__': self.__dict__, '__fields_set__': self.__fields_set__}
+        return {
+            '__dict__': self.__dict__,
+            '__fields_set__': self.__fields_set__,
+            '__private_attribute_values__': {k: getattr(self, k, Undefined) for k in self.__private_attributes__},
+        }
 
     def __setstate__(self, state: 'DictAny') -> None:
-        object.__setattr__(self, '__dict__', state['__dict__'])
-        object.__setattr__(self, '__fields_set__', state['__fields_set__'])
+        object_setattr(self, '__dict__', state['__dict__'])
+        object_setattr(self, '__fields_set__', state['__fields_set__'])
+        for name, value in state.get('__private_attribute_values__', {}).items():
+            if value is not Undefined:
+                object_setattr(self, name, value)
+
+    def _init_private_attributes(self) -> None:
+        for name, private_attr in self.__private_attributes__.items():
+            default = private_attr.get_default()
+            if default is not Undefined:
+                object_setattr(self, name, default)
 
     def dict(
         self,
@@ -530,8 +562,9 @@ class BaseModel(Representation, metaclass=ModelMetaclass):
         values, fields_set, validation_error = validate_model(cls, obj)
         if validation_error:
             raise validation_error
-        object.__setattr__(m, '__dict__', values)
-        object.__setattr__(m, '__fields_set__', fields_set)
+        object_setattr(m, '__dict__', values)
+        object_setattr(m, '__fields_set__', fields_set)
+        m._init_private_attributes()
         return m
 
     @classmethod
@@ -544,10 +577,11 @@ class BaseModel(Representation, metaclass=ModelMetaclass):
         # default field values
         fields_values = {name: field.get_default() for name, field in cls.__fields__.items() if not field.required}
         fields_values.update(values)
-        object.__setattr__(m, '__dict__', fields_values)
+        object_setattr(m, '__dict__', fields_values)
         if _fields_set is None:
             _fields_set = set(values.keys())
-        object.__setattr__(m, '__fields_set__', _fields_set)
+        object_setattr(m, '__fields_set__', _fields_set)
+        m._init_private_attributes()
         return m
 
     def copy(
@@ -580,8 +614,15 @@ class BaseModel(Representation, metaclass=ModelMetaclass):
 
         cls = self.__class__
         m = cls.__new__(cls)
-        object.__setattr__(m, '__dict__', v)
-        object.__setattr__(m, '__fields_set__', self.__fields_set__.copy())
+        object_setattr(m, '__dict__', v)
+        object_setattr(m, '__fields_set__', self.__fields_set__.copy())
+        for name in self.__private_attributes__:
+            value = getattr(self, name, Undefined)
+            if value is not Undefined:
+                if deep:
+                    value = deepcopy(value)
+                object_setattr(m, name, value)
+
         return m
 
     @classmethod
diff --git a/pydantic/schema.py b/pydantic/schema.py
index 079e08c19..be2b454ce 100644
--- a/pydantic/schema.py
+++ b/pydantic/schema.py
@@ -25,7 +25,6 @@ from typing import (
 )
 from uuid import UUID
 
-from .class_validators import ROOT_KEY
 from .fields import (
     SHAPE_FROZENSET,
     SHAPE_ITERABLE,
@@ -59,7 +58,7 @@ from .types import (
     constr,
 )
 from .typing import ForwardRef, Literal, get_args, get_origin, is_callable_type, is_literal_type, literal_values
-from .utils import get_model, lenient_issubclass, sequence_like
+from .utils import ROOT_KEY, get_model, lenient_issubclass, sequence_like
 
 if TYPE_CHECKING:
     from .dataclasses import DataclassType  # noqa: F401
diff --git a/pydantic/utils.py b/pydantic/utils.py
index e67f336c1..8019a1924 100644
--- a/pydantic/utils.py
+++ b/pydantic/utils.py
@@ -28,6 +28,7 @@ from .version import version_info
 
 if TYPE_CHECKING:
     from inspect import Signature
+    from pathlib import Path
 
     from .dataclasses import DataclassType  # noqa: F401
     from .fields import ModelField  # noqa: F401
@@ -45,14 +46,19 @@ __all__ = (
     'almost_equal_floats',
     'get_model',
     'to_camel',
+    'is_valid_field',
+    'smart_deepcopy',
     'PyObjectStr',
     'Representation',
     'GetterDict',
     'ValueItems',
     'version_info',  # required here to match behaviour in v1.3
     'ClassAttribute',
+    'path_type',
+    'ROOT_KEY',
 )
 
+ROOT_KEY = '__root__'
 # these are types that are returned unchanged by deepcopy
 IMMUTABLE_NON_COLLECTIONS_TYPES: Set[Type[Any]] = {
     int,
@@ -160,13 +166,14 @@ def in_ipython() -> bool:
 KeyType = TypeVar('KeyType')
 
 
-def deep_update(mapping: Dict[KeyType, Any], updating_mapping: Dict[KeyType, Any]) -> Dict[KeyType, Any]:
+def deep_update(mapping: Dict[KeyType, Any], *updating_mappings: Dict[KeyType, Any]) -> Dict[KeyType, Any]:
     updated_mapping = mapping.copy()
-    for k, v in updating_mapping.items():
-        if k in mapping and isinstance(mapping[k], dict) and isinstance(v, dict):
-            updated_mapping[k] = deep_update(mapping[k], v)
-        else:
-            updated_mapping[k] = v
+    for updating_mapping in updating_mappings:
+        for k, v in updating_mapping.items():
+            if k in updated_mapping and isinstance(updated_mapping[k], dict) and isinstance(v, dict):
+                updated_mapping[k] = deep_update(updated_mapping[k], v)
+            else:
+                updated_mapping[k] = v
     return updated_mapping
 
 
@@ -574,6 +581,30 @@ class ClassAttribute:
         raise AttributeError(f'{self.name!r} attribute of {owner.__name__!r} is class-only')
 
 
+path_types = {
+    'is_dir': 'directory',
+    'is_file': 'file',
+    'is_mount': 'mount point',
+    'is_symlink': 'symlink',
+    'is_block_device': 'block device',
+    'is_char_device': 'char device',
+    'is_fifo': 'FIFO',
+    'is_socket': 'socket',
+}
+
+
+def path_type(p: 'Path') -> str:
+    """
+    Find out what sort of thing a path is.
+    """
+    assert p.exists(), 'path does not exist'
+    for method, name in path_types.items():
+        if getattr(p, method)():
+            return name
+
+    return 'unknown'
+
+
 Obj = TypeVar('Obj')
 
 
@@ -591,3 +622,13 @@ def smart_deepcopy(obj: Obj) -> Obj:
         # faster way for empty collections, no need to copy its members
         return obj if obj_type is tuple else obj.copy()  # type: ignore  # tuple doesn't have copy method
     return deepcopy(obj)  # slowest way when we actually might need a deepcopy
+
+
+def is_valid_field(name: str) -> bool:
+    if not name.startswith('_'):
+        return True
+    return ROOT_KEY == name
+
+
+def is_valid_private_name(name: str) -> bool:
+    return not is_valid_field(name) and name not in {'__annotations__', '__module__', '__annotations__', '__qualname__'}
diff --git a/requirements.txt b/requirements.txt
index ee9342558..80e8d9c1a 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,7 +1,7 @@
 # requirements for compilation and from setup.py so dependabot prompts us to test with latest version of these packages
 
 Cython==0.29.21;sys_platform!='win32'
-devtools==0.6.0
+devtools==0.6.1
 email-validator==1.1.1
 dataclasses==0.6; python_version < '3.7'
 typing-extensions==3.7.4.1; python_version < '3.8'


Test patch:
diff --git a/tests/conftest.py b/tests/conftest.py
index 098df7a02..6cde3411b 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -1,13 +1,13 @@
+import importlib
 import inspect
 import os
 import secrets
-import subprocess
 import sys
 import textwrap
-from importlib.machinery import SourceFileLoader
 from types import FunctionType
 
 import pytest
+from _pytest.assertion.rewrite import AssertionRewritingHook
 
 
 def _extract_source_code_from_function(function):
@@ -26,11 +26,11 @@ def _extract_source_code_from_function(function):
     return textwrap.dedent(code_lines)
 
 
-def _create_module(code, tmp_path, name):
+def _create_module_file(code, tmp_path, name):
     name = f'{name}_{secrets.token_hex(5)}'
     path = tmp_path / f'{name}.py'
     path.write_text(code)
-    return name, path
+    return name, str(path)
 
 
 class SetEnv:
@@ -57,36 +57,31 @@ def env():
 
 @pytest.fixture
 def create_module(tmp_path, request):
-    def run(module_source_code=None):
+    def run(source_code_or_function, rewrite_assertions=True):
         """
-        Creates module and loads it with SourceFileLoader().load_module()
-        """
-        if isinstance(module_source_code, FunctionType):
-            module_source_code = _extract_source_code_from_function(module_source_code)
-        name, path = _create_module(module_source_code, tmp_path, request.node.name)
-        return SourceFileLoader(name, str(path)).load_module()
-
-    return run
+        Create module object, execute it and return
+        Can be used as a decorator of the function from the source code of which the module will be constructed
 
+        :param source_code_or_function string or function with body as a source code for created module
+        :param rewrite_assertions: whether to rewrite assertions in module or not
 
-@pytest.fixture
-def run_as_module(tmp_path, request):
-    def run(module_source_code=None):
-        """
-        Creates module source and runs it in subprocess
-
-        This way is much slower than SourceFileLoader().load_module(),
-        but executes module as __main__ with a clear stack (https://docs.python.org/3/library/__main__.html)
         """
-        if isinstance(module_source_code, FunctionType):
-            module_source_code = _extract_source_code_from_function(module_source_code)
-        _, path = _create_module(module_source_code, tmp_path, request.node.name)
-        result = subprocess.run([sys.executable, str(path)], stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=5)
-        if result.returncode != 0:
-            pytest.fail(
-                f'Running {path} failed with non-zero return code: {result.returncode}\n'
-                f'Captured stdout:\n{result.stdout.decode()}\n'
-                f'Captured stderr:\n{result.stderr.decode()}'
-            )
+        if isinstance(source_code_or_function, FunctionType):
+            source_code = _extract_source_code_from_function(source_code_or_function)
+        else:
+            source_code = source_code_or_function
+
+        module_name, filename = _create_module_file(source_code, tmp_path, request.node.name)
+
+        if rewrite_assertions:
+            loader = AssertionRewritingHook(config=request.config)
+            loader.mark_rewrite(module_name)
+        else:
+            loader = None
+
+        spec = importlib.util.spec_from_file_location(module_name, filename, loader=loader)
+        sys.modules[module_name] = module = importlib.util.module_from_spec(spec)
+        spec.loader.exec_module(module)
+        return module
 
     return run
diff --git a/tests/test_construction.py b/tests/test_construction.py
index 93abb6e03..00657153c 100644
--- a/tests/test_construction.py
+++ b/tests/test_construction.py
@@ -3,7 +3,7 @@ from typing import Any, List
 
 import pytest
 
-from pydantic import BaseModel, Field
+from pydantic import BaseModel, Field, PrivateAttr
 
 
 class Model(BaseModel):
@@ -58,6 +58,8 @@ def test_simple_copy():
 
 
 class ModelTwo(BaseModel):
+    __foo__ = PrivateAttr({'private'})
+
     a: float
     b: int = 10
     c: str = 'foobar'
@@ -66,6 +68,7 @@ class ModelTwo(BaseModel):
 
 def test_deep_copy():
     m = ModelTwo(a=24, d=Model(a='12'))
+    m.__foo__ = {'new value'}
     m2 = m.copy(deep=True)
 
     assert m.a == m2.a == 24
@@ -74,6 +77,8 @@ def test_deep_copy():
     assert m.d is not m2.d
     assert m == m2
     assert m.__fields__ == m2.__fields__
+    assert m.__foo__ == m2.__foo__
+    assert m.__foo__ is not m2.__foo__
 
 
 def test_copy_exclude():
@@ -215,6 +220,7 @@ def test_recursive_pickle():
     assert m.d.a == 123.45
     assert m2.d.a == 123.45
     assert m.__fields__ == m2.__fields__
+    assert m.__foo__ == m2.__foo__
 
 
 def test_immutable_copy():
diff --git a/tests/test_decorator.py b/tests/test_decorator.py
index 75928f27f..4710d5141 100644
--- a/tests/test_decorator.py
+++ b/tests/test_decorator.py
@@ -124,6 +124,7 @@ def test_var_args_kwargs():
 @skip_pre_38
 def test_positional_only(create_module):
     module = create_module(
+        # language=Python
         """
 from pydantic import validate_arguments
 
diff --git a/tests/test_forward_ref.py b/tests/test_forward_ref.py
index c6431a731..f762eb11c 100644
--- a/tests/test_forward_ref.py
+++ b/tests/test_forward_ref.py
@@ -10,6 +10,7 @@ skip_pre_37 = pytest.mark.skipif(sys.version_info < (3, 7), reason='testing >= 3
 @skip_pre_37
 def test_postponed_annotations(create_module):
     module = create_module(
+        # language=Python
         """
 from __future__ import annotations
 from pydantic import BaseModel
@@ -25,6 +26,7 @@ class Model(BaseModel):
 @skip_pre_37
 def test_postponed_annotations_optional(create_module):
     module = create_module(
+        # language=Python
         """
 from __future__ import annotations
 from typing import Optional
@@ -39,64 +41,60 @@ class Model(BaseModel):
 
 
 def test_basic_forward_ref(create_module):
-    module = create_module(
-        """
-from typing import Optional
-from pydantic import BaseModel
-from pydantic.typing import ForwardRef
+    @create_module
+    def module():
+        from typing import Optional
 
-class Foo(BaseModel):
-    a: int
+        from pydantic import BaseModel
+        from pydantic.typing import ForwardRef
 
-FooRef = ForwardRef('Foo')
+        class Foo(BaseModel):
+            a: int
 
-class Bar(BaseModel):
-    b: Optional[FooRef]
-"""
-    )
+        FooRef = ForwardRef('Foo')
+
+        class Bar(BaseModel):
+            b: Optional[FooRef]
 
     assert module.Bar().dict() == {'b': None}
     assert module.Bar(b={'a': '123'}).dict() == {'b': {'a': 123}}
 
 
 def test_self_forward_ref_module(create_module):
-    module = create_module(
-        """
-from pydantic import BaseModel
-from pydantic.typing import ForwardRef
+    @create_module
+    def module():
+        from pydantic import BaseModel
+        from pydantic.typing import ForwardRef
 
-Foo = ForwardRef('Foo')
+        Foo = ForwardRef('Foo')
 
-class Foo(BaseModel):
-    a: int = 123
-    b: 'Foo' = None
+        class Foo(BaseModel):
+            a: int = 123
+            b: 'Foo' = None
 
-Foo.update_forward_refs()
-    """
-    )
+        Foo.update_forward_refs()
 
     assert module.Foo().dict() == {'a': 123, 'b': None}
     assert module.Foo(b={'a': '321'}).dict() == {'a': 123, 'b': {'a': 321, 'b': None}}
 
 
 def test_self_forward_ref_collection(create_module):
-    module = create_module(
-        """
-from typing import List, Dict
-from pydantic import BaseModel
-from pydantic.typing import ForwardRef
+    @create_module
+    def module():
+        from typing import Dict, List
 
-Foo = ForwardRef('Foo')
+        from pydantic import BaseModel
+        from pydantic.typing import ForwardRef
 
-class Foo(BaseModel):
-    a: int = 123
-    b: Foo = None
-    c: List[Foo] = []
-    d: Dict[str, Foo] = {}
+        Foo = ForwardRef('Foo')
 
-Foo.update_forward_refs()
-    """
-    )
+        class Foo(BaseModel):
+            a: int = 123
+            b: Foo = None
+            c: List[Foo] = []
+            d: Dict[str, Foo] = {}
+
+        Foo.update_forward_refs()
 
     assert module.Foo().dict() == {'a': 123, 'b': None, 'c': [], 'd': {}}
     assert module.Foo(b={'a': '321'}, c=[{'a': 234}], d={'bar': {'a': 345}}).dict() == {
@@ -114,56 +112,53 @@ Foo.update_forward_refs()
 
 
 def test_self_forward_ref_local(create_module):
-    module = create_module(
-        """
-from pydantic import BaseModel
-from pydantic.typing import ForwardRef
+    @create_module
+    def module():
+        from pydantic import BaseModel
+        from pydantic.typing import ForwardRef
 
-def main():
-    Foo = ForwardRef('Foo')
+        def main():
+            Foo = ForwardRef('Foo')
 
-    class Foo(BaseModel):
-        a: int = 123
-        b: Foo = None
+            class Foo(BaseModel):
+                a: int = 123
+                b: Foo = None
+
+            Foo.update_forward_refs()
+            return Foo
 
-    Foo.update_forward_refs()
-    return Foo
-    """
-    )
     Foo = module.main()
     assert Foo().dict() == {'a': 123, 'b': None}
     assert Foo(b={'a': '321'}).dict() == {'a': 123, 'b': {'a': 321, 'b': None}}
 
 
 def test_missing_update_forward_refs(create_module):
-    module = create_module(
-        """
-from pydantic import BaseModel
-from pydantic.typing import ForwardRef
+    @create_module
+    def module():
+        from pydantic import BaseModel
+        from pydantic.typing import ForwardRef
 
-Foo = ForwardRef('Foo')
+        Foo = ForwardRef('Foo')
+
+        class Foo(BaseModel):
+            a: int = 123
+            b: Foo = None
 
-class Foo(BaseModel):
-    a: int = 123
-    b: Foo = None
-    """
-    )
     with pytest.raises(ConfigError) as exc_info:
         module.Foo(b=123)
     assert str(exc_info.value).startswith('field "b" not yet prepared so type is still a ForwardRef')
 
 
 def test_forward_ref_dataclass(create_module):
-    module = create_module(
-        """
-from pydantic import AnyUrl
-from pydantic.dataclasses import dataclass
+    @create_module
+    def module():
+        from pydantic import AnyUrl
+        from pydantic.dataclasses import dataclass
+
+        @dataclass
+        class Dataclass:
+            url: AnyUrl
 
-@dataclass
-class Dataclass:
-    url: AnyUrl
-    """
-    )
     m = module.Dataclass('http://example.com  ')
     assert m.url == 'http://example.com'
 
@@ -171,6 +166,7 @@ class Dataclass:
 @skip_pre_37
 def test_forward_ref_dataclass_with_future_annotations(create_module):
     module = create_module(
+        # language=Python
         """
 from __future__ import annotations
 from pydantic import AnyUrl
@@ -186,28 +182,25 @@ class Dataclass:
 
 
 def test_forward_ref_sub_types(create_module):
-    module = create_module(
-        """
-from typing import Union
-from pydantic import BaseModel
-from pydantic.typing import ForwardRef
-
-class Leaf(BaseModel):
-    a: str
+    @create_module
+    def module():
+        from typing import Union
 
+        from pydantic import BaseModel
+        from pydantic.typing import ForwardRef
 
-TreeType = Union[ForwardRef('Node'), Leaf]
+        class Leaf(BaseModel):
+            a: str
 
+        TreeType = Union[ForwardRef('Node'), Leaf]
 
-class Node(BaseModel):
-    value: int
-    left: TreeType
-    right: TreeType
+        class Node(BaseModel):
+            value: int
+            left: TreeType
+            right: TreeType
 
+        Node.update_forward_refs()
 
-Node.update_forward_refs()
-    """
-    )
     Node = module.Node
     Leaf = module.Leaf
     data = {'value': 3, 'left': {'a': 'foo'}, 'right': {'value': 5, 'left': {'a': 'bar'}, 'right': {'a': 'buzz'}}}
@@ -218,28 +211,25 @@ Node.update_forward_refs()
 
 
 def test_forward_ref_nested_sub_types(create_module):
-    module = create_module(
-        """
-from typing import Tuple, Union
-from pydantic import BaseModel
-from pydantic.typing import ForwardRef
-
-class Leaf(BaseModel):
-    a: str
+    @create_module
+    def module():
+        from typing import Tuple, Union
 
+        from pydantic import BaseModel
+        from pydantic.typing import ForwardRef
 
-TreeType = Union[Union[Tuple[ForwardRef('Node'), str], int], Leaf]
+        class Leaf(BaseModel):
+            a: str
 
+        TreeType = Union[Union[Tuple[ForwardRef('Node'), str], int], Leaf]
 
-class Node(BaseModel):
-    value: int
-    left: TreeType
-    right: TreeType
+        class Node(BaseModel):
+            value: int
+            left: TreeType
+            right: TreeType
 
+        Node.update_forward_refs()
 
-Node.update_forward_refs()
-    """
-    )
     Node = module.Node
     Leaf = module.Leaf
     data = {
@@ -254,18 +244,18 @@ Node.update_forward_refs()
 
 
 def test_self_reference_json_schema(create_module):
-    module = create_module(
-        """
-from typing import List
-from pydantic import BaseModel
+    @create_module
+    def module():
+        from typing import List
 
-class Account(BaseModel):
-  name: str
-  subaccounts: List['Account'] = []
+        from pydantic import BaseModel
+
+        class Account(BaseModel):
+            name: str
+            subaccounts: List['Account'] = []
+
+        Account.update_forward_refs()
 
-Account.update_forward_refs()
-    """
-    )
     Account = module.Account
     assert Account.schema() == {
         '$ref': '#/definitions/Account',
@@ -291,6 +281,7 @@ Account.update_forward_refs()
 @skip_pre_37
 def test_self_reference_json_schema_with_future_annotations(create_module):
     module = create_module(
+        # language=Python
         """
 from __future__ import annotations
 from typing import List
@@ -326,23 +317,23 @@ Account.update_forward_refs()
 
 
 def test_circular_reference_json_schema(create_module):
-    module = create_module(
-        """
-from typing import List
-from pydantic import BaseModel
+    @create_module
+    def module():
+        from typing import List
 
-class Owner(BaseModel):
-  account: 'Account'
+        from pydantic import BaseModel
 
-class Account(BaseModel):
-  name: str
-  owner: 'Owner'
-  subaccounts: List['Account'] = []
+        class Owner(BaseModel):
+            account: 'Account'
+
+        class Account(BaseModel):
+            name: str
+            owner: 'Owner'
+            subaccounts: List['Account'] = []
+
+        Account.update_forward_refs()
+        Owner.update_forward_refs()
 
-Account.update_forward_refs()
-Owner.update_forward_refs()
-    """
-    )
     Account = module.Account
     assert Account.schema() == {
         '$ref': '#/definitions/Account',
@@ -375,6 +366,7 @@ Owner.update_forward_refs()
 @skip_pre_37
 def test_circular_reference_json_schema_with_future_annotations(create_module):
     module = create_module(
+        # language=Python
         """
 from __future__ import annotations
 from typing import List
@@ -422,28 +414,29 @@ Owner.update_forward_refs()
 
 
 def test_forward_ref_with_field(create_module):
-    create_module(
-        """
-from typing import List
-from pydantic import BaseModel, Field
-from pydantic.typing import ForwardRef
+    @create_module
+    def module():
+        from typing import List
 
-Foo = ForwardRef('Foo')
+        import pytest
 
-try:
-    class Foo(BaseModel):
-        c: List[Foo] = Field(..., gt=0)
-except ValueError:
-    pass
-else:
-    raise AssertionError('error not raised')
-    """
-    )
+        from pydantic import BaseModel, Field
+        from pydantic.typing import ForwardRef
+
+        Foo = ForwardRef('Foo')
+
+        with pytest.raises(
+            ValueError, match='On field "c" the following field constraints are set but not enforced: gt.'
+        ):
+
+            class Foo(BaseModel):
+                c: List[Foo] = Field(..., gt=0)
 
 
 @skip_pre_37
 def test_forward_ref_optional(create_module):
     module = create_module(
+        # language=Python
         """
 from __future__ import annotations
 from pydantic import BaseModel, Field
diff --git a/tests/test_generics.py b/tests/test_generics.py
index 6936216ca..a37c2a8e5 100644
--- a/tests/test_generics.py
+++ b/tests/test_generics.py
@@ -708,24 +708,11 @@ def test_generic_model_redefined_without_cache_fail(create_module):
         assert cause.args[0] == expected_message, f'{cause.args[0]} != {expected_message}'
 
 
-def test_get_caller_module_name_from_function():
-    def get_current_module_name():
-        return get_caller_module_name()
-
-    assert get_current_module_name() == __name__
-
-
-def test_get_caller_module_name_from_module(create_module):
+def test_get_caller_module_name(create_module):
     @create_module
     def module():
         from pydantic.generics import get_caller_module_name
 
-        def get_current_module_name():
-            return get_caller_module_name()
-
-        module_name = get_current_module_name()
-        assert module_name == __name__, f'{module_name} != {__name__}'
-
         def get_current_module_name():
             return get_caller_module_name()
 
@@ -757,31 +744,37 @@ def test_is_call_from_module(create_module):
         function()
 
 
-def test_is_call_from_module_called_in_module(run_as_module):
-    @run_as_module
+def test_is_call_from_module_called_in_module(create_module):
+    @create_module
     def module():
+        from unittest.mock import patch
+
         import pytest
 
         from pydantic.generics import is_call_from_module
 
         with pytest.raises(RuntimeError, match='This function must be used inside another function') as exc_info:
-            is_call_from_module()
+            with patch('inspect.stack', new=lambda: [..., ...]):
+                is_call_from_module()
 
         e = exc_info.value
-        assert isinstance(e.__cause__, IndexError), e.__cause__
-        assert isinstance(e.__context__, IndexError), e.__context__
+        assert isinstance(e.__cause__, IndexError)
+        assert isinstance(e.__context__, IndexError)
 
 
-def test_get_caller_module_called_from_module(run_as_module):
-    @run_as_module
+def test_get_caller_module_called_from_module(create_module):
+    @create_module
     def module():
+        from unittest.mock import patch
+
         import pytest
 
         from pydantic.generics import get_caller_module_name
 
         with pytest.raises(RuntimeError, match='This function must be used inside another function') as exc_info:
-            get_caller_module_name()
+            with patch('inspect.stack', new=lambda: [..., ...]):
+                get_caller_module_name()
 
         e = exc_info.value
-        assert isinstance(e.__cause__, IndexError), e.__cause__
-        assert isinstance(e.__context__, IndexError), e.__context__
+        assert isinstance(e.__cause__, IndexError)
+        assert isinstance(e.__context__, IndexError)
diff --git a/tests/test_private_attributes.py b/tests/test_private_attributes.py
new file mode 100644
index 000000000..07f702c7f
--- /dev/null
+++ b/tests/test_private_attributes.py
@@ -0,0 +1,162 @@
+from typing import ClassVar
+
+import pytest
+
+from pydantic import BaseModel, Extra, PrivateAttr
+from pydantic.fields import Undefined
+
+
+def test_private_attribute():
+    default = {'a': {}}
+
+    class Model(BaseModel):
+        __foo__ = PrivateAttr(default)
+
+    assert Model.__slots__ == {'__foo__'}
+    assert repr(Model.__foo__) == "<member '__foo__' of 'Model' objects>"
+    assert Model.__private_attributes__ == {'__foo__': PrivateAttr(default)}
+
+    m = Model()
+    assert m.__foo__ == default
+    assert m.__foo__ is not default
+    assert m.__foo__['a'] is not default['a']
+
+    m.__foo__ = None
+    assert m.__foo__ is None
+
+    assert m.dict() == {}
+    assert m.__dict__ == {}
+
+
+def test_private_attribute_factory():
+    default = {'a': {}}
+
+    def factory():
+        return default
+
+    class Model(BaseModel):
+        __foo__ = PrivateAttr(default_factory=factory)
+
+    assert Model.__slots__ == {'__foo__'}
+    assert repr(Model.__foo__) == "<member '__foo__' of 'Model' objects>"
+    assert Model.__private_attributes__ == {'__foo__': PrivateAttr(default_factory=factory)}
+
+    m = Model()
+    assert m.__foo__ == default
+    assert m.__foo__ is default
+    assert m.__foo__['a'] is default['a']
+
+    m.__foo__ = None
+    assert m.__foo__ is None
+
+    assert m.dict() == {}
+    assert m.__dict__ == {}
+
+
+def test_private_attribute_annotation():
+    class Model(BaseModel):
+        __foo__: str
+
+        class Config:
+            underscore_attrs_are_private = True
+
+    assert Model.__slots__ == {'__foo__'}
+    assert repr(Model.__foo__) == "<member '__foo__' of 'Model' objects>"
+    assert Model.__private_attributes__ == {'__foo__': PrivateAttr(Undefined)}
+
+    m = Model()
+    with pytest.raises(AttributeError):
+        m.__foo__
+
+    m.__foo__ = '123'
+    assert m.__foo__ == '123'
+
+    m.__foo__ = None
+    assert m.__foo__ is None
+
+    del m.__foo__
+
+    with pytest.raises(AttributeError):
+        m.__foo__
+
+    m.__foo__ = '123'
+    assert m.__foo__ == '123'
+
+    assert m.dict() == {}
+    assert m.__dict__ == {}
+
+
+def test_underscore_attrs_are_private():
+    class Model(BaseModel):
+        __foo__: str = 'abc'
+        __bar__: ClassVar[str] = 'cba'
+
+        class Config:
+            underscore_attrs_are_private = True
+
+    assert Model.__slots__ == {'__foo__'}
+    assert repr(Model.__foo__) == "<member '__foo__' of 'Model' objects>"
+    assert Model.__bar__ == 'cba'
+    assert Model.__private_attributes__ == {'__foo__': PrivateAttr('abc')}
+
+    m = Model()
+    assert m.__foo__ == 'abc'
+    m.__foo__ = None
+    assert m.__foo__ is None
+
+    with pytest.raises(ValueError, match='"Model" object has no field "__bar__"'):
+        m.__bar__ = 1
+
+
+def test_private_attribute_intersection_with_extra_field():
+    class Model(BaseModel):
+        __foo__ = PrivateAttr('private_attribute')
+
+        class Config:
+            extra = Extra.allow
+
+    assert Model.__slots__ == {'__foo__'}
+    m = Model(__foo__='field')
+    assert m.__foo__ == 'private_attribute'
+    assert m.__dict__ == m.dict() == {'__foo__': 'field'}
+
+    m.__foo__ = 'still_private'
+    assert m.__foo__ == 'still_private'
+    assert m.__dict__ == m.dict() == {'__foo__': 'field'}
+
+
+def test_private_attribute_invalid_name():
+    with pytest.raises(
+        NameError,
+        match='Private attributes "foo" must not be a valid field name; '
+        'Use sunder or dunder names, e. g. "_foo" or "__foo__"',
+    ):
+
+        class Model(BaseModel):
+            foo = PrivateAttr()
+
+
+def test_slots_are_ignored():
+    class Model(BaseModel):
+        __slots__ = (
+            'foo',
+            '_bar',
+        )
+
+        def __init__(self):
+            super().__init__()
+            for attr_ in self.__slots__:
+                object.__setattr__(self, attr_, 'spam')
+
+    assert Model.__private_attributes__ == {}
+    assert set(Model.__slots__) == {'foo', '_bar'}
+    m = Model()
+    for attr in Model.__slots__:
+        assert object.__getattribute__(m, attr) == 'spam'
+        with pytest.raises(ValueError, match=f'"Model" object has no field "{attr}"'):
+            setattr(m, attr, 'not spam')
+
+
+def test_default_and_default_factory_used_error():
+    with pytest.raises(TypeError, match='default and default_factory args can not be used together'):
+        PrivateAttr(default=123, default_factory=lambda: 321)
diff --git a/tests/test_settings.py b/tests/test_settings.py
index 6e734481c..b954b4279 100644
--- a/tests/test_settings.py
+++ b/tests/test_settings.py
@@ -1,11 +1,12 @@
 import os
+import sys
 import uuid
 from pathlib import Path
 from typing import Dict, List, Optional, Set
 
 import pytest
 
-from pydantic import BaseModel, BaseSettings, Field, NoneStr, ValidationError, dataclasses
+from pydantic import BaseModel, BaseSettings, Field, HttpUrl, NoneStr, SecretStr, ValidationError, dataclasses
 from pydantic.env_settings import SettingsError, read_env_file
 
 try:
@@ -423,7 +424,7 @@ def test_env_takes_precedence(env):
         foo: int
         bar: str
 
-        def _build_values(self, init_kwargs, _env_file, _env_file_encoding):
+        def _build_values(self, init_kwargs, _env_file, _env_file_encoding, _secrets_dir):
             return {**init_kwargs, **self._build_environ()}
 
     env.set('BAR', 'env setting')
@@ -443,7 +444,7 @@ def test_config_file_settings_nornir(env):
         param_b: str
         param_c: str
 
-        def _build_values(self, init_kwargs, _env_file, _env_file_encoding):
+        def _build_values(self, init_kwargs, _env_file, _env_file_encoding, _secrets_dir):
             config_settings = init_kwargs.pop('__config_settings__')
             return {**config_settings, **init_kwargs, **self._build_environ()}
 
@@ -754,3 +755,100 @@ def test_frozenset(env):
     assert Settings().dict() == {'foo': 'default foo'}
     env.set('foo_a', 'x')
     assert Settings().dict() == {'foo': 'x'}
+
+
+def test_secrets_path(tmp_path):
+    p = tmp_path / 'foo'
+    p.write_text('foo_secret_value_str')
+
+    class Settings(BaseSettings):
+        foo: str
+
+        class Config:
+            secrets_dir = tmp_path
+
+    assert Settings().dict() == {'foo': 'foo_secret_value_str'}
+
+
+def test_secrets_path_url(tmp_path):
+    (tmp_path / 'foo').write_text('http://www.example.com')
+    (tmp_path / 'bar').write_text('snap')
+
+    class Settings(BaseSettings):
+        foo: HttpUrl
+        bar: SecretStr
+
+        class Config:
+            secrets_dir = tmp_path
+
+    assert Settings().dict() == {'foo': 'http://www.example.com', 'bar': SecretStr('snap')}
+
+
+def test_secrets_missing(tmp_path):
+    class Settings(BaseSettings):
+        foo: str
+
+        class Config:
+            secrets_dir = tmp_path
+
+    with pytest.raises(ValidationError) as exc_info:
+        Settings()
+    assert exc_info.value.errors() == [{'loc': ('foo',), 'msg': 'field required', 'type': 'value_error.missing'}]
+
+
+def test_secrets_invalid_secrets_dir(tmp_path):
+    p1 = tmp_path / 'foo'
+    p1.write_text('foo_secret_value_str')
+
+    class Settings(BaseSettings):
+        foo: str
+
+        class Config:
+            secrets_dir = p1
+
+    with pytest.raises(SettingsError, match='secrets_dir must reference a directory, not a file'):
+        Settings()
+
+
+@pytest.mark.skipif(sys.platform.startswith('win'), reason='windows paths break regex')
+def test_secrets_missing_location(tmp_path):
+    class Settings(BaseSettings):
+        foo: str
+
+        class Config:
+            secrets_dir = tmp_path / 'does_not_exist'
+
+    with pytest.raises(SettingsError, match=f'directory "{tmp_path}/does_not_exist" does not exist'):
+        Settings()
+
+
+@pytest.mark.skipif(sys.platform.startswith('win'), reason='windows paths break regex')
+def test_secrets_file_is_a_directory(tmp_path):
+    p1 = tmp_path / 'foo'
+    p1.mkdir()
+
+    class Settings(BaseSettings):
+        foo: Optional[str]
+
+        class Config:
+            secrets_dir = tmp_path
+
+    with pytest.warns(UserWarning, match=f'attempted to load secret file "{tmp_path}/foo" but found a directory inste'):
+        Settings()
+
+
+@pytest.mark.skipif(not dotenv, reason='python-dotenv not installed')
+def test_secrets_dotenv_precedence(tmp_path):
+    s = tmp_path / 'foo'
+    s.write_text('foo_secret_value_str')
+
+    e = tmp_path / '.env'
+    e.write_text('foo=foo_env_value_str')
+
+    class Settings(BaseSettings):
+        foo: str
+
+        class Config:
+            secrets_dir = tmp_path
+
+    assert Settings(_env_file=e).dict() == {'foo': 'foo_env_value_str'}
diff --git a/tests/test_utils.py b/tests/test_utils.py
index 989176c20..e3db6beb1 100644
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -21,6 +21,7 @@ from pydantic.utils import (
     get_model,
     import_string,
     lenient_issubclass,
+    path_type,
     smart_deepcopy,
     truncate,
     unique_list,
@@ -357,6 +358,32 @@ def test_all_literal_values():
     assert sorted(all_literal_values(L312)) == sorted(('1', '2', '3'))
 
 
+def test_path_type(tmp_path):
+    assert path_type(tmp_path) == 'directory'
+    file = tmp_path / 'foobar.txt'
+    file.write_text('hello')
+    assert path_type(file) == 'file'
+
+
+def test_path_type_unknown(tmp_path):
+    p = type(
+        'FakePath',
+        (),
+        {
+            'exists': lambda: True,
+            'is_dir': lambda: False,
+            'is_file': lambda: False,
+            'is_mount': lambda: False,
+            'is_symlink': lambda: False,
+            'is_block_device': lambda: False,
+            'is_char_device': lambda: False,
+            'is_fifo': lambda: False,
+            'is_socket': lambda: False,
+        },
+    )
+    assert path_type(p) == 'unknown'
+
+
 @pytest.mark.parametrize(
     'obj',
     (1, 1.0, '1', b'1', int, None, test_all_literal_values, len, test_all_literal_values.__code__, lambda: ..., ...),