Repository: samuelcolvin/pydantic (2021-12-29 ~ 2022-08-04)

problem_statement: V1.9 Name Collisions JSON model_as_dict
### Checks

* [ ] I added a descriptive title to this issue
* [ ] I have searched (google, github) for similar issues and couldn't find anything
* [ ] I have read and followed [the docs](https://pydantic-docs.helpmanual.io/) and still think this is a bug

<!-- Sorry to sound so draconian, but every second saved replying to issues is time spend improving pydantic :-) -->

# Bug

In 1.9 Code changed in this merge #2650 The "models_as_dict" option for json has collision issues when the class names are the same but in multiple files. This is because of the change to __name__ in custom_pydantic_encoder. In the following code this will result in an error as DBUser.User does not have the field Address. If we force the encoder to be defined after the class this issue does not happen. The other option would be modification of update_forward_refs but that seems a lot worse. 

Output of `python -c "import pydantic.utils; print(pydantic.utils.version_info())"`:
```
  pydantic version: 1.9.0a2
  pydantic compiled: True
  install path: /PycharmProjects/pythonProject/venv/lib/python3.8/site-packages/pydantic
  python version: 3.8.9 (default, Oct 26 2021, 07:25:53)  [Clang 13.0.0 (clang-1300.0.29.30)]
 
 Tried it in 3.10 as well. 
 ```
<!-- or if you're using pydantic prior to v1.3, manually include: OS, python version and pydantic version -->

<!-- Please read the [docs](https://pydantic-docs.helpmanual.io/) and search through issues to
confirm your bug hasn't already been reported. -->

<!-- Where possible please include a self-contained code snippet describing your bug: -->

```py
##DBusers.py
from pydantic import BaseModel
from uuid import UUID

class User(BaseModel):
    name: str
    user_id: UUID


```


```py
##this one throws Exception
from typing import List, Optional

from pydantic import BaseModel
from DBUser import User as DBUser
from uuid import uuid4

class Address(BaseModel):
    city: str
    country: str


class User(BaseModel):
    name: str
    address: Address
    friends: Optional[List['User']] = None
    connections: Optional[List[DBUser]]=None

    class Config:
        json_encoders = {
            Address: lambda a: f'{a.city} ({a.country})',
            "User": lambda u: f'{u.name} in {u.address.city} '
                              f'({u.address.country[:2].upper()})',
        }


User.update_forward_refs()

wolfgang = User(
    name='Wolfgang',
    address=Address(city='Berlin', country='Deutschland'),
    friends=[
        User(name='Pierre', address=Address(city='Paris', country='France')),
        User(name='John', address=Address(city='London', country='UK')),
    ],
    connections=[
        DBUser(name="joe",user_id=uuid4()),
        DBUser(name="Jeff", user_id=uuid4()),
    ]
)
wolfgang.json(indent=2,models_as_dict=False) ##Raises AttributeError: 'User' object has no attribute 'address'


```

```py
##this one does not throw Exception

from typing import List, Optional
from pydantic import BaseModel
from DBUser import User as DBUser
from uuid import uuid4

class Address(BaseModel):
    city: str
    country: str


class User(BaseModel):
    name: str
    address: Address
    friends: Optional[List["User"]] = None
    connections: Optional[List[DBUser]]=None

    class Config:
        json_encoders = {
            Address: lambda a: f'{a.city} ({a.country})',
        }


User.update_forward_refs()

User.Config.json_encoders[User] =lambda u: f'{u.name} in {u.address.city} ' f'({u.address.country[:2].upper()})'


wolfgang = User(
    name='Wolfgang',
    address=Address(city='Berlin', country='Deutschland'),
    friends=[
        User(name='Pierre', address=Address(city='Paris', country='France')),
        User(name='John', address=Address(city='London', country='UK')),
    ],
    connections=[
        DBUser(name="joe",user_id=uuid4()),
        DBUser(name="Jeff", user_id=uuid4()),
    ]
)
print(wolfgang.json(indent=2,models_as_dict=False)) ##Does not explode


```

date: 2021-12-29
patch: diff --git a/pydantic/typing.py b/pydantic/typing.py
--- a/pydantic/typing.py
+++ b/pydantic/typing.py
@@ -523,3 +523,4 @@
             continue
 
+        json_encoders[new_key] = json_encoders.pop(key)
 


----

problem_statement: Self-referenced ClassVar causes TypeError: issubclass() arg 1 must be a class (in 1.9.0, worked in 1.8.2)
### Checks

* [X] I added a descriptive title to this issue
* [X] I have searched (google, github) for similar issues and couldn't find anything -> Issue #545 is similar, but this case is a regression in 1.9.
* [X] I have read and followed [the docs](https://pydantic-docs.helpmanual.io/) and still think this is a bug

<!-- Sorry to sound so draconian, but every second saved replying to issues is time spend improving pydantic :-) -->

# Bug

Output of `python -c "import pydantic.utils; print(pydantic.utils.version_info())"`:
```
pydantic version: 1.9.0
pydantic compiled: True
install path: C:\Program Files\Python39\Lib\site-packages\pydantic
python version: 3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]
platform: Windows-10-10.0.19041-SP0
optional deps. installed: ['email-validator', 'typing-extensions']
```
Also occurs on Linux with Python 3.9.9.


```py
from __future__ import annotations
import pydantic
import typing

class WithClassVar(pydantic.BaseModel):
    Instances : typing.ClassVar[dict[str,WithClassVar]] = {} # fails with issubclass() arg 1 must be a class; arg  1 is typing.ClassVar object 
    # Instances : typing.ClassVar[dict[str,'WithClassVar']] = {} # works
    # instance : dict[str,WithClassVar] = {} # no classvar - works

...
```
I believe this case is specific to a ClassVar referencing itself. The argument passed to `issubclass `is not a class, but an instance of `typing.ClassVar`. 

Call stack:

```
__subclasscheck__ (c:\Program Files\Python39\Lib\typing.py:789)
pydantic.fields.ModelField._type_analysis (c:\Program Files\Python39\Lib\site-packages\pydantic\fields.py:638)
pydantic.fields.ModelField.prepare (c:\Program Files\Python39\Lib\site-packages\pydantic\fields.py:534)
pydantic.typing.update_field_forward_refs (c:\Program Files\Python39\Lib\site-packages\pydantic\typing.py:451)
pydantic.typing.update_model_forward_refs (c:\Program Files\Python39\Lib\site-packages\pydantic\typing.py:480)
pydantic.main.BaseModel.__try_update_forward_refs__ (c:\Program Files\Python39\Lib\site-packages\pydantic\main.py:773)
pydantic.main.ModelMetaclass.__new__ (c:\Program Files\Python39\Lib\site-packages\pydantic\main.py:292)
<module> (.... \pydantic_bug.py:5)
```



date: 2022-05-11
patch: diff --git a/pydantic/typing.py b/pydantic/typing.py
--- a/pydantic/typing.py
+++ b/pydantic/typing.py
@@ -465,3 +465,4 @@
     # this is an ugly workaround for class vars that contain forward references and are therefore themselves
     # forward references, see #3679
+    if ann_type.__class__ == ForwardRef and ann_type.__forward_arg__.startswith('ClassVar['):
         return True


----

problem_statement: Pydantic fails to read content of files in secrets_dir on Linux
### Checks

* [x] I added a descriptive title to this issue
* [x] I have searched (google, github) for similar issues and couldn't find anything
* [x] I have read and followed [the docs](https://pydantic-docs.helpmanual.io/) and still think this is a bug

<!-- Sorry to sound so draconian, but every second saved replying to issues is time spend improving pydantic :-) -->

# Bug

To reproduce, on Linux OS: 
- build a config manager that subclasses `BaseSettings`
- add `secrets_dir` config pointing to dir with text file(s), as per docs
- add class attributes to use the files in `secrets_dir`, as per docs
- run the code and inspect/print the attributes
- observe that vars coming from .env file work fine, but those from `secrets_dir` are None, or error if not Optional, and there is no user warning for the `secrets_dir` itself

Repeat the same on MacOS - it should work OK, the content is read correctly.

I have added a repo with Dockerfile replicating the setup: https://github.com/dimitri-b/pydantic-secrets-dir-issue

Output of `python -c "import pydantic.utils; print(pydantic.utils.version_info())"`:
```
             pydantic version: 1.8.2
            pydantic compiled: True
                 install path: /usr/local/lib/python3.8/site-packages/pydantic
               python version: 3.8.12 (default, Sep  3 2021, 02:35:42)  [GCC 8.3.0]
                     platform: Linux-5.10.47-linuxkit-x86_64-with-glibc2.2.5
     optional deps. installed: ['dotenv', 'typing-extensions']
```
<!-- or if you're using pydantic prior to v1.3, manually include: OS, python version and pydantic version -->

<!-- Please read the [docs](https://pydantic-docs.helpmanual.io/) and search through issues to
confirm your bug hasn't already been reported. -->

<!-- Where possible please include a self-contained code snippet describing your bug: -->


```py
import pydantic

class SettingsLocal(pydantic.BaseSettings):
    NORMAL_VAR: Optional[str]
    SECRET_VAR: Optional[str]

    class Config:
        env_file = "./.env"
        secrets_dir = "./secrets-in-project"
...
```

date: 2022-08-04
patch: diff --git a/pydantic/env_settings.py b/pydantic/env_settings.py
index 35b96970f..73945e31e 100644
--- a/pydantic/env_settings.py
+++ b/pydantic/env_settings.py
@@ -269,7 +269,11 @@ class SecretsSettingsSource:
 
         for field in settings.__fields__.values():
             for env_name in field.field_info.extra['env_names']:
-                path = secrets_path / env_name
+                path = find_case_path(secrets_path, env_name, settings.__config__.case_sensitive)
+                if not path:
+                    # path does not exist, we curently don't return a warning for this
+                    continue
+
                 if path.is_file():
                     secret_value = path.read_text().strip()
                     if field.is_complex():
@@ -279,12 +283,11 @@ class SecretsSettingsSource:
                             raise SettingsError(f'error parsing JSON for "{env_name}"') from e
 
                     secrets[field.alias] = secret_value
-                elif path.exists():
+                else:
                     warnings.warn(
                         f'attempted to load secret file "{path}" but found a {path_type(path)} instead.',
                         stacklevel=4,
                     )
-
         return secrets
 
     def __repr__(self) -> str:
@@ -304,3 +307,15 @@ def read_env_file(
         return {k.lower(): v for k, v in file_vars.items()}
     else:
         return file_vars
+
+
+def find_case_path(dir_path: Path, file_name: str, case_sensitive: bool) -> Optional[Path]:
+    """
+    Find a file within path's directory matching filename, optionally ignoring case.
+    """
+    for f in dir_path.iterdir():
+        if f.name == file_name:
+            return f
+        elif not case_sensitive and f.name.lower() == file_name.lower():
+            return f
+    return None

test_patch: diff --git a/tests/test_settings.py b/tests/test_settings.py
index c11cdbb73..1e85a546e 100644
--- a/tests/test_settings.py
+++ b/tests/test_settings.py
@@ -890,6 +890,33 @@ def test_secrets_path(tmp_path):
     assert Settings().dict() == {'foo': 'foo_secret_value_str'}
 
 
+def test_secrets_case_sensitive(tmp_path):
+    (tmp_path / 'SECRET_VAR').write_text('foo_env_value_str')
+
+    class Settings(BaseSettings):
+        secret_var: Optional[str]
+
+        class Config:
+            secrets_dir = tmp_path
+            case_sensitive = True
+
+    assert Settings().dict() == {'secret_var': None}
+
+
+def test_secrets_case_insensitive(tmp_path):
+    (tmp_path / 'SECRET_VAR').write_text('foo_env_value_str')
+
+    class Settings(BaseSettings):
+        secret_var: Optional[str]
+
+        class Config:
+            secrets_dir = tmp_path
+            case_sensitive = False
+
+    settings = Settings().dict()
+    assert settings == {'secret_var': 'foo_env_value_str'}
+
+
 def test_secrets_path_url(tmp_path):
     (tmp_path / 'foo').write_text('http://www.example.com')
     (tmp_path / 'bar').write_text('snap')
@@ -940,6 +967,7 @@ def test_secrets_missing(tmp_path):
 
     with pytest.raises(ValidationError) as exc_info:
         Settings()
+
     assert exc_info.value.errors() == [{'loc': ('foo',), 'msg': 'field required', 'type': 'value_error.missing'}]