Repository: napari/napari

Issue:
SettingsManager overwriting env var settings
## 🐛 Bug
Pydantic BaseSettings provides a nice way to set settings using [environment variables](https://pydantic-docs.helpmanual.io/usage/settings/#environment-variable-names), but at the moment, the `SettingsManager._load` method is overwriting this:

```python
# test.py
from napari.utils.settings import SETTINGS, CORE_SETTINGS

print("Application Settings:", CORE_SETTINGS[0]().theme)
print("SettingsManager:", SETTINGS.application.theme)
```

```sh
$ NAPARI_THEME=light python test.py
Application Settings: light
SettingsManager: dark
```

In the case where a settings is defined multiple ways, I think we should aim to match the [default field priority](https://pydantic-docs.helpmanual.io/usage/settings/#field-value-priority) in pydantic:

- Arguments passed to the `Settings` class initialiser.
- Environment variables, e.g. `napari_setting`
- Variables loaded from a dotenv (.env) file, (or, in our case, yaml settings) 
- Variables loaded from the secrets directory.
- The default field values for the Settings model.

This does open a bit of a challenge for the concept of persistence, since I think that loading with `NAPARI_THEME=light` would probably then override the yaml file itself on close... which is less desirable (I think it would be nice to be able to use environment variables to override settings at startup, without permanently changing the preferences)


Discussion:
adding this here in case it's useful.  Pydantic offers a [hook](https://pydantic-docs.helpmanual.io/usage/settings/#customise-settings-sources) in `BaseSettings.config.customise_sources` to know where the settings are coming from... so, for instance:

```python
#napari.utils.settings._defaults

class ApplicationSettings(BaseSettings, EventedModel):
    class Config:
        @classmethod
        def customise_sources(
            cls, init_settings, env_settings, file_secret_settings
        ):
            cls._env_settings = env_settings  # or otherwise save it somehow
            return init_settings, env_settings, file_secret_settings
```

then you can query it as follows, for instance, if I start IPython with `NAPARI_THEME=light ipython`:

```python
In [1]: from napari.utils.settings._defaults import ApplicationSettings

In [2]: s = ApplicationSettings()

In [3]: ApplicationSettings.__config__._env_settings(s)
Out[3]: {'theme': 'light'}
```

It might also make sense to move the logic from the global `SettingsManager._load` into a custom `customise_sources` method... since pydantic already provides a fair amount of support for the merging of settings from disparate sources, maybe we should just hook into that?
(should have included this link)... Pydantic actually gives an example of add json file settings source [in their documentation](https://pydantic-docs.helpmanual.io/usage/settings/#adding-sources).  that looks like a great place to import our yaml
>Pydantic actually gives an example of add json file settings source in their documentation. that looks like a great place to import our yaml

I have been looking into this, and this is not as nice as it should. Currently the load method is doing validation of things and making sure Napari starts even there are problems with the file. By using `customise_sources` we have to assume yaml file is correct. This might not be the case and will result in Napari crashing if errors on the yaml file, like invalid sections, invalid keys or invalid values.

We cannot really use the models now to validate values, since it will enter a recursion, so I am not sure how to go around this.

Having Napari start no matter what the settings file contains seems like a much important thing than having the ability to use these env variables. I can try to make it work in a different way, but using `customise_sources` will not help to make the configuration system robust.

So I can use this:

```python
#napari.utils.settings._defaults

class ApplicationSettings(BaseSettings, EventedModel):
    class Config:
        @classmethod
        def customise_sources(
            cls, init_settings, env_settings, file_secret_settings
        ):
            cls._env_settings = env_settings  # or otherwise save it somehow
            return init_settings, env_settings, file_secret_settings
```

But keep the loading similar to what we have now. Just taking into account these values
> Having Napari start no matter what the settings file contains seems like a much important thing than having the ability to use these env variables

absolutely!  sounds good.  
> By using customise_sources we have to assume yaml file is correct. 

actually, I have a question... why is that the case?  From [docs example](https://pydantic-docs.helpmanual.io/usage/settings/#adding-sources)... why can't you just put the same validation logic from SettingsManager._load in the loading function ("`json_config_settings_source`" in their example)?
Cause to be able to validate I was using the model and catching errors. But doing that inside the custom sources that will also use custom sources when using the model so it will get into a recursion. I tried setting each thing manually with setattr on the model but that dis not work either. Will try again tomorrow morning with fresh eyes
Bump for next sprint, we might want to work on this cc @goanpeca @ppwadhwa 
Yes. I could not get a solution that made me happy 🤣.

Need to revisit

Resolving patch:
diff --git a/napari/utils/settings/_defaults.py b/napari/utils/settings/_defaults.py
--- a/napari/utils/settings/_defaults.py
+++ b/napari/utils/settings/_defaults.py
@@ -5,9 +5,9 @@
 import os
 from enum import Enum
 from pathlib import Path
-from typing import Dict, List, Set, Tuple
+from typing import Any, Dict, List, Set, Tuple
 
-from pydantic import BaseSettings, Field
+from pydantic import BaseSettings, Field, ValidationError
 from typing_extensions import TypedDict
 
 from .._base import _DEFAULT_LOCALE
@@ -175,6 +175,48 @@ class QtBindingChoice(str, Enum):
     pyqt5 = 'pyqt5'
 
 
+def yaml_config_settings_source(settings: BaseSettings) -> Dict[str, Any]:
+    """Load and validate settings coming from configuration file."""
+    yaml_settings = {}
+    model_class = settings.__class__
+
+    # This is set by the SettingsManager
+    loaded_data = getattr(settings, "_LOADED_DATA", {})
+
+    validate = getattr(settings, "_IGNORE_YAML_SOURCE", False)
+    if not validate and loaded_data:
+        # This variable prevents recursion when using the model for validation
+        model_class._IGNORE_YAML_SOURCE = True
+
+        # Get defaults from the schema
+        model_schema = model_class.schema()
+        section = model_schema["section"]
+        default_properties = model_schema.get("properties", {})
+        yaml_settings = loaded_data.get(section, {}).copy()
+
+        try:
+            model_class(**yaml_settings)
+        except ValidationError as e:
+            # Handle extra fields
+            model_data_replace = {}
+            for error in e.errors():
+                # Grab the first error entry
+                item = error["loc"][0]
+                try:
+                    model_data_replace[item] = default_properties[item][
+                        "default"
+                    ]
+                except KeyError:
+                    yaml_settings.pop(item)
+
+            yaml_settings.update(model_data_replace)
+
+        # This variable restores the normal sources loading behavior
+        model_class._IGNORE_YAML_SOURCE = False
+
+    return yaml_settings
+
+
 class BaseNapariSettings(BaseSettings, EventedModel):
     class Config:
         # Pydantic specific configuration
@@ -182,10 +224,19 @@ class Config:
         use_enum_values = True
         validate_all = True
 
+        @classmethod
+        def customise_sources(
+            cls, init_settings, env_settings, file_secret_settings
+        ):
+            return (
+                init_settings,
+                env_settings,
+                yaml_config_settings_source,
+                file_secret_settings,
+            )
 
-class AppearanceSettings(BaseNapariSettings):
-    """Appearance Settings."""
 
+class AppearanceSettings(BaseNapariSettings):
     # 1. If you want to *change* the default value of a current option, you need to
     #    do a MINOR update in config version, e.g. from 3.0.0 to 3.1.0
     # 2. If you want to *remove* options that are no longer needed in the codebase,
@@ -210,7 +261,11 @@ class AppearanceSettings(BaseNapariSettings):
 
     class Config:
         # Pydantic specific configuration
-        title = trans._("Appearance")
+        schema_extra = {
+            "title": trans._("Appearance"),
+            "description": trans._("Appearance settings."),
+            "section": "appearance",
+        }
 
     class NapariConfig:
         # Napari specific configuration
@@ -218,8 +273,6 @@ class NapariConfig:
 
 
 class ApplicationSettings(BaseNapariSettings):
-    """Main application settings."""
-
     # 1. If you want to *change* the default value of a current option, you need to
     #    do a MINOR update in config version, e.g. from 3.0.0 to 3.1.0
     # 2. If you want to *remove* options that are no longer needed in the codebase,
@@ -274,7 +327,11 @@ class ApplicationSettings(BaseNapariSettings):
 
     class Config:
         # Pydantic specific configuration
-        title = trans._("Application")
+        schema_extra = {
+            "title": trans._("Application"),
+            "description": trans._("Main application settings."),
+            "section": "application",
+        }
 
     class NapariConfig:
         # Napari specific configuration
@@ -307,8 +364,6 @@ class PluginHookOption(TypedDict):
 
 
 class PluginsSettings(BaseNapariSettings):
-    """Plugins Settings."""
-
     schema_version: SchemaVersion = (0, 1, 1)
     call_order: CallOrderDict = Field(
         None,
@@ -322,7 +377,11 @@ class PluginsSettings(BaseNapariSettings):
 
     class Config:
         # Pydantic specific configuration
-        title = trans._("Plugins")
+        schema_extra = {
+            "title": trans._("Plugins"),
+            "description": trans._("Plugins settings."),
+            "section": "plugins",
+        }
 
     class NapariConfig:
         # Napari specific configuration
diff --git a/napari/utils/settings/_manager.py b/napari/utils/settings/_manager.py
--- a/napari/utils/settings/_manager.py
+++ b/napari/utils/settings/_manager.py
@@ -3,10 +3,10 @@
 
 import json
 import os
+import warnings
 from pathlib import Path
 
 from appdirs import user_config_dir
-from pydantic import ValidationError
 from yaml import safe_dump, safe_load
 
 from ...utils.translations import trans
@@ -15,6 +15,7 @@
     CORE_SETTINGS,
     AppearanceSettings,
     ApplicationSettings,
+    BaseNapariSettings,
     PluginsSettings,
 )
 
@@ -68,7 +69,6 @@ def __init__(self, config_path: str = None, save_to_disk: bool = True):
         self._save_to_disk = save_to_disk
         self._settings = {}
         self._defaults = {}
-        self._models = {}
         self._plugins = []
 
         if not self._config_path.is_dir():
@@ -87,13 +87,6 @@ def __dir__(self):
     def __str__(self):
         return safe_dump(self._to_dict(safe=True))
 
-    @staticmethod
-    def _get_section_name(setting) -> str:
-        """
-        Return the normalized name of a section based on its config title.
-        """
-        return setting.__name__.replace("Settings", "").lower()
-
     def _to_dict(self, safe: bool = False) -> dict:
         """Convert the settings to a dictionary."""
         data = {}
@@ -117,53 +110,43 @@ def _save(self):
     def _load(self):
         """Read configuration from disk."""
         path = self.path / self._FILENAME
-        for setting in CORE_SETTINGS:
-            section = self._get_section_name(setting)
-            self._defaults[section] = setting()
-            self._settings[section] = setting()
-            self._models[section] = setting
-            self._settings[section] = setting()
 
         if path.is_file():
             try:
                 with open(path) as fh:
                     data = safe_load(fh.read()) or {}
             except Exception as err:
-                import warnings
-
                 warnings.warn(
                     trans._(
                         "The content of the napari settings file could not be read\n\nThe default settings will be used and the content of the file will be replaced the next time settings are changed.\n\nError:\n{err}",
+                        deferred=True,
                         err=err,
                     )
                 )
                 data = {}
 
-            # Check with models
-            for section, model_data in data.items():
-                try:
-                    model = self._models[section](**model_data)
-                    model.events.connect(lambda x: self._save())
-                    self._settings[section] = model
-                except KeyError:
-                    pass
-                except ValidationError as e:
-                    # Handle extra fields
-                    model_data_replace = {}
-                    for error in e.errors():
-                        # Grab the first error entry
-                        item = error["loc"][0]
-                        try:
-                            model_data_replace[item] = getattr(
-                                self._defaults[section], item
-                            )
-                        except AttributeError:
-                            model_data.pop(item)
-
-                    model_data.update(model_data_replace)
-                    model = self._models[section](**model_data)
-                    model.events.connect(lambda x: self._save())
-                    self._settings[section] = model
+            # Load data once and save it in the base class
+            BaseNapariSettings._LOADED_DATA = data
+
+        for setting in CORE_SETTINGS:
+            section = setting.schema().get("section", None)
+            if section is None:
+                raise ValueError(
+                    trans._(
+                        "Settings model {setting!r} must provide a `section` in the `schemas_extra`",
+                        deferred=True,
+                        setting=setting,
+                    )
+                )
+
+            _section_defaults = {}
+            for option, option_data in setting.schema()["properties"].items():
+                _section_defaults[option] = option_data.get("default", None)
+
+            self._defaults[section] = setting(**_section_defaults)
+            model = setting()
+            model.events.connect(lambda x: self._save())
+            self._settings[section] = model
 
         self._save()
 


Test patch:
diff --git a/napari/utils/settings/_tests/test_settings.py b/napari/utils/settings/_tests/test_settings.py
--- a/napari/utils/settings/_tests/test_settings.py
+++ b/napari/utils/settings/_tests/test_settings.py
@@ -15,24 +15,16 @@ def settings(tmp_path):
 
 def test_settings_file(tmp_path):
     SettingsManager(tmp_path, save_to_disk=True)
-    fpath = tmp_path / "settings.yaml"
+    fpath = tmp_path / SettingsManager._FILENAME
     assert fpath.exists()
 
 
 def test_settings_file_not_created(tmp_path):
     SettingsManager(tmp_path, save_to_disk=False)
-    fpath = tmp_path / "settings.yaml"
+    fpath = tmp_path / SettingsManager._FILENAME
     assert not fpath.exists()
 
 
-def test_settings_get_section_name():
-    class SomeSectionSettings:
-        pass
-
-    section = SettingsManager._get_section_name(SomeSectionSettings)
-    assert section == "somesection"
-
-
 def test_settings_loads(tmp_path):
     data = """
 appearance:
@@ -162,3 +154,26 @@ def test_model_fields_are_annotated():
 
     if errors:
         raise ValueError("\n\n".join(errors))
+
+
+def test_settings_env_variables(tmp_path, monkeypatch):
+    value = 'light'
+    monkeypatch.setenv('NAPARI_THEME', value)
+    settings = SettingsManager(tmp_path, save_to_disk=True)
+    assert CORE_SETTINGS[0]().theme == value
+    assert settings.appearance.theme == value
+
+
+def test_settings_env_variables_fails(tmp_path, monkeypatch):
+    value = 'FOOBAR'
+    monkeypatch.setenv('NAPARI_THEME', value)
+    with pytest.raises(pydantic.error_wrappers.ValidationError):
+        SettingsManager(tmp_path, save_to_disk=True)
+
+
+def test_core_settings_are_class_variables_in_settings_manager():
+    for setting in CORE_SETTINGS:
+        schema = setting.schema()
+        section = schema["section"]
+        assert section in SettingsManager.__annotations__
+        assert setting == SettingsManager.__annotations__[section]