Repository: sopel-irc/sopel

Issue:
config: Warn if unexpected settings exist
When loading the config file, Sopel should warn the user if it finds a setting name that isn't defined in the `StaticSection` for that section. (Obviously, this logic will have to ignore the per-channel config sections added in #1235—or handle those, too, since they also have clearly defined properties.)

Such a warning _should not_ prevent starting the bot or halt/block loading the module to whom the setting belongs; it should be purely informational.

This would very much help users avoid typos after manually editing their config files, and aid in debugging issues like the one @HumorBaby and I just spent literally _hours_ on—which turned out to be the result of a misnamed setting in the `.cfg` file Sopel was using. (The fact that the incorrect name made more sense than the correct one is a bug for another repo.)


Discussion:
>(Obviously, this logic will have to ignore the per-channel config sections added in #1235—or handle those, too, since they also have clearly defined properties.)

It could just validate `StaticSection`s only. Since the per-channel config sections are not of that type, it wouldn't be thrown off by those :smile: Just adding my thoughts... not that I had anything else to add about the actual implementation itself :stuck_out_tongue: I was just reminded of this because the `admin` module explicitly checks for whether a section is `StaticSection`, and it took me forever to realize why such a check was even needed... for cases like the per-channel config (i.e., non-module config)!

Huh, actually that `StaticSection` type check came from a813671, which fixed an old bug I reported (and introduced a new bug that I was surprised Python itself wouldn't complain about at runtime… see if you can spot it in the diff before looking at the fix in 111fc5c7).

It has nothing to do with the per-channel stuff, and everything to do with the fact that, pre-6.0, the config sections were lawless lands that could contain whatever you might toss in. Sopel 6 brought validation of config attributes and the `StaticSection` definitions we know and love today. My guess is the error handling you found is for legacy support of some weird use cases that rely on having arbitrary config values not defined in a section prototype, I dunno.

All of that is by way of saying, cool, whoever implements this can just do the same kind of check. 😁
> (and introduced a new bug that I was surprised Python itself wouldn't complain about at runtime… see if you can spot it in the diff before looking at the fix in 111fc5c).

...introduced this control-flow one too that has been hanging around this whole time! :sweat_smile: 
https://github.com/sopel-irc/sopel/blob/a8136716d71b4db9f2442e3b4d2af6522df67e1b/willie/modules/admin.py#L198-L200

> It has nothing to do with the per-channel stuff, and everything to do with the fact that, pre-6.0, the config sections were lawless lands that could contain whatever you might toss in ... My guess is the error handling you found is for legacy support of some weird use cases that rely on having arbitrary config values not defined in a section prototype

... figured it was something like that, since of course per-channel config didn't exist :grin: I got about 5-6 years back in the revision history before I gave up trying to put it all together. Regardless, I'm glad it stuck around and now can directly apply!
That's a nice control flow bug. Parentheses are useful, hmm?

It's close to the password-censoring logic, too, which (after our adventure yesterday) should probably also censor any option ending in "secret". Off topic, but hey—I know you like making little PRs. 😝
>Off topic, but hey—I know you like making little PRs

It's a serious problem :grimacing: 

> That's a nice control flow bug. Parentheses are useful, hmm?

@Exirel pointed it out in review for #1556 :smile: so the fix is in there. Let me know if it should go in a smaller PR for merge sooner...

> should probably also censor any option ending in "secret".

Okay. But what about settings that don't `endswith` "secret" (e.g., "secret_codeword") or "password"... `is in` may be better no? Better safe than sorry... 

The other option is to add a boolean attribute to `config.types.BaseValidated` called `is_secret` or something and only display values to the owner (or not at all)...? Which would obviously be a much larger change, wrt the API, and may not be worth it.
>The other option is to add a boolean attribute to config.types.BaseValidated called is_secret or something and only display values to the owner (or not at all)...? Which would obviously be a much larger change, wrt the API, and may not be worth it.

probably overkill just for `admin.py`...
Pretty definitely overkill. We should take further discussion of that to a new issue, though, if it needs bikeshedding (and it seems to).
This feature is a fine candidate for a minor release, and doesn't need to block 7.0.
@Exirel This would definitely help users out a lot when configuring plugins, e.g. if they mistype a setting name. Yes, we can encourage the use of `sopel-plugins configure`, but supporting that isn't required for plugins. Seems not too invasive.
So, I've something in progress for that, but I need #1933 to be merged to prevent possible merge conflict with the test directory.

Resolving patch:
diff --git a/sopel/bot.py b/sopel/bot.py
--- a/sopel/bot.py
+++ b/sopel/bot.py
@@ -254,7 +254,7 @@ def setup(self):
         """
         self.setup_logging()
         self.setup_plugins()
-        self._scheduler.start()
+        self.post_setup()
 
     def setup_logging(self):
         """Set up logging based on config options."""
@@ -322,6 +322,34 @@ def setup_plugins(self):
         else:
             LOGGER.warning("Warning: Couldn't load any plugins")
 
+    # post setup
+
+    def post_setup(self):
+        """Perform post-setup actions.
+
+        This method handles everything that should happen after all the plugins
+        are loaded, and before the bot can connect to the IRC server.
+
+        At the moment, this method checks for undefined configuration options,
+        and starts the job scheduler.
+
+        .. versionadded:: 7.1
+        """
+        settings = self.settings
+        for section_name, section in settings.get_defined_sections():
+            for option_name in settings.parser.options(section_name):
+                if not hasattr(section, option_name):
+                    LOGGER.warning(
+                        'Config option `%s.%s` is not defined by its section '
+                        'and may not be recognized by Sopel.',
+                        section_name,
+                        option_name,
+                    )
+
+        self._scheduler.start()
+
+    # plugins management
+
     def reload_plugin(self, name):
         """Reload a plugin.
 
@@ -448,6 +476,8 @@ def get_plugin_meta(self, name):
 
         return self._plugins[name].get_meta_description()
 
+    # callable management
+
     @deprecated(
         reason="Replaced by specific `unregister_*` methods.",
         version='7.1',
@@ -605,6 +635,8 @@ def msg(self, recipient, text, max_messages=1):
         """
         self.say(text, recipient, max_messages)
 
+    # message dispatch
+
     def call_rule(self, rule, sopel, trigger):
         # rate limiting
         if not trigger.admin and not rule.is_unblockable():
@@ -852,6 +884,8 @@ def _update_running_triggers(self, running_triggers):
             self._running_triggers = [
                 t for t in running_triggers if t.is_alive()]
 
+    # event handlers
+
     def on_scheduler_error(self, scheduler, exc):
         """Called when the Job Scheduler fails.
 
@@ -965,6 +999,8 @@ def _shutdown(self):
         # Avoid calling shutdown methods if we already have.
         self.shutdown_methods = []
 
+    # URL callbacks management
+
     def register_url_callback(self, pattern, callback):
         """Register a ``callback`` for URLs matching the regex ``pattern``.
 
diff --git a/sopel/config/__init__.py b/sopel/config/__init__.py
--- a/sopel/config/__init__.py
+++ b/sopel/config/__init__.py
@@ -159,6 +159,30 @@ def homedir(self):
         else:
             return os.path.dirname(self.filename)
 
+    def get_defined_sections(self):
+        """Retrieve all defined static sections of this configuration.
+
+        :return: all instances of :class:`~sopel.config.types.StaticSection`
+                 defined for this configuration file
+        :rtype: list
+
+        When a plugin defines a section (using :meth:`define_section`), it
+        associates a :class:`~sopel.config.types.StaticSection` for the section.
+        This method allows to retrieve these instances of ``StaticSection``,
+        and only these.
+
+        .. versionadded:: 7.1
+        """
+        sections = (
+            (name, getattr(self, name))
+            for name in self.parser.sections()
+        )
+        return [
+            (name, section)
+            for name, section in sections
+            if isinstance(section, types.StaticSection)
+        ]
+
     def save(self):
         """Write all changes to the config file.
 
diff --git a/sopel/config/types.py b/sopel/config/types.py
--- a/sopel/config/types.py
+++ b/sopel/config/types.py
@@ -66,20 +66,23 @@ def __init__(self, config, section_name, validate=True):
         self._parent = config
         self._parser = config.parser
         self._section_name = section_name
+
         for value in dir(self):
+            if value in ('_parent', '_parser', '_section_name'):
+                # ignore internal attributes
+                continue
+
             try:
                 getattr(self, value)
             except ValueError as e:
                 raise ValueError(
-                    'Invalid value for {}.{}: {}'.format(section_name, value,
-                                                         str(e))
-                )
+                    'Invalid value for {}.{}: {}'.format(
+                        section_name, value, str(e)))
             except AttributeError:
                 if validate:
                     raise ValueError(
-                        'Missing required value for {}.{}'.format(section_name,
-                                                                  value)
-                    )
+                        'Missing required value for {}.{}'.format(
+                            section_name, value))
 
     def configure_setting(self, name, prompt, default=NO_DEFAULT):
         """Return a validated value for this attribute from the terminal.


Test patch:
diff --git a/test/test_config.py b/test/test_config.py
--- a/test/test_config.py
+++ b/test/test_config.py
@@ -36,6 +36,9 @@
     "#startquote
     &endquote"
     "&quoted"
+
+[somesection]
+is_defined = no
 """  # noqa (trailing whitespaces are intended)
 
 TEST_CHANNELS = [
@@ -343,3 +346,21 @@ def test_save_modified_config(multi_fakeconfig):
         '&endquote"',
         '"&quoted"',  # doesn't start with a # so it isn't escaped
     ]
+
+
+def test_get_defined_sections(multi_fakeconfig):
+    assert multi_fakeconfig.parser.has_section('core')
+    assert multi_fakeconfig.parser.has_section('fake')
+    assert multi_fakeconfig.parser.has_section('spam')
+    assert multi_fakeconfig.parser.has_section('somesection')
+
+    results = multi_fakeconfig.get_defined_sections()
+
+    assert len(results) == 3, 'There should be 3 defined sections'
+
+    items = dict(results)
+    assert 'core' in items, 'core must be always defined'
+    assert 'fake' in items
+    assert 'spam' in items
+    assert 'somesection' not in items, (
+        'somesection was not defined and should not appear as such')