(#21023) Flush caches on preferred_run_mode change

`Setting.preferred_run_mode` affects a lot of settings values -- such as the
modulepath that gets used for environments. Prior to f4e229e the `run_mode` was
an actual setting rather than an attribute of the settings class. This meant
that changing the value of the `run_mode` would trigger a cache flush on
Settings objects so that things like `modulepath` would be updated.

This patch:

  - Moves cache flushing out of the `Settings.set_value` method and into a new
    method `flush_cache`.

  - Adds tests to ensure that `flush_cache` is called by both `set_value` and
    `preferred_run_mode=`.
Merge pull request #1707 from Sharpie/21023-fix-preferred-run-mode

(#21023) Flush caches on preferred_run_mode change