Package: settings-collector
Settings CollectorThis Python 3.6+ package is meant to be used by other packages (i.e., not
directly by final projects). Its purpose is to collect settings defined in
various frameworks, so you don't have to make special implementations for
those.A typical use case would be to define something like this:fromsettings_collectorimportSettingsCollector,SC_Settingclassmy_settings(SettingsCollector):foo=SC_Setting("bar")and then usemy_settings.fooin your code. The value offoowould come from
Django'ssettingsif your package is used in a Django app, from Flask's app
config if it's used in a Flask project, etc.ContentSupported frameworksHow to useSettings definitionsPrefixName casesScopesFine tuningLocal function argumentsSettings in projects with no frameworksCustom loadersTesting custom loadersSupported frameworksCurrently, Settings Collector supportsBottle,CherryPy,Django,Flask,Pyramid, andTurboGears.
Please note that the support for most of those was written just by reading
their docs. I am experienced only with Django and Flask.Each framework is handled by its loader class. These are located in thesettings_collector.loaderpackage. Apart from the loaders for specific
frameworks, there is also a loader for environment variables, calledSC_EnvironLoader.Settings for frameworks are not normally loaded directly from the environment,
but they come from some sort of config file (which might import some of them
from the environment), soSC_EnvironLoaderis disabled by default. To enable
it, one can runfromsettings_collectorimportSC_EnvironLoaderSC_EnvironLoader.enabled=TrueHow to useThe most general way to use this is to define a class similar to the models in
ORM frameworks:fromsettings_collectorimportSettingsCollector,SC_Settingclassmy_settings(SettingsCollector):foo=SC_Setting("bar")After this, just usemy_settings.foo, which will fetch valuefoofrom your
project's configuration (which can be standard Django, Flask, etc). If the
value is not defined, it will be set to the default value ("bar", in this
example).Note that the loaders adjust character cases as needed. This means that yourmy_settings.foowill come fromdjango.conf.settings.FOOif your packages is
used in a Django app, but fromcherrypy.config["foo"]orcherrypy.request.app.config["foo"]if it's used in a CherryPy project. For
details, seeName cases.If one needs to define only the default values for the settings, without
further adjustments, this can be used as well:fromsettings_collectorimportSettingsCollectorclassmy_settings(SettingsCollector):defaults={"bar":"foo"}This example is equivalent to the previous one, but it doesn't allow
adjustments described in the next section.Settings definitionsThe constructor ofSC_Settingaccepts the following arguments (presented here
with their default values):default_value[optional]: The default value to be returned if it's not
defined in settings. If neither this nor the setting's value in the
framework's config is set, aValueErrorexception is raised.no_cache=False[optional, keyword only]: IfTrue, skip cashing. This
means that the actual value of the setting is fetched every time it is
requested. Normally, the settings are set up when the app runs and they do
not change, so caching is usually the best way to go.value_type=None[optional, keyword only]: A type to convert the value to
(for example,int). This can be used to ensure the correct type of the
value, even if the settings provide something else (for example, a string, as
is normal foros.environ).default_on_error=True[optional, keyword only]: This affects the behaviour
when the value of a setting is set, but the casting to the givenvalue_typefails. If set toTrue, the returned value isdefault_value(aValueErrorexception is raised if that value is not set). However, if this
is set toFalse, requesting the value not defined in the app will result inTypeErrorexception, regardless ofdefault_value.PrefixIf you want your config settings to be distinguished from all others (those
used by other packages or the framework itself), you can define a prefix:fromsettings_collectorimportSettingsCollector,SC_Settingclassmy_settings(SettingsCollector):classSC_Config:prefix="test"foo=SC_Setting("bar")Now, the value ofmy_settings.foowill come fromtest__fooin the
framework's config.Name casesEach loader (the class responsible for the actual loading of the settings'
values from the frameworks' configs) can define the case for the names. For
example, Django keeps settings in variables that have upper-case names, so the
corresponding loader will change your names to upper-case as will the one foros.environ, but not the one for TurboGears.So, in the previous example, the value ofmy_settings.foowill come fromTEST__FOOif the running framework is Django, but it'll come fromtest__fooif your package is used in a TurboGears app.ScopesSettings can get their values relative to scopes, not unlike names given to
loggers. The default separator for scopes' names is double underscore ("__").Let us take a look at the following Django example. Let's say that the settings
are defined as follows (in some framework that uses upper-case names):FOO="bar"SC1__SC2__FOO="bard"We then define our settings collector:fromsettings_collectorimportSettingsCollector,SC_Settingclassmy_settings(SettingsCollector):foo=SC_Setting("bar")The values obtained will be as follows:my_settings.foo == "bar"is defined in the settings asFOO=bar;my_settings("sc1").foo == "bar"is inherited from the root scope because
there is nofoodefined in the scope"sc1"(i.e., there is noSC1__FOO);my_settings("sc1__sc2").foo == "bard"is directly defined in the settings
asSC1__SC2__FOO = "bard";my_settings("sc1__sc2__sc3").foo == "bard"is again inherited from the
parent scope"sc1__sc2"because there is noSC1__SC2__SC3__FOOdefinition.Fine tuningEach subclass ofSettingsCollectorcan have a classSC_Configin its
definition, which has a similar role toMetain Django's models: it defines
various properties that define the settings collector's behaviour. The
available attributes are as follows:prefix[default:None]: Explainedhere.sep[default:"__"]: A string containing a separator used between prefix,
scopes, and settings' names.loaders[default:None]: A sequence of loaders' names, defining the
loaders that should be skipped or used (seeexcludeproperty). This list
can contain loaders that are unknown to the Settings Collector, making it
possible to explicitly include or exclude loaders that might be defined only
in some frameworks (i.e., outside ofsettings_collector).exclude[default:True]: IfTrue, the loaders listed inloadersare
skipped; otherwise, they are the only ones used. Note that the loaders withenabledset toFalsewill always be skipped.load_all[default:False]: IfFalse, the settings are loaded by the
first loader that can provide them (i.e., the other loaders are not used). If
this is changed toTrue, the values are fetched from all the loaders, with
the latter ones overriding the former ones. This'll rarely make sense, since
each loader covers one framework and a project is not likely to use more than
one of them, but it might make sense if you want to combine settings in the
environment variables with those in the framework.greedy_load[default:True]: IfTrue, then all the settings are loaded
when one of them is requested, thus minimising the overhead of the settings
collector. If this is changed toFalse, each setting is loaded when
requested and not before.Local function argumentsBecause Settings Collectors are meant to be used by packages to pull the
settings' default values from various frameworks, the packages themselves are
not meant to be tied to any specific framework, which means that there could be
no framework nor central configuration at all. One would still want their
functions and classes to be configurable, and this is normally done through
functions' and methods' arguments.To make it easier to achieve this, the package provides@sc_defaultsdecorator. Its purpose is to populate functions' and methods' arguments from a
settings collector during runtime, assuming that the values were not given in
the call nor as function's or method's defaults.A typical use case is this:classmy_settings(SettingsCollector):foo=SC_Setting("food")bar=SC_Setting("bard")@sc_defaults(my_settings)deff(x,foo):returnf"{x}:{foo}"Here, we can do several different calls (assuming thatfooandbarare not
set in the framework):f(17)returns"17:food"becausex = 17comes from the call, whilefoois unset and thus gets its value frommy_settings.foo.f(17, "afoot"),f(17, foo="afoot"), andf(x=17, foo="afoot")all return"17:afoot"because both values were provided in the function call.The same can be done with methods in classes, with one caveat: if the method in
question is a class method, the@classmethoddecorator must be put before@sc_defaults.If some argument has its default defined in the function's or method's
signature, its value will never be picked from the attached settings collector.Scopes are supported by an optional keyword argumentscope_argwhich holds
the name of the argument that defines scope. If not set, the scopes will not be
used. Here is an example:@sc_defaults(my_settings,scope_arg="namespace")deff(foo,namespace=None):returnf"{foo}"print("Argument-provided foo: ",f("foot"))print("Default scope's foo:   ",f())print("Value of foo in scope1:",f(namespace="scope1"))For more usage examples, seetests/test_defaults.py.Settings in projects with no frameworksProjects that do not use any of the supported or supporting frameworks can
still adjust settings for those packages that support Settings Collector. This
is done through a dictionary-like object calledsc_settingsand is used like
this:fromsettings_collectorimportsc_settings# Set values one by one:sc_settings["prefix1__some_setting"]=...sc_settings["prefix2__scope1__scope2__some_setting"]=......# Or, in bulk:sc_settings.update({"prefix1__some_setting"]:...,"prefix2__scope1__scope2__some_setting"]:...,...}Note that you should never dosc_setting = .... If you do, the loader
responsible for reading this data will rebel by raising aSC_SettingsErrorexception.This can be used even without any other packages using Settings Collector, as a
placeholder for your project's configuration, although it's a bit questionable
what the benefits would be (over just having your own Flask-likeconfigdictionary or Django-like classsettings), especially if one does not need
scopes.Custom loadersAdding the support for Settings Collector to your own framework is easy.
Obviously, you can submit it to me for inclusion in this package, but it is
just as easy to include it in the package itself.All you need to do is create a module:try:fromsettings_collectorimportSC_LoaderBaseexceptImportError:passelse:classSC_MyFrameworkLoader(SC_LoaderBase):...The name of your loader class must begin withSC_and end withLoader.Make sure that this module is imported. The creation of a subclass ofSC_LoaderBaseautomatically registers it with the one that Settings Collector
can recognise.Notice that this module does not requiresettings_collector, so your
framework won't require it either. If no packages requiringsettings_collectorare used, the import above will be silently ignored.Since the configurations are usually defined as attributes in modules or
classes (like in Django) or as keys in a dictionary (like in Flask oros.environ), there are subclasses that make it easier to implement loaders
for any framework using one of these approaches. These areSC_LoaderFromAttribsandSC_LoaderFromDict. To see how they are used, see
the source code forSC_DjangoLoaderand forSC_EnvironLoader.Testing custom loadersOne can easily test their shiny new loader.Install Settings Collector in some project using your framework:pipinstallsettings_collectorAdd your loader to the project and import its module (so that the loader
gets registered).Run the following code:importsettings_collectorsettings_collector.sc_test_print_expected()This will print the testing variables as the test function expects them.Add these settings to your project's config. Make sure that they are named
consistently with how it is done in your framework. For example, the names
of these settings would be uppercased in Django's config, despite them
having mixed cases in the output ofsc_test_print_expected.Restart your project and run:importsettings_collectorsettings_collector.sc_test_run()Note that you can also runsettings_collector.sc_test_run(False)if you want
a bit less verbose output.