Package: revo
Revo –ReferencedValues inObjectsRevois a non-intrusive variable substitution solution for config files.Revoislisted on PyPI.Revocasts a Python built-indictorlistinto a mutable tree, where the
value of each tree node is either a string or a number, while any node can
reference any other node in the tree with GNU make style$(var)variable
syntax.Although a tiny (under 200 lines of code) library,revoprovides some
level of programmability to plain old Python objects and thus great
flexibility to many applications, especially configuration processing. No
matter what format (JSON,YAML,TOML, etc.) your config file is, as long
as it maps to Python built-in types cleanly,revoprovides Makefile-style
variable substitution to it.Design ideasRevotreats a Python built-in object as a tree of data, like an XML
doc. The top-level object must bedictorlist. Values on tree nodes must
be eitherstr,bool, or built-in number (intorfloat).In order to referenceanynode in an object tree, we need to design a path
mechanism. Yes, the idea is likeXPathfor Python objects, but much simpler.In fact, we can almost simply borrow the design of UNIX path: slash-separated
string, with only a small adaptation: path segments in integer literals are
treated as numeric indexes forlist.An exmple would be nice, isn't it?# This is legal Python code, making an object from literals.# You can also construct it from a YAML or JSON file.conf={"project":{"name":"revo","version":"0.1.0","rules":{"Homepage":"https://github.com/sunyj/$(project/name)"}},"classifiers":["Programming Language :: Python :: 3","License :: OSI Approved :: MIT License","Operating System :: OS Independent"],"tag":"$(classifiers/0)",# reference a list element with integer index"v1":"project","v2":"version","v3":"$($(v1)/$(v2))"# support variables in variable names}# Resolve variable references with revofromrevoimportRevoobj=Revo(conf).resolve()print(obj['project/rules/Homepage'])# https://github.com/sunyj/revoprint(obj['tag'])# Programming Language :: Python :: 3print(obj['v3'])# 0.1.0Typical use casesAlmost everyone who works long enough with config files appreciates the great
value of a robust and flexible variable substitution mechanism. It helps to
reduce tedious config-processing logic by moving them to config files.Revo was designed to be a non-intrusive solution for that purpose. It resolves
on Python objects, not the files storing those objects, so it's transparent to
the config file format.Design detailsThe core mechanism is the path to reach any node in the object tree. With a
unique path for every node, an object can be melted down to a set of flat dict
of key-value mappings. The definition of classRevoreflects that.classRevo(MutableMapping):......As the path of every node is unique within an object, it may serve as thenamefor the corresponding node's value. Voilà!variable, as in the
context of "variable substitution", can now be properly defined.ConceptImplementationvariabletree nodevariable namenode pathvariable valuenode valueVariables Make Makefiles Simpler,
equally true for config files. We simply use the variable reference syntax ofGNU make.Variable overridesAnother common need for config files is overriding values from command line or
other input sources.Revosupports that with itsoverridemethod:fromrevoimportRevooverride_specs=['date=20200110','conf/path=/another/path/with=/in/it']# construct with overridesconf_obj=load_my_json_config(...)conf=Revo(conf_obj,override_specs)# or in a separate callconf=Revo(load_my_yaml_config(...))conf.override(override_specs).resolve()Override spec parsing rules:First=is used as the name-value separator. Subsequent=characters are all put into the value string.Revo always tries parsing the value string as a Python literal, and falls back to string if that fails.Fault toleranceKeyword-only boolean argumentmercycontrols ifrevoraises exceptions
on errors during the resolution process. Typical errors are:Unknown variable.Illegal variable syntax.Self-reference or circular reference.It defaults toFalse.  Whenmercy=True, unresolved or partially-resolved
values are left in the object.Definition mergingTop-level overrides may contain variables that are not in the object under
resolve, for example:importrevoconf=revo.Revo({'name':'hello $(date)'},['date=20220101'])conf['name']# 'hello 20220101'conf['date']# what do you expect?Such top-level overrides are calleddefinitions.  Definitions may stay in
the object after the resolution.  Keyword-only boolean argumentabsorbcontrols this, it defaults toFalse.Definition extendingOverrides may also contain variables with new leaf-node values, for example:importrevoobj={'data':{'name':'foo'}}conf=revo.Revo(obj,overrides=['data/func=bar'])# good, as 'func' is a leaf nodeobj['data']['func']# 'bar'# not good if extend is turned offconf=revo.Revo(obj,overrides=['data/func=bar'],extend=False)# error, as new is NOT a leaf nodeconf=revo.Revo(obj,overrides=['new/func=bar'])Keyword-only boolean argumentextendcontrols this, it defaults toTrue.Type retainingKeyword-only boolean argumentretaincontrols ifrevotries to keep
variable value type in substitutions.It defaults toTrue.  Whenretain=False, values resolved are alwaysstr.importrevoconf=revo.Revo({'name':'n$(val)','x':'$(val)','val':10},retain=True)conf.resolve()type(conf['name'])# <class 'str'>type(conf['x'])# <class 'int'>LimitationsVariable substitutions are resolved bottom-up, which meansrevocopies and
iterates over all values in loops until no more incremental substitution
happens. This algorithm is easy to understand and implement, but it's pretty
slow. For average config files with typically dozens or hundreds of entries,
performance should not be a concern on modern computers. However, if you plan
to use it on very large config files, make performance tests before you invest
further.