Package: configzen
configzenv0.10.0configzen– easily create and maintain complex, statically-typed configurations with validation in Python.It's important to keep your configuration safe and consistent. Give a shot toconfigzen🚀⭐ SupportsPython 3.8 or above,⭐ Isfully typed,⭐ SupportsYAML, JSON, TOML, INI, XML, ConfigObj, BSON, CBOR, Amazon Ion, propertiesandshellvars,⭐ Supportsreading and writing configuration files, fully or partially, with the ability to preserve the original file structure (without comments[^1]),⭐ Supportsconfiguration preprocessing(extending, including and copying configuration files without the need to change the code),⭐ Supportsvariable interpolation(runtime value substitution),⭐ Supportsmodular configuration with type validation(wrapping runtime Python modules in-place and outside them),⭐ Supportssynchronous and asynchronous file operations,⭐ Supports loading configuration fromenvironment variables and secret files.While being built on top ofpydantic,configzeninherits most of its features, includingdata validation,schema generation,custom data types, good integration withRich, and more.Learn more below.FeaturesManaging contentHaving a YAML configuration file like this:# database.ymlhost:127.0.0.1port:5432user:postgresYou can create aconfigzenconfiguration model for it like this:# model.pyfromipaddressimportIPv4Address,IPv6AddressfromconfigzenimportConfigField,ConfigMeta,ConfigModelclassDatabaseConfig(ConfigModel):host:IPv4Address|IPv6Addressport:intuser:strpassword:str=ConfigField(exclude=True)classConfig(ConfigMeta):resource="database.yml"env_prefix="DB_"db_config=DatabaseConfig.load()And you can load your configuration from a file as well as from the environment variablesDB_HOST,DB_PORT,DB_USERandDB_PASSWORD. Sincepasswordis a field created with
the optionexclude=True, it will not be included in the configuration's exported data: that
guarantees that your password does never leak intodatabase.ymlon save – but you may still pass it
through an environment variable (here – the mentionedDB_PASSWORD). Secret files are also supported,
seethe pydantic documentationfor more information.pydanticwill naturally take care of parsing and validating the loaded data.
Configuration models inherit from thepydantic.BaseSettingsclass, so you can use all of its features:
schema generation, type conversion, validation, etc.There are additional features brought to you byconfigzenworth checking out, though.You can use thedb_configobject defined above to access the configuration values:>>>db_config.hostIPv4Address('127.0.0.1')modify them, if the pydantic model validation allows
it (<Your model>.Config.validate_assignmentwill
beTrueby default):>>>db_config.host="0.0.0.0">>>db_config.hostIPv4Address('0.0.0.0')as well as reload particular values, without touching the rest of the configuration:>>>db_config.at("port").reload()5432>>>db_configDatabaseConfig(host=IPv4Address('0.0.0.0'),port=5432,user='postgres',password='password')>>>db_config.at("host").reload()IPv4Address('127.0.0.1')>>>db_configDatabaseConfig(host=IPv4Address('127.0.0.1'),port=5432,user='postgres',password='password')or reload the whole configuration:>>>db_config.port=1234>>>db_config.reload()DatabaseConfig(host=IPv4Address('127.0.0.1'),port=5432,user='postgres',password='password')or save a particular value, without touching the rest of the configuration:>>>db_config.host="0.0.0.0">>>db_config.port=443>>>db_configDatabaseConfig(host=IPv4Address('0.0.0.0'),port=443,user='postgres',password='password')>>>db_config.at("host").save()40>>>db_config.reload()DatabaseConfig(host=IPv4Address('0.0.0.0'),port=5432,user='postgres',password='password')or save the whole configuration:>>>db_config.save()39PreprocessingTo see supported preprocessing directives,
seeSupported preprocessing directives.Basic usageHaving a base configuration file like this (base.json):{"i18n":{"language":"en","timezone":"UTC"},"app":{"debug":true,"expose":8000}}create another configuration file like this, overriding desired sections as needed:# production.yml^extend:base.json+app:debug:falseand load theproduction.ymlconfiguration file. No explicit changes to the code indicating the use of thebase.jsonfile are needed.Note: Using+in front of a key will update the section already defined at that key,
instead of overwriting it entirely.Notice how configuration file formats do not matter inconfigzen: you can
extend JSON configurations with YAML, but that might be as well any other format
among the supported ones (see theSupported file formatssection).The above example is equivalent to as if you used:# production.ymli18n:language:entimezone:UTCapp:debug:falseexpose:8000but with a significant difference: when you save the above configuration, the^extendrelation to the base
configuration filebase.jsonis preserved.
This basically means that changes made in the base configuration file will apply to the configuration model instance
loaded from the^extend-ing configuration file.
Any changes made locally to the model will result in+sections being automatically added to the exported
configuration data.Supported preprocessing directivesDirectiveIs the referenced file preprocessed?Is the directive preserved on export?^extendYesYes^includeYesNo^copyNoNoInterpolationBasic interpolationYou can use interpolation in your configuration files:cpu:cores:4num_workers:${cpu.cores}>>>fromconfigzenimportConfigModel...>>>classCPUConfig(ConfigModel):...cores:int...>>>classAppConfig(ConfigModel):...cpu:CPUConfig...num_workers:int...>>>app_config=AppConfig.load("app.yml")>>>app_configAppConfig(cpu=CPUConfig(cores=4),num_workers=4)Reusable configuration with namespacesYou can share independent configuration models as namespaces through inclusion:# database.ymlhost:${app_config::db_host}port:${app_config::expose}# app.ymldb_host:localhostexpose:8000>>>fromconfigzenimportConfigModel,include>>>fromipaddressimportIPv4Address>>>>>>@include("app_config")...classDatabaseConfig(ConfigModel):...host:IPv4Address...port:int...>>>classAppConfig(ConfigModel):...db_host:str...expose:int...>>>app_config=AppConfig.load("app.yml")>>>app_configAppConfig(db_host='localhost',expose=8000)>>>db_config=DatabaseConfig.load("database.yml")>>>db_configDatabaseConfig(host=IPv4Address('127.0.0.1'),port=8000)>>>db_config.dict(){'host':IPv4Address('127.0.0.1'),'port':8000}>>>db_config.export()# used when saving{'host':'${app_config::db_host}','port':'${app_config::expose}'}You do not have to pass a variable name to@include, though.@includelets you overwrite the main interpolation namespace
or one with a separate name (here:app_config) with configuration models, dictionaries and their factories.Modular configurationWrapping modules in-placeYou can wrap modules in-place with configuration models:Without writing a model class:# config.pyfromconfigzenimportConfigModule# Annotate config fieldsHOST:str="localhost"PORT:int=8000ConfigModule.wrap_this_module()With a model class:# config.pyfromconfigzenimportConfigModel# Annotations are optionalHOST="localhost"PORT=8000classAppConfig(ConfigModel):HOST:strPORT:intAppConfig.wrap_this_module()Now valuesHOSTandPORTwill be validated asstrandintdata types, respectively:>>>importconfig# <configuration module 'config' from 'config.py'>>>>config.HOST'localhost'>>>config.PORT8000>>>config.PORT="8000">>>config.PORT8000>>>config.PORT="abc"Traceback(mostrecentcalllast):...Wrapping interchangeable modulesYou can wrap modules outside them with configuration models:# setup.pyfromconfigzenimportConfigModelclassAppConfig(ConfigModel):HOST:str="localhost"PORT:int=8000config_model=AppConfig.wrap_module("config")# config.pyHOST:str="0.0.0.0"PORT:int=443>>>fromsetupimportconfig_model>>>config_model.HOST'0.0.0.0'>>>config_model.PORT443>>>config_model.PORT="8000">>>config_model.PORT8000>>>importconfig>>>config.HOST'0.0.0.0'>>>config.PORT8000Supported file formatsconfigzenusesanyconfigto serialize and deserialize data and does not operate on any protocol-specific entities.
As an example result, comments in your configuration files are lost on save[^1], but you can exchange file formats without any hassle.The following table shows the supported file formats, their requirements, file extensions, and the backend libraries used to accomplish this goal.File FormatTo use, install:Recognized File Extension(s)Backend LibraryJSON-jsonjson(standard library)INI-ini,cfg,confconfigparser(standard library)TOML-tomltomlYAML-yaml,ymlpyyaml/ruamel.yamlXML-xmlxml(standard library)ConfigObjanyconfig-configobj-backendconfigobjconfigobjBSONanyconfig-bson-backendbsonbsonCBOR(RFC 8949)anyconfig-cbor2-backendcbor,cbor2cbor2Amazon Ionanyconfig-ion-backendionionCBOR (deprecated,RFC 7049)anyconfig-cbor-backendcborcborproperties-properties(native)shellvars-shellvars(native)If your file extension is not recognized, you can register your own file extension by callingConfigAgent.register_file_extension(file_extension, parser_name).If your favorite backend library is not supported, please let me know by reporting it as an issue.
Using custom backends is to be supported in the future.[^1]: A suggested alternative for comments is to use thedescriptionparameter in your configuration models' fields:ConfigField(description=...).
The provided field descriptions are included in JSON schemas generated by the default implementation of theConfigModel.schema()method.SetupIn order to useconfigzenin your project, install it with your package manager, for examplepip:pipinstallconfigzenIf you are willing to contribute toconfigzen, which is awesome, simply clone this repository and install its
dependencies withpoetry:poetryinstall--withdev--all-extrasAfter that, install thepre-commithooks:pre-commitinstall--hook-typepre-commit--hook-typepre-pushYou might also need to install required stubs. First, activate your virtual environment:poetryshelland run (on Linux orPowerShell7.0+):mypyconfigzen/||(echoyes|mypy--install-types)or, if you are using an older version of PowerShell:mypyconfigzen/;if(-not$?){echoyes|mypy--install-types}Note: UsingWSL2+pyenv(with Python 3.8) for developing configzen is strongly recommended in case you use Windows.And you are good to go. 🚀Contributions are welcome! Feel free toopen an issuewhenever
you encounter a bug or have a feature request orsubmit a pull requestwith your changes.LicenseMIT LicenseCredits@Lunarmagpieforcrucialdesign tips and ideas.Authorbswck(contact:bswck.dev@gmail.comor viaDiscordbswck)