Package: pkonfig
PKonfigPstands forPython.PrerequisitesPythonic configuration management helpers.Multiple sources of configs (environment variables, dotenv files, YAML, JSON, TOML, INI)
with agile order configuration.Configs validation mechanics based on type hints or user defined classes.Minimal external dependencies.FollowFail-fastprinciple.Autocomplete in modern IDEs.FeaturesUser defined config source order.Multilevel configs for environment variables and dotenv config sources.Custom aliases for fields or groups of configs.Configs type castingConfig values validation based on type and/or value.High performance.Extendable API.InstallationTo install basic PKonfig without YAML and TOML support run:pipinstallpkonfigYAML files parsing is handled withPyYaml:pipinstallpkonfig[yaml]TOML files handled with help ofTomli:pipinstallpkonfig[toml]And if both TOML and YAML is needed:pipinstallpkonfig[toml,yaml]For production no.envfiles are needed but propper environment variables should be set.
In case some of required variables missingKeyErrorexception raised whileAppConfiginstantiation.QuickstartThe most basic usage example when environment variables are used for production
environment and DotEnv files are used for local development.Createconfigmoduleconfig.py:fromcollectionsimportChainMapfrompkonfigimportConfig,EmbeddedConfig,Env,DotEnv,LogLevel,ChoiceclassPG(EmbeddedConfig):host="localhost"port=5432user:strpassword:strclassAppConfig(Config):db=PG()log_level=LogLevel("INFO")env=Choice(["local","prod","test"],default="prod")storage=ChainMap(DotEnv(".env",missing_ok=True),Env())config=AppConfig(storage)For local development create DotEnv file in root app folder.env:APP_DB_HOST=localhost
APP_DB_USER=postgres
APP_DB_PASSWORD=postgres
APP_ENV=local
APP_LOG_LEVEL=debugThen elsewhere in app you could run:fromconfigimportconfigprint(config.env)# localprint(config.log_level)# 20print(config.db.host)# localhostprint(config.db.port)# 5432print(config.db.user)# postgresprint(config.db.password)# postgresUsageConfig sourcesPKonfigimplements several config sources out of the box.
All config sources implementMappingprotocol and default values could be set up during initialization.Environment variablesThe most common way to configure application is environment variables.
To parse environment variables and store values in multilevel structure classEnvcould be used.
Common pattern is naming variables with multiple words describing the exact purpose
more precise:PG_HOST,PG_PORTandREDIS_HOST,REDIS_PORTcould be treated as two groups:PGHOSTPORTREDISHOSTPORTPKonfig respects this convention so thatEnvhas two optional arguments:delimiterstring that will be used to split configuration levels taken from keys;prefixstring that is used to identify keys that are related to the given app and omit everything else.fromosimportenvironfrompkonfigimportEnvenviron["APP_OUTER"]="foo"environ["APP_INNER_KEY"]="baz"environ["NOPE"]="qwe"source=Env(delimiter="_",prefix="APP",some_key="some")print(source["outer"])# fooprint(source["inner"]["key"])# bazprint(source["nope"])# raises KeyErrorEnvignores key cases and ignores all keys starting not fromprefix.
To change this behaviour setprefixtoNone.
In this case you will get all key value pairs:fromosimportenvironfrompkonfigimportEnvenviron["NOPE"]="qwe"source=Env(prefix=None)print(source["nope"])# qweDotEnvIn the same manner as environment variables DotEnv files could be used.DotEnvrequires file name as a string or a path and also acceptsdelimiterandprefixoptional arguments.missing_okargument defines whetherDotEnvraises exception when given file not found.
When file not found andmissing_okis setDotEnvcontains empty dictionary.frompkonfigimportDotEnvconfig_source=DotEnv("test.env",delimiter="_",prefix="APP",missing_ok=True)IniINIfiles are quite common and classIniis build on top ofconfigparser.ConfigParser:frompkonfigimportInistorage=Ini("config.ini",missing_ok=False)print(storage["bitbucket.org"]["User"])# hgprint(storage["bitbucket.org"]["ServerAliveInterval"])# 45In case whenconfig.ini:[DEFAULT]ServerAliveInterval=45[bitbucket.org]User=hgInialso acceptsmissing_okargument to ignore missing file.
Most ofConfigParserarguments are also accepted to modify parser behaviour.JsonJsonclass usesjson.loadto read given JSON file and respectsmissing_okargument:frompkonfigimportJsonstorage=Json("config.json",missing_ok=False)YamlTo parse YAML filesPyYamlcould be used wrapped withYamlclass:frompkonfigimportYamlstorage=Yaml("config.yaml",missing_ok=False)TomlTOML files are parsed withtomliwrapped withTomlhelper class:frompkonfigimportTomlstorage=Toml("config.toml",missing_ok=False)Source orderAny source forBaseConfigshould implementMapperprotocol.
So it is easy to implement custom or combine existing implementations.
Recommended way to combine multiple sources of configs isChainMap:fromcollectionsimportChainMapfrompkonfigimportEnv,DotEnv,Yamlconfig_source=ChainMap(DotEnv("test.env",missing_ok=True),Env(),Yaml("base_config.yaml"))In this example we createdChainMapthat looks for key until finds one in the given mappers sequence.
The first one source for configs istest.envfile that might not exist and could be used for local development only.
Environment variables are used as the second one config source.
Dotenv file will be preferred source in this example.
The last one source isbase_config.yamlthat should exist orFileNotFoundErrorexception raised.You can customize source order in this way or even create your own logic implementingMapperprotocol.ConfigTo implement application config class user should inherit frompkonfig.config.Configclass and define
required fields:frompkonfigimportConfigclassAppConfig(Config):foo:floatbaz:intstorage={"foo":"0.33","baz":1}config=AppConfig(storage)print(config.foo)# 0.33print(config.baz)# 1To build more granular config structureEmbeddedConfigclass is used:frompkonfigimportConfig,EmbeddedConfigclassInner(EmbeddedConfig):key:strclassAppConfig(Config):inner=Inner()foo:floatbaz:intstorage={"foo":"0.33","baz":1,"inner":{"key":"value"}}config=AppConfig(storage)print(config.inner.key)# valueMultilevel ConfigGrouping might be useful when there are lots of config parameters.
To achieve thisEmbeddedConfigclass should be inherited:frompkonfigimportDotEnv,Config,EmbeddedConfigclassPgConfig(EmbeddedConfig):host:strport:int=5432classRedisConfig(EmbeddedConfig):host:strport:int=6379classAppConfig(Config):pg=PgConfig()redis=RedisConfig()config=AppConfig(DotEnv(".env",delimiter="__",prefix="APP"))print(config.pg.host)# db_hostprint(config.pg.port)# 6432print(config.redis.host)# redis.envcontent:APP__PG__HOST=db_host
APP__PG__PORT=6432
APP__REDIS__HOST=redisIn this example we customized delimiter with two underscores, default is '_'.AliasesAllConfigfields acceptaliasargument.
When storage class searches for config attribute in its source either attribute
name is used or alias when it is set.config.py:frompkonfigimportDotEnv,EmbeddedConfig,Config,Int,StrclassHostConfig(EmbeddedConfig):host:strport:intuser:strpassword=Str(alias="pass")classAppConfig(Config):pg=HostConfig(alias="db")foo_baz=Int(alias="my_alias")config=AppConfig(DotEnv(".env",delimiter="__")).envcontent:APP__DB__HOST=db_host
APP__DB__PORT=6432
APP__DB__PASS=password
APP__DB__USER=postgres
APP__MY_ALIAS=123In this example storage will seek in dotenv file parameters named by given alias.
Elsewhere in an app:fromconfigimportconfigprint(config.foo_baz)# 123print(config.pg.password)# passwordPKonfig fieldsAll simple Python data types are implemented in field types:Bool,Int,Float,Str,Byte,ByteArray.
All fields with known type converted to descriptors during class creation.
Fields inConfigclasses may be defined in several ways:Using types:frompathlibimportPathfrompkonfigimportConfigclassAppConfig(Config):foo:strbaz:intflag:boolfile:PathUsing default values:frompathlibimportPathfrompkonfigimportConfigclassAppConfig(Config):foo="some"baz=1flag=Falsefile=Path("some.text")Given values will be used as default values.Using PKonfig fields directlyfrompkonfigimportConfig,PathField,Str,Int,BoolclassAppConfig(Config):foo=Str()baz=Int()flag=Bool()file=PathField()CachingAllPKonfigfield types are Python descriptors that are responsible for type casting and data validation.
In most cases there is no need to do this job every time the value is accessed.
To avoid undesirable calculations caching is used.
So that type casting and validation is done only once
duringConfigobject initialization.
In case when configuration may change during application lifecycle user may disable this behaviour:frompkonfigimportConfig,IntclassAppConfig(Config):attr=Int(no_cache=True)In given exampleattrwill do type casting and validation every time this attribute is accessed.Default valuesIf value is not set in config source user can use default value.Nonecould be used as default value:frompkonfigimportConfig,Int,StrclassAppConfig(Config):int_attr=Int(None)str_attr=Str(None)config=AppConfig({})print(config.str_attr)# Noneprint(config.int_attr)# NoneWhenNoneis default value field is treated as nullable.Field nullabilityTo handle type casting and validation fields should not be nullable.
In caseNoneis a valid value and should be used without casting and validation
optionnullablecould be set:frompkonfigimportInt,ConfigclassAppConfig(Config):int_attr=Int(nullable=True)config=AppConfig(dict(int_attr=None))print(config.int_attr)# NoneIn this exampleNonecomes from storage and type casting is omitted.Custom descriptor or propertyfrompkonfigimportConfigclassAppConfig(Config):flag=Truebaz="test"@propertydefvalue(self):returnself.flagandself.baz=="test"config=AppConfig({})print(config.value)# TrueCustom field typesUser can customize how field validation and casting is done.
The recommended way is to implementvalidatemethod:frompkonfigimportConfig,IntclassOnlyPositive(Int):defvalidate(self,value):ifvalue<0:raiseValueError("Only positive values accepted")classAppConfig(Config):positive=OnlyPositive()Custom type casting is also available.
To achieve this user should inherit abstract classFieldand implement methodcast:fromtypingimportListfrompkonfigimportFieldclassListOfStrings(Field):defcast(self,value:str)->List[str]:returnvalue.split(",")Available fieldsBuiltin Python types has appropriateFieldtypes:bool ->Boolint ->Intfloat ->FloatDecimal ->DecimalFieldstr ->Strbytes ->Bytebytearray ->ByteArrayThe only reason to use this types directly is customising field nullability and cache policy.PathFieldBasic path type that is parental for other two types and is used when you define field usingpathlib.Path.
This type raisesFileNotFoundErrorexception during initialization if given path doesn't exist:frompkonfigimportConfig,PathFieldclassAppConfig(Config):mandatory_existing_path=PathField()optional_path=PathField(missing_ok=True)In given example fieldoptional_pathmay not exist during initialization.FileFileinheritsPathFieldbut also checks whether given path is a file.FolderFolderinheritsPathFieldand does checking whether given path is a folder.EnumFieldThis field uses custom enum to validate input and cast it to givenEnum:fromenumimportEnumfrompkonfigimportConfig,EnumFieldclassUserType(Enum):guest=1user=2admin=3classAppConfig(Config):user_type=EnumField(UserType)config=AppConfig({"user_type":"admin"})print(config.user_typeisUserType.admin)# TrueLogLevelLogLevelfield is useful to definelogginglevel through configs.LogLevelaccepts strings that define log level and casts
that string tologginglevel integer value:importloggingfrompkonfigimportConfig,LogLevelclassAppConfig(Config):some_level=LogLevel()another_level=LogLevel()config=AppConfig({"some_level":"info","another_level":"Debug",})print(config.some_level)# 20print(config.another_level)# 10print(config.another_levelislogging.DEBUG)# TrueChoiceChoicefield validates that config value is a member of the given sequence and also does optional type casting:frompkonfigimportConfig,ChoiceclassAppConfig(Config):one_of_attr=Choice([10,100],cast_function=int)config=AppConfig({"one_of_attr":"10"})print(config.one_of_attr==10)# Trueconfig=AppConfig({"one_of_attr":"2"})# raises TypeError exceptionWhencast_functionis not given raw values from storage are used.DebugFlagDebugFlaghelps to set widely useddebugoption.DebugFlagignores value case and treats'true'string asTrueand any other value asFalse:frompkonfigimportConfig,DebugFlagclassAppConfig(Config):lower_case=DebugFlag()upper_case=DebugFlag()random_string=DebugFlag()config=AppConfig({"lower_case":"true","upper_case":"TRUE","random_string":"foo",})print(config.lower_case)# Trueprint(config.upper_case)# Trueprint(config.random_string)# FalseTypes to Fields mappingAll fields forBaseConfigchildren classes are converted to descriptors internally.
Classpkonfig.config.DefaultMapperdefines how field types will be replaced with descriptors.
This mapping is used by default:{
    bool: Bool,
    int: Int,
    float: Float,
    str: Str,
    bytes: Byte,
    bytearray: ByteArray,
    Path: PathField,
    Decimal: DecimalField,
}When field type is not found in this mapper it is ignored and won't be taken from storage source while resolving.User can modify default mapper giving dictionary of types and appropriate fields:fromdecimalimportDecimalfrompkonfigimportConfig,DefaultMapper,DecimalFieldclassAppConfig(Config):_mapper=DefaultMapper({float:DecimalField})foo:floatconfig=AppConfig(dict(foo=1/3))assertisinstance(config.foo,Decimal)# TruePer-environment config filesWhen your app is configured with different configuration files
and each file is used only in an appropriate environment you can create a function
to find which file should be used:frompkonfigimportEnv,Yaml,Config,ChoiceCONFIG_FILES={"prod":"configs/prod.yaml","staging":"configs/staging.yaml","local":"configs/local.yaml",}defget_config_file():class_Config(Config):env=Choice(["prod","local","staging"],cast_function=str.lower,default="prod")_config=_Config(Env())returnCONFIG_FILES[_config.env]get_config_fileuses environment variables and predefined config files pathes
to check whetherAPP_ENVvar is set, validate this variable and return appropriate
config file name.
Then actual application configuration:fromcollectionsimportChainMapfrompkonfigimportEnv,Yaml,Config,ChoiceCONFIG_FILES={"prod":"configs/prod.yaml","staging":"configs/staging.yaml","local":"configs/local.yaml",}defget_config_file():class_Config(Config):env=Choice(["prod","local","staging"],cast_function=str.lower,default="prod")_config=_Config(Env())returnCONFIG_FILES[_config.env]classAppConfig(Config):env=Choice(["prod","local","staging"],cast_function=str.lower,default="prod")...storage=ChainMap(Env(),Yaml(get_config_file()),)config=AppConfig(storage)