Package: configoat
confiGOATconfiGOAT is a powerful, flexible, and developer-friendly management tool for all your
environment variables and configurations. 🔥🔥🔥FeaturesHere are some of the features that confiGOAT provides:Manage all environment variables or configuration parameters from a single setup.Support all development, testing, and production environments.Define configurations once, use it everywhere.Define configuration parameters using both YAML and Python scripts.Cast values before using them.Powerful reference mechanism to reuse variablesfrom any levels at any levels in any direction.Multiple resource types in the YAML to support the vast majority of use cases.Support both simple use cases and complex, multi-layered nested configurations.Use dynamic modules to access the parameters through import interface in Python.Use a single exposed API to interact with the layered configurations.Support nested structures to model the configurations as per business needs.🎉🚀🌟InstallationconfiGOAT can be installed withpip:pipinstallconfigoatAlternatively, you can grab the latest source code fromGitHub:gitclonehttps://github.com/aag13/configoatcdconfigoat
pipinstall.How to UseInitial SetupconfiGOAT provides a user-friendly CLI command to initialize the configuration setup
for your project.Go to the root directory of your project and run the following command
in the terminal.configoatinitFor the directory name, enter the name of the folder that will contain all
the configuration files and scripts. E.g."environments/"will create a directory
inside the root directory of your project. Default is"configs/".For the main configuration file, provide a name that confiGOAT will look
for when initializing the setup for your project. (we will see how to do that
later). Default is"main.yaml".While selecting the type of configurations, you have 3 options. Choose the one that
best suits your specific project needs. 👇Single (Only one YAML file): Use this for small projects where all your configurations will fit
in one single YAML file.Nested (Parent-child YAML files): Use this for projects where it makes sense to structure the
configuration files in a nested hierarchy.Nested with Scripts (Includes .py scripts): Use this for large-scale projects where someconfigurations need to be resolved using python scripts on top of a multi-layered hierarchy.For example, if you can fit all your configuration variables in a single file, choose option 1. Choose option
2 if you want to structure all the configuration parameters in separate files based on their types, such as,
security configurations insecurity.yamlfile and database configurations indatabase.yamlfile. Option 3
is best suited for cases where you need both the nested hierarchy structure and need to resolve values
for some parameters that require executing some code.❄️❄️❄️Preparing Configuration FilesNow that you have set up the configuration folder and starting file(s), you need to populate all your
configuration and environment variables in the generated YAML and python script files.First, open themain.yaml(or whatever name you gave during the initial setup) file.Each YAML file has 2 root properties, namelydocandresources. 👇You can usedocto provide basic description on the type of environment or configuration variables
which are defined in this YAML.resourcescontains all the configuration variables that are defined in this file. If you want to
create a new variable, you need to define it insideresources.doc:'Thisisthemainconfigfilewheretheprocessingstartsfrom'resources:...You can define 4 different types of resources in confiGOAT. 👇normal: Use this type if the value of the variable will be different for different environments. You can
use different data types for different environments, such as string, integer, float, boolean, list, and dictionary.var1:type:'normal'value:dev:'valueofvar1fordev'stage:'valueofvar1forstage'uat:'valueofvar1foruat'production:'valueofvar1forproduction'qa:'valueofvar1forqa'common: Use this type if the value of the variable will be same across all environments. We support
the following data types - string, integer, float, boolean, list, and dictionary.var2:type:'common'value:"valueofvar2"var3:type:'common'value:Falsevar4:type:'common'value:100var5:type:'common'value:["Banana","Mango","Apple"]var6:type:'common'value:{"name":"RaihanTheBoss","age":66,"address":{"city":"Dhaka","country":"Bangladesh",}}nested: Use this type for a nested YAML file (Available if you chose option 2 or 3 during setup).pathis the relative path from the project root folder to that nested YAML file, e.g.configs/yamls/nested.yaml.
All the variables which are defined in the specified nested YAML file will be available under the
namespace of thevar7variable. ✨var7:type:'nested'path:'path/from/project/root/to/nested.yaml'script: Use this type for a python script file (Available if you chose option 3 during setup).pathis the relative path from the project root folder to that nested script file, e.g.configs/scripts/script.py.
Only the variables which are defined in thevariable_listproperty will be available from specified
nested script file under the namespace of thevar9variable. ✨var9:type:'script'variable_list:['a','b','c','d','e']path:'path/from/project/root/to/script.py'For thenestedtype, confiGOAT will process the specified YAML filerecursively, so that if
the nested YAML file contains othernestedvariables, it will resolve all those nested YAML files
recursively as well. 👀If you want to reuse the value from another variable using reference, either in the same file or any file
in the configuration hierarchy, you need to use$ref(variable_name)format by replacing the
variable_name with the actual variable in your configuration that you want to refer to. 👇Let's consider a couple of scenarios to demonstrate how variable referencing works. 🌅First, you need to understand the difference between thesourceandtargetvariables. Here,sourcevariable is the one whose reference is being used andtargetvariable is the one that is
using the reference. 🎁🎁Targetvariable in the main config YAML file: If thetargetvariable is in the main config file,
such asmain.yaml, then you can accomplish that in two ways depending on where thesourcevariable is.Source variable in the main config file: If thesourcevariable is in the main config
file, then use$ref(SIBLING)in thetarget variable. This will get the value from a variable
calledSIBLINGto thetargetvariable like this. ✨target_var:type:'common'value:"$ref(SIBLING)"Source variable in a nested file: If there is anestedvariable callednested1and inside
the YAML of this file, there exists a variable calledvarAA, then the fulldot notation pathto this variable
from the main config file isnested1.varAA. So use$ref(nested1.varAA)in thetarget variable. ✨target_var:type:'common'value:"$ref(nested1.varAA)"Targetvariable in any other YAML/script file: If thetargetvariable is in any file other than
the main config file, then you can accomplish that in two ways depending on where thesourcevariable is.Source variable in the same file: If thesourcevariable is in the same file as thetarget variable, then use$ref(SIBLING)in thetarget variable. This will get the value from a variable
calledSIBLINGto thetargetvariable like this. ✨target_var:type:'common'value:"$ref(SIBLING)"Source variable in any other file: If there is anestedvariable callednested1in the
main config file and inside the YAML of thisnestedvariable, there exists a variable calledvarAA, then the fulldot notation pathto this variable is@.nested1.varAA.
So use$ref(@.nested1.varAA)in thetarget variable. ✨target_var:type:'common'value:"$ref(@.nested1.varAA)"So, when to use@in the reference: If thetargetvariable is in the root config file, then you don't need
to add@in the fulldot notation path. Because you can use the dotted path to follow the nested
variable hierarchy since you are already in the root config file. However, if thetargetvariable
is in anywhere other than the root config file, then you need to prepend thedot notation pathwith@to indicate whether to start looking for thenestedvariable from the root file or the
current file. Starting the dotted path with@simply means tostart looking for this variable
from the root config file. 🎂🎂NOTE: Referencing a variable is bidirectional and depth agnostic, meaning that any variable can be referencedat any depth from any depth in any direction, as long as no circular dependency is created during the
referencing of another variable.Circular Dependencymeans that definitions of two variables are dependent
on each other and neither variable can be resolved due to this dependency. In case of any circular dependency,
confiGOAT will raise an exception indicating the circular dependency.🌟🌟🌟Using Configuration Parameters in the ProjectNow that you have prepared the configurations for your project, you need to use them in your project.First, you need to initialize and load all the configuration and environment variables.fromconfigoatimportconfconf.initialize(config="configs/main.yaml",env="dev",module="all_config")configdenotes the path to the main configuration YAML file to start loading the variables from.envdenotes which environment the variables should be loaded for.moduledenotes the name of the namespace under which all variables will be made available for the
dynamic module access.In practice, you don't want to hardcode the environment value like this,env="dev". This way, you
won't be able to change it dynamically on the different environments your app is running on. We
recommend getting this value from another source that can be resolved during runtime. E.g. if you are
using confiGOAT in a Django app, then using CI/CD or starting script, inject the environment value
as the command line argument during the project run. Then, before initialization,
fetch this value fromoslike below,importosfromconfigoatimportconfcurrent_env=os.getenv("YOUR_ENVIRONMENT_VARIABLE")conf.initialize(config="configs/main.yaml",env=current_env,module="all_config")To access the configuration variables, you have 2 options.Dot notation: You can use theconfobject to access any variable by providing its fulldot notation pathfrom the root configuration file.@denotes the root of the configuration,
i.e. the main configuration file. You can also pass theconfvariable around like any other variable in python
and access values like shown below.print(conf("@.VARIABLE_NAME"))print(conf("@.NESTED.VARIABLE_NAME"))print(conf.get("@.VARIABLE_NAME"))print(conf.get("@.NESTED.VARIABLE_NAME"))Dynamic module: Use python's module and import mechanism to access any configuration
variable. In this approach, you import the module that you defined during the initialization step,
e.g.all_config. When confiGOAT initialized your configuration variables, it also created dynamic
modules and attributes in those modules following your configuration nested hierarchy. All these
dynamic modules are inserted under the provided namespace, e.g. theall_configmodule name. After
initialization, you can import this module anywhere in your project and access the variables like any other modules
and their attributes. Some examples are given below on how variables can be accessed using dynamic module.# accessing variables from the root module name, i.e. all_configimportall_configprint(all_config.var3)print(all_config.var2.var4)# importing all variables using * from the root module name, i.e. all_configfromall_configimport*print(var3)print(var2.var4)fromall_configimportvar2ascurrentprint(current.var4)You can also provide a default value in case the variable is not found and a casting function to transform
the final value before returning. Casting and default value features are available only when
accessing values usingconf()orconf.get().print(conf("@.VARIABLE_NAME",default=10,cast=int))print(conf.get("@.VARIABLE_NAME",default=10,cast=int))IssuesPlease let us know if you find any issue byfiling an issue.Maintainers@banna(Hasan-UL-Banna)@galib(Asadullah Al Galib)👋=======
History