Package: pyjeb
PyjebPyJeb is a powerfull lightweight library to check and variabilize your configuration files.
The main features of pyjeb are:Control the structure of a configuration file (missing value, type, format, valid set)Add default value for missing fieldsSetup variable values (system or user defined)Allow case insensitive parametersAllow to add executable functions in configurationGet startedStepsInstall PyJeb packageSetupcontrol fileSetup configuration fileCallcontrol_and_setupfunctionInstall PyJebInstall fromPyPipackage manager:pipinstallpyjebExempleSetup a configuration file for your script.In this exemple a yaml file will be used, but JSON can also be used.Exemple of yaml configuration file - configuration.yamlHR - Employees:source:path:"/Landing/HR/Employees/$sys.timestamp('YYYY-MM-DD')"pattern:"*.csv"target:path:"/Bronze/HR/Employees"HR - Managers:source:path:"/Landing/HR/Managers/$sys.timestamp('YYYY-MM-DD')"target:path:"/Bronze/HR/Managers"HR - Payroll:source:path:"/Landing/HR/Payroll/$sys.timestamp('YYYY-MM-DD')"target:path:"/Bronze/HR/Payroll"Setup a configuration to describe the previous configuration file and add default value for non mendatory fieldExemple of yaml configuration file - control.yamlcontrol=[{"name":"source","type":"dict"},{"name":"source.path"},{"name":"source.pattern","default":"*"},{"name":"target","type":"dict"},{"name":"target.path"},]Load the configuration and apply functioncontrol_and_setupfunction with control parameters.This function will:Setup default if the value is not definedSetup value of variable ($sys.timestamp in this exemple)Set configuration as object (callable with dots)importyamlfrompyjebimportcontrol_and_setup# load configuration filewithopen("configuration.yaml")asf:configuration=yaml.load(f,Loader=yaml.loader.SafeLoader)# control configurationcontrol=[{"name":"source","type":"dict"},{"name":"source.path"},{"name":"source.pattern","default":"*"},{"name":"target","type":"dict"},{"name":"target.path"},]# loop on each item in configurationforiteminconfiguration:item_configuration=configuration[item]# apply the control and instantiate variablesitem_configuration=control_and_setup(item_configuration,control,to_object=True)# display valuesprint(f"---------------{item}")print(f"source.path = '{item_configuration.source.path}'")print(f"source.pattern = '{item_configuration.source.pattern}'")print(f"target.path = '{item_configuration.target.path}'")Output of the script--------------- HR - Employeessource.path='/Landing/HR/Employees/2023-12-14'source.pattern='*.csv'target.path='/Bronze/HR/Employees'--------------- HR - Managerssource.path='/Landing/HR/Managers/2023-12-14'source.pattern='*'target.path='/Bronze/HR/Managers'--------------- HR - Payrollsource.path='/Landing/HR/Payroll/2023-12-14'source.pattern='*'target.path='/Bronze/HR/Payroll'setup_and_control functionThe functioncontrol_and_setupis the only one to use in PyJeb. It use to apply controls and setup default and variables values.Definition:control_and_setup(
<configuration>,
<controls>,
[<variables>],
[<functions>]
)NameMandatoryTypeDescriptionconfigurationyesDictA dictionary (of scalar or array or dictonnary) who contain the configuration to control and variabilizecontrolsyesListA list of dictionnaries who define the structure of configuration. See all about the structure incontrols pagevariablesnoDictCustom variables accessible by configuration. See all about it incustom variable pagefunctionsnoDictCustom parametrized variables accessible by configuration. See all about it incustom variable pageto_objectnoBoolTransform the output into an object. If false the output keeps dictionary format (default: false)