Package: auror-core
Auror CoreSimple Flow creation for AzkabanInstallpipinstallauror_coreSupported Job TypesV1CommandFlowV2CommandUsageCreating a simple Azkaban flow with one commandCreating a simple V2 Azkaban flow with one commandCreating Flows with dependenciesSharing job attributesJob with extra customization and configurationIntegrating with Flow (just for V1)Using Flow ParamsUsing Flow Environment VariablesUsing Flow Environment Variables and ParamsJoin multiple variables in oneLoad jobs from YAML File (just for V2)Dump memory flows to a Python File (just for V2)Creating a simple Azkaban flow with one commandYou just need to import job type and projectfromauror_core.v1.jobimportJob,Commandfromauror_coreimportProjectcom1=Job()\.as_type(Command)\.with_name("commands job")\.with_command("bash echo 1")\.with_another_command("bash echo 2")Project("folder_to_generate_files",com1).write()Creating a simple V2 Azkaban flow with one commandV2 flow is implemented under v2 subfolder with same job typesfromauror_core.v2.jobimportJob,Commandfromauror_coreimportProjectcom1=Job()\.as_type(Command)\.with_name("commands job")\.with_command("bash echo 1")\.with_another_command("bash echo 2")Project("folder_to_generate_files",com1).is_v2().write()Creating Flows with dependenciesfromauror_core.v2.jobimportJob,Commandfromauror_coreimportProjectcom1=Job()\.as_type(Command)\.with_name("commands job")\.with_command("bash echo 1")\.with_another_command("bash echo 2")com2=Command()\.with_name("sub command job")\.with_command("bash echo 1")\.with_dependencies(com1)Project("folder_to_generate_files",com1,com2).is_v2().write()Sharing job attributesOrganize jobs with same configurationfromauror_core.v2.jobimportCommandfromauror_coreimportProjectcom=Command()\.with_command("bash echo 1")com1=com.with_name("commands job")\.with_another_command("bash echo 2")com2=com.with_name("sub command job")\.with_dependencies(com1)Project("folder_to_generate_files",com1,com2).is_v2().write()Job with extra customization and configurationSimulating a Command with base Job (NOT RECOMMENDED)fromauror_core.v1.jobimportJobfromauror_coreimportProjectcom1=Job()\.with_name("commands job")\.with_(command="bash echo 1")com1._type="command"Project("folder_to_generate_files",com1).write()Integrating with Flow (just for V1)V2 already have flow includedfromauror_core.v1.jobimportCommand,Flow,Jobfromauror_coreimportProjectcom1=Command()\.with_name("commands job")\.with_command("bash echo 1")flow=Job()\.as_type(Flow)\.with_name("flow")\.with_dependencies(com1)Project("folder_to_generate_files",com1,flow).write()Using Flow Paramsfromauror_core.v2.jobimportCommandfromauror_core.v2.paramsimportParamsfromauror_coreimportProjectparams=Params(teste1="my test",variable="my variable")com=Command()\.with_command("bash echo ${variable}")Project("folder_to_generate_files",com)\.is_v2()\.with_params(params)\.write()Using Flow Environment Variablesfromauror_core.v2.jobimportCommandfromauror_core.v2.paramsimportEnvfromauror_coreimportProjectenv=Env(TESTE="my test",VARIABLE="my variable")com=Command()\.with_command("bash echo $VARIABLE")Project("folder_to_generate_files",com)\.is_v2()\.with_params(env)\.write()Using Flow Environment Variables and Paramsfromauror_core.v2.jobimportCommandfromauror_core.v2.paramsimportEnv,Paramsfromauror_coreimportProjectenv=Env(TESTE="my test",VARIABLE="my variable")params=Params(teste1="my test",variable="my variable")com=Command()\.with_command("bash echo $VARIABLE ${variable}")Project("folder_to_generate_files",com)\.is_v2()\.with_params(params,env)\.write()Join multiple variables in onefromauror_core.v2.jobimportCommandfromauror_core.v2.paramsimportEnvfromauror_coreimportProjectenv=Env(TESTE="env test",VARIABLE="env variable")params=Params(teste1="my test",variable="my variable")one_param=ParamsJoin("params_strange_name",",")## param name and separatorcom=Command()\.with_command("bash echo ${params_strange_name}")## it will print: my test,my variable,env test,env variable## THERE IS NO ORDER GUARANTEE, JUST Python 3.6 >Project("folder_to_generate_files",com)\.is_v2()\.with_params(one_param(params,env))\.write()Load jobs from YAML File (just for V2)You can find some YAML File examples onAzkaban Flow Documentationfromauror_core.v2.loaderimportLoaderloader=Loader('/path/to/file/flow.yaml')jobs=loader.as_job_objects()Or you can export these jobs as a Python Filefromauror_core.v2.loaderimportLoaderloader=Loader('/path/to/file/flow.yaml')jobs=loader.as_python_file('/path/to/desired/directory')# will be dumped with 'flow.py' nameDump memory flows to a Python File (just for V2)fromauror_core.v2.dumperimportDumpercom1=Job()\.with_name("commands job 1")\.with_(command="bash echo 1")com2=Job()\.with_name("commands job 2")\.with_(command="bash echo 2")dumper=Dumper('/path/to/desired/directory')# will be dumped with 'flow.py' namedumper.dump_jobs(com1,com2)PluginsPlugins are just extensions from auror_coreThere is a cookiecutter for new azkaban jobtypes with Auror template too:https://github.com/globocom/azkaban-jobtype-cookiecutterWe already have email plugin:https://github.com/globocom/azkaban-jobtype-emailContributeFor development and contributing, please followContributing Guideand ALWAYS respect theCode of Conduct