Package: doit-interface
This package provides a functional interface for reducing boilerplate indodo.pyof thepydoitbuild system. In short, all tasks are created and managed using a.Manager. Mostfeatures<features>are exposed using python context manager, e.g., grouping tasks.Basic usage>>>importdoit_interfaceasdi>>># Get a default manager (or create your own to use as a context manager).>>>manager=di.Manager.get_instance()>>># Create a single task.>>>manager(basename="create_foo",actions=["touch foo"],targets=["foo"]){'basename':'create_foo','actions':['touch foo'],'targets':['foo'],...}>>># Group multiple tasks.>>>withdi.group_tasks("my_group")asmy_group:...member=manager(basename="member")>>>my_group<doit_interface.contexts.group_tasksobjectat0x...>named`my_group`with1taskNoteThe default manager obtained by calling.Manager.get_instancehas a number of default contexts enabled:.SubprocessAction.use_as_defaultto use.SubprocessActionby default for string actions..create_target_dirsto create target directories if they are missing..normalize_dependenciessuch that task objects can be used as file and task dependencies.It also injects a defaultDOIT_CONFIGconfiguration variable if the filename isdodo.py.If you want to override this default behavior, you can create a dedicated manager and call.Manager.set_default_instanceor modify the.Manager.context_stackof the default manager.FeaturesTraceback for failed tasksThe.DoitInterfaceReporterprovides more verbose progress reports and points you to the location where a failing task was defined. TheDOIT_CONFIGis used by default if you use.Manager.get_instanceto get a.Manager.>>>DOIT_CONFIG={"reporter":DoitInterfaceReporter}>>>manager(basename="false",actions=["false"]){'basename':'false','actions':['false'],'meta':{'filename':'...','lineno':1}}$doitEXECUTE:falseFAILED:false(declaredat...:1)...Group tasksGroup tasks to easily execute all of them using.group_tasks. Tasks can be added to groups using a context manager (as shown below) or by calling the group to add an existing task. Groups can be nested arbitrarily.>>>withgroup_tasks("vgg16")asvgg16:...train=manager(basename="train",actions=[...])...validate=manager(basename="validate",actions=[...])>>>vgg16<doit_interface.contexts.group_tasksobjectat0x...>named`vgg16`with2tasksAutomatically create target directoriesUse.create_target_dirsto automatically create directories for each of your targets. This can be particularly useful if you generate nested data structures, e.g., for machine learning results based on different architectures, seeds, optimizers, learning rates, etc.>>>withcreate_target_dirs():...task=manager(basename="bar",targets=["foo/bar"],actions=[...])>>>task["actions"][(<functioncreate_folderat0x...>,['foo']),...]Share default values across tasksUse.defaultsto share default values across tasks, such asfile_dep.>>>withdefaults(file_dep=["data.pt"]):...train=manager(basename="train",actions=[...])...validate=manager(basename="validate",actions=[...])>>>train["file_dep"]['data.pt']>>>validate["file_dep"]['data.pt']Use tasks asfile_deportask_dep.normalize_dependenciesnormalizes file and task dependencies such that task objects can be used as dependencies (in addition file and task names).>>>withnormalize_dependencies():...base_task=manager(basename="base",name="output",targets=["output.txt"])...file_dep_task=manager(basename="file_dep_task",file_dep=[base_task])...task_dep_task=manager(basename="task_dep_task",task_dep=[base_task])>>>file_dep_task["file_dep"]['output.txt']>>>task_dep_task["task_dep"]['base:output']Add prefixes to paths or other attributesPath prefixes can be added using the.path_prefixcontext if file dependencies or targets share common directories. General prefixes are also available using.prefix.>>>withpath_prefix(targets="outputs",file_dep="inputs"):...manager(basename="task",targets=["out.txt"],file_dep=["in1.txt","in2.txt"]){'basename':'task','targets':['outputs/out.txt'],'file_dep':['inputs/in1.txt','inputs/in2.txt'],...}Subprocess actionThe.SubprocessActionlets you spawn subprocesses akin todoit.action.CmdActionyet with a few small differences. First, it does not capture output of the subprocess which is helpful for development but may add too much noise for deployment. Second, it supportsMakefilestyle variable substitutions and f-string substitutions for any attribute of the parent task. Third, it allows for global environment variables to be set that are shared across all, e.g., to limit the number ofOpenMPthreads. You can use it by default for string-actions using the.SubprocessAction.use_as_defaultcontext.Interfacedoit_interface
 :members: