Package: labtech
labtechGitHub-DocumentationLabtech makes it easy to define multi-step experiment pipelines and
run them with maximal parallelism and result caching:Defining tasks is simple; write a class with a singlerun()method and parameters as dataclass-style attributes.Flexible experiment configuration; simply create task objects
for all of your parameter permutations.Handles pipelines of tasks; any task parameter that is itself a
task will be executed first and make its result available to its
dependent task(s).Implicit parallelism; Labtech resolves task dependencies and
runs tasks in sub-processes with as much parallelism as possible.Implicit caching and loading of task results; configurable and
extensible options for how and where task results are cached.Integration withmlflow; Automatically
log task runs to mlflow with all of their parameters.Installationpip install labtechUsagefromtimeimportsleepimportlabtech# Decorate your task class with @labtech.task:@labtech.taskclassExperiment:# Each Experiment task instance will take `base` and `power` parameters:base:intpower:intdefrun(self):# Define the task's run() method to return the result of the experiment:labtech.logger.info(f'Raising{self.base}to the power of{self.power}')sleep(1)returnself.base**self.power# Configure Experiment parameter permutationsexperiments=[Experiment(base=base,power=power,)forbaseinrange(5)forpowerinrange(5)]# Configure a Lab to run the experiments:lab=labtech.Lab(# Specify a directory to cache results in (running the experiments a second# time will just load results from the cache!):storage='demo_lab',# Control the degree of parallelism:max_workers=5,)# Run the experiments!results=lab.run_tasks(experiments)print([results[experiment]forexperimentinexperiments])Labtech can also produce graphical progress bars inJupyterwhen theLabis initialized withnotebook=True:Tasks parameters can be any of the following types:Simple scalar types:str,bool,float,int,NoneCollections of any of these types:list,tuple,dict,EnumTask types: A task parameter is a "nested task" that will be
executed before its parent so that it may make use of the nested
result.Here's an example of defining a single long-running task to produce a
result for a large number of dependent tasks:fromtimeimportsleepimportlabtech@labtech.taskclassSlowTask:base:intdefrun(self):sleep(5)returnself.base**2@labtech.taskclassDependentTask:slow_task:SlowTaskmultiplier:intdefrun(self):returnself.multiplier*self.slow_task.resultsome_slow_task=SlowTask(base=42)dependent_tasks=[DependentTask(slow_task=some_slow_task,multiplier=multiplier,)formultiplierinrange(10)]lab=labtech.Lab(storage='demo_lab')results=lab.run_tasks(dependent_tasks)print([results[task]fortaskindependent_tasks])To learn more, see:Cookbook of common patterns(as an interactive notebook)The API reference for Labs and TasksMore options for cache formats and storage providersMore examplesMypy PluginFormypytype-checking of classes decorated
withlabtech.task, simply enable the labtech mypy plugin in yourmypy.inifile:[mypy]plugins=labtech.mypy_pluginContributingInstall Poetry dependencies withmake depsRun linting, mypy, and tests withmake checkDocumentation:Run local server:make docs-serveBuild docs:make docs-buildDeploy docs to GitHub Pages:make docs-githubDocstring style follows theGoogle style guideTODOAdd unit tests