Package: curifactory
CurifactoryCurifactory is a library and CLI tool designed to help organize and manage
research experiments in python.Experiment management must fulfill several tasks, including experiment orchestration,
parameterization, caching, reproducibility, reporting, and parallelization.
Existing projects such as MLFlow, MetaFlow, Luigi, and Pachyderm
support these tasks in several different ways and to various degrees.
Curifactory provides a different opinion, with a heavier focus on supporting general
research experiment workflows for individuals or small teams working primarily
in python.You can read more about these design principles in our paper in theSciPy 2022
proceedings.FeaturesAdds a CLI layer on top of your codebase, a single entrypoint for running experimentsAutomatic caching of intermediate data and lazy loading of stored objectsJupyter notebook output for further exploration of an experiment runDocker container output with copy of codebase, conda environment, full experiment run cache, and jupyter run notebookHTML report output from each run with graphviz-rendered diagram of experimentEasily report plots and values to HTML reportConfiguration files are python scripts, allowing programmatic definition, parameter composition, and parameter inheritanceOutput logs from every runRun experiments directly from CLI or other python code, notebooks, etc.InstallationpipinstallcurifactoryGraphviz is required for certain features and can be installed through conda
via:condainstallpython-graphvizTab completionFor tab-completion in bash/zsh, install theargcompletepackage (if using curifactory inside
a conda environment, you'll need to install this in your system python.)pipinstallargcompleteTo enable, you can either use argcomplete's global hookactivate-global-python-argcomplete, which
will enable tab complete on all argcomplete-enabled python packages (e.g. pytest), or you can addeval "$(register-python-argcomplete experiment)"to your shell's rc file. Curifactory can add
this line for you automatically with:curifactorycompletion[--bash|--zsh]# use the shell flag appropriateOnce enabled, theexperimentcommand will provide tab complete for experiment names, parameter names, and flags.RequirementsOS: We primarily develop and test Curifactory on Linux, but it runs on Windows and MacOS as well.Python: 3.9-3.11Optional:Conda/MambaGraphvizDockerJupyter notebook/labDocumentationThe documentation for the latest version of Curifactory can be found at:https://ornl.github.io/curifactory/latest/index.html.ExamplesSeveral small example can be found in theexamplesfolder.examples/notebooksincludes walkthroughs demonstrating usage of curifactory
solely in Jupyter.CitationPlease use the following BibTeX if citing this project:@article{Martindale2023,doi={10.21105/joss.05793},url={https://doi.org/10.21105/joss.05793},year={2023},publisher={The Open Journal},volume={8},number={90},pages={5793},author={Nathan Martindale and Scott L. Stewart and Jason Hite and Mark B. Adams},title={Curifactory: A research experiment manager},journal={Journal of Open Source Software}}Similar ProjectsCurifactory is one tool and one opinion among many, other projects that have similar goals and/or approaches:KedroTangoSacredMLFlow