Package: experiment-utilities
Current version: 0.3.6 (09/11/2023)IntroductionExperiment Utilities (exputils) contains various tools for the management of scientific experiments and their experimental data.
It is especially designed to handle experimental repetitions, including to run different repetitions, to effectively store and load data for them, and to visualize their results.Main features:Easy definition of default configurations using nested python dictionaries.Setup of experimental configuration parameters using an ODF file.Running of experiments and their repetitions in parallel.Logging of experimental data (numpy, json).Loading and filtering of experimental data.Interactive Jupyter widgets to load, select and plot data as line, box and bar plots.RequirementsDeveloped and tested for Python 3.6 to 3.9 on Linux (Ubuntu 18.04).Depends on the following python packages which will be automatically installed during the setup:numpy >= 1.19.5six >= 1.15.0notebook <= 6.5.6  # exputils notebook widgets do not support new notebook version of 7ipywidgets >= 7.5.1,<= 7.6.5  # needs older version due tohttps://github.com/quantopian/qgrid/issues/372jupyter_contrib_nbextensions >= 0.7.0qgrid >= 1.3.1plotly >= 4.13.0cloudpickle >= 1.6.0dill >= 0.3.3odfpy >= 1.4.1tabulate >= 0.8.9scipy >= 1.5.4tensorboard >= 1.15.0fasteners >= 0.18pyyaml >= 6.0Setup1) ExputilsVia PIPpip install experiment-utilitiesFrom SourceClone the repository via git and install via pip:git clone git@gitlab.inria.fr:creinke/exputils.git .
pip install ./exputils(To install the library as a developer so that changes to its source code are directly usable in other projects:pip install -e ./exputils)2) Jupiter NotebookFor using the exputils GUIs for loading and plotting of data in Jupyter Notebook, theqgridwidget must be activated.
(Note: The GUI is currently only tested for Jupyter notebooks. For Jupyterlab, other installation procedures are necessary.)
Activateqgridwith:jupyter contrib nbextension install --user
jupyter nbextension enable --py --sys-prefix widgetsnbextension
jupyter nbextension enable --py --sys-prefix qgridIt is recommended to use theJupyter Notebooks Extensionsto allow folding of code and headlines.
This makes the notebooks more readable.
Activate the extensions with:jupyter nbextension enable codefolding/main
jupyter nbextension enable collapsible_headings/mainOverviewBesides the exputils library (the python package) the project also contains example code and unit tests.
It is recommended to look at these items to learn about the usage of the exputils components.The exputils package has the following structure:manage: Managing of experiments. Generation of code for experiments and repetitions from ODS configurations and source templates. Running of experiments and repetitions (can be used to run experiments on clusters.)data: Logging and loading of experimental data including filtering of data.gui: GUI components for Jupyter to load and plot experimental data.misc: Miscellaneous helper functions.io: Input-output functions to save and load data of various formats, including numpy and json.Experiments are stored in a specific folder structure which allows to save and load experimental data in a structured manner.
Please note that  it represents a default structure which can be adapted if required.
Elements in brackets (<custom name>) can have custom names.Folder structure:<main>folder: Holds several experimental campaigns. A campaign holds experiments of the same kind but with different parameters.analyzefolder: Scripts such as Jupyter notebooks to analyze the different experimental campaigns in this main-folder.<experimental campaign>folders:analyzefolder: Scripts such as Jupyter notebooks to analyze the different experiments in this experimental campaign.experiment_configurations.odsfile: ODS file that contains the configuration parameters of the different experiments in this campaign.srcfolder: Holds code templates of the experiments.<repetition code>folders: Code templates that are used under the repetition folders of th experiments. These contain the acutal experimental code that should be run.<experiment code>folders: Code templates that are used under the experiment folder of the experiment. These contain usually code to compute statistics over all repetitions of an experiment.generate_code.shfile: Script file that generates the experimental code under theexperimentsfolder using the configuration in theexperiment_configurations.odsfile and the code under thesrcfolder.experimentsfolder: Contains generated code for experiments and the collected experimental data.experiment_{id}folders:repetition_{id}folders:datafolder: Experimental data for the single repetitions, such as logs.code files: Generated code and resource files.datafolder: Experimental data for the whole experiment, e.g. statistics that are calculated over all repetitions.<code>files: Generated code and resource files.<run scripts>.shfiles: Various shell scripts to run experiments and calculate statistics locally or on clusters.Development TeamMainChris Reinke:chris.reinke@inria.frContributorsGaetan Lepage