Package: xanity
xanity is experiment sanityXanity is meant to allow easy and free creation of multiple experiments among which
there is shared python codebase and system environment.Xanity helps manage the following:experimetal environments ( via [ana/mini]conda )ensures experiments are run in the proper environmentcreates new environmentsupdates environments when new packages are neededexperiment and analysis source codearchives all source code at every run and keeps it with the datamaps associations between individual experiments and analysesexperiment parametersbuilds parameter sweepsruns experiments repeatedly with swept parameterscatalogs all sweep dataexperiment dataorganizes all experimental dataensures everything produced during runtime is properly catalogedlogsprovides easy logging interfacegenerates a master log filegenerates per-experiment log filesDependencies:conda - (from Anaconda or, my preference,Miniconda)[https://conda.io/miniconda.html]Usage:xanity commands generally assume they're being called from a $PWD inside a xanity
project directory tree.Some commands accept additional arguments to specify
the xanity project path:xanity init[ialize] [--with-examples] [new-dir]
  "    init               "             "

    Create a bare-bones xanity project directory tree in either the $PWD
    or a [new-dir].


xanity env setup

    Create or update the conda environment associated with the project.


xanity status

    Print the status of the current xanity project.


xanity run [experiment_names] [-a analyses[...]]

    Run all (or the specified) experiments and optionally, analyses.


xanity analyze [-a analyses] [run_data_path]
  "    analyse       "              "
  "    anal          "              "
  "    analysis      "              "

    Run all (or the specified) analyses on the most recent (or specified) data.


xanity session
  "     sess
  "     sesh

    Drops you into a new bash shell inside your project's environment.

xanity data
    suite of tools to manage data from previously run experimentsExample:install miniconda if you need it:wgethttps://repo.anaconda.com/miniconda/Miniconda2-latest-Linux-x86_64.sh
bashMiniconda2-latest-Linux-x86_64.shinstall xanity into your system's python(3) (or if you want, a conda env):pip[3]installxanityinitialize a xanity project and move inside with:xanityinit--with-examplesxanity_test_projcdxanity_test_projthis will create a skeleton directory tree for your experiments and analyses
which will be populated with some examples.Open xanity_test_proj/experiments/experiment1.py and have a look.  This is a
skeletal experiment.Open xanity_test_proj/analyses/analysis1.py and have a look.  This is a
skeletal analysis.find the conda-environment.yaml file and tweak its contents to suit your requiremnts.resolve these requirements (create/update a conda env) with:xanityenvsetuprun everything with:xanityrunyou will find all the experimental data organized under the data/runs directory tree.
Source-code snapshots are tarred and kept with the data they produced. Logs are kept too.you can run an analysis script on a completed run:xanity analyze experiment1this will look for the most recent (or specified) dir of run data, and run the analysis found
at analyses/myfaveexp on that data.relax. collect Nobel.Experiment file skeleton:(xanity-proj-root/experiments/*.py)Each experiment module must have a main() function defined:xanity will look for and invoke the main() function in each experiment.Any parameters to the experiment should be arguments to the main() function.Thexanity.experiment_parameters()call registers the parameter sweeps
to use when running the experiment.Include thexanity.run()function call.
- The run() hook will run the experiment if it's invoked directly as
a script or module:# experiments/example_experiment.pyimportxanityimportnumpyasnp# flag this experiment for analysisxanity.analyze_this()# register parameter sweeps you'd like to doxanity.experiment_parameters({'n_trials':[100,150,200],'train_frac':[0.9,0.5,0.1],'scale':[1,2,3,]})# parameters the experiment will acceptdefmain(n_trials=200,scale=5,main_frac=0.8):fakevar=scale*np.random.rand(n_trials)**2xanity.log("here is a print from experiment 1")xanity.save_variable(fakevar)if__name__=='__main__':xanity.run()Analysis file skeleton:(xanity-proj-root/analyses/*.py)Each analysis module must have a main() function defined.xanity will look for and invoke the main() function in each analysis.The only parameter to the analysis is the path to the root of a run (or set of runs).The call to xanity.associated_experiments() registers the names of experiments
to associate with this analysis.Include thexanity.run()function call.
- The run() hook will run the analysis if it's invoked directly as a script or module# analyses/example_analysis.pyimportxanityimportmatplotlib.pyplotasplt# define which experiments to associate this analysis withxanity.associated_experiments(['experiment1',#'experiment2',#'experiment3',])# the analysis takes a single argument... path of data (xanity will provide)defmain(data_dir):data,paths=xanity.load_variable('fakevar')plt.figure()fordindata:d.sort()plt.plot(d)plt.show()if__name__=='__main__':xanity.run()