DOI: 10.17617/3.3r0qyi
title: Supplementary data for the vassi Python package

description: This data repository provides supplementary data for&lt;br&gt;
&lt;br&gt;
"&lt;i&gt;vassi&lt;/i&gt; - verifiable, automated scoring of social interactions in animal groups"&lt;br&gt;
&lt;br&gt;
Documentation and example usage of the package are available online at &lt;a href="https://vassi.readthedocs.io/en/latest/"&gt;https://vassi.readthedocs.io/en/latest/&lt;a/&gt;. The source code is under version control at &lt;a href="https://vassi.readthedocs.io/en/latest/"&gt;https://github.com/pnuehrenberg/vassi/&lt;a/&gt; and also archived here (&lt;i&gt;vassi_source.zip&lt;/i&gt;).

&lt;h3&gt;1. &lt;i&gt;social cichlids&lt;/i&gt; dataset&lt;/h3&gt;

We tested our package on this novel dataset consisting of nine video recordings of groups of cichlid fish (15 &lt;i&gt;Neolamprologus multifasciatus&lt;/i&gt; per group). The dataset also contains individual trajectories for each fish (stored in a HDF5 file, can be loaded in Python as numpy arrays; posture data and corresponding time stamps) and behavioral annotations (CSV files, one behavioral event per row). Reencoded video files (compressed using FFMPEG) are available in &lt;i&gt;datasets/social_cichlids/videos&lt;/i&gt;.&lt;br&gt;
&lt;br&gt;
All scripts and notebooks from which results were presented in the paper used the same configuration for feature extraction (&lt;i&gt;examples/social_cichlids/features-cichlids.yaml&lt;/i&gt;). We provide intermediate results (&lt;i&gt;examples/social_cichlids/results.h5&lt;/i&gt; and &lt;i&gt;examples/social_cichlids/k_fold_predictions.h5&lt;/i&gt;) for the &lt;i&gt;examples/social_cichlids/results_and_figures-cichlids.ipynb&lt;/i&gt; notebook (available in GitHub repository or &lt;i&gt;vassi_source.zip&lt;/i&gt;. This notebook reproduces the figures as presented in our paper.&lt;br&gt;
&lt;br&gt;
We also provide the results obtained from hyperparameter optimization using the optuna framework in the same directory (&lt;i&gt;examples/social_cichlids/optimization/*&lt;/i&gt;).&lt;br&gt;
&lt;br&gt;
The results from k-fold prediction on the entire dataset (for visualization of networks as presented in the paper) are available in &lt;i&gt;examples/social_cichlids/k_fold_predictions_predictions.csv&lt;/i&gt;, which can be loaded as a dataset when complemented with the trajectories file (see &lt;i&gt;vassi_source.zip/examples/social_cichlids/results_and_figures-cichlids.ipynb&lt;/i&gt; for details).&lt;br&gt;
&lt;br&gt;
Our paper also presents a comparison between model predictions (behavior counts) and association time as an alternate behavioral proxy for interactions. The raw data files and the corresponding r script are available at &lt;i&gt;examples/social_cichlids/predictions_vs_association&lt;/i&gt;.

&lt;h3&gt;2. &lt;i&gt;CALMS21&lt;/i&gt; dataset&lt;/h3&gt;

In addition, we tested our package on an existing benchmark dataset, the CALMS21 mouse resident-intruder dataset. For convenience, we provide Python scripts to download or convert the original dataset (&lt;i&gt;vassi_source.zip/src/vassi/case_studies/calms21/download.py&lt;/i&gt; and &lt;i&gt;vassi_source.zip/src/vassi/case_studies/calms21/convert.py&lt;/i&gt;; or available after &lt;i&gt;vassi&lt;/i&gt; was installed, see online documentation for more details).&lt;br&gt;
&lt;br&gt;
The original CALMS21 dataset can be downloaded here:&lt;br&gt;
&lt;a href="https://data.caltech.edu/records/s0vdx-0k302"&gt;https://data.caltech.edu/records/s0vdx-0k302&lt;/a&gt;&lt;br&gt;
&lt;br&gt;
&lt;ul&gt;
 &lt;li&gt;[Dataset] Jennifer J. Sun, Tomomi Karigo, David J. Anderson, Pietro Perona, Yisong Yue, & Ann Kennedy. (2021). Caltech Mouse Social Interactions (CalMS21) Dataset (1.0) [Data set]. CaltechDATA. &lt;a href="https://doi.org/10.22002/D1.1991"&gt;https://doi.org/10.22002/D1.1991&lt;/a&gt;&lt;/li&gt;
 &lt;li&gt;[Paper] Sun JJ, Karigo T, Chakraborty D, Mohanty SP, Wild B, Sun Q, Chen C, Anderson DJ, Perona P, Yue Y, Kennedy A. The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions. Adv Neural Inf Process Syst. 2021 Dec;2021(DB1):1-15. PMID: 38706835; PMCID: PMC11067713.&lt;/li&gt;
&lt;/ul&gt;
All scripts and notebooks from which results were presented in the paper used the same configuration for feature extraction (&lt;i&gt;examples/CALMS21/features-mice.yaml&lt;/i&gt;). As for the other example dataset, we provide intermediate results (&lt;i&gt;examples/CALMS21/results.h5&lt;/i&gt;) for the &lt;i&gt;vassi_source.zip/examples/CALMS21/results_and_figures-mice.ipynb&lt;/i&gt; notebook to reproduce the figures presented in our paper.&lt;br&gt;
&lt;br&gt;
We also provide the results obtained from hyperparameter optimization using the optuna framework in the same directory (&lt;i&gt;examples/CALMS21/optimization/*&lt;/i&gt;).
&lt;h3&gt;Files&lt;/h3&gt;
&lt;pre&gt;
datasets/
└── social_cichlids/
 ├── videos/
 │ ├── GH010423.MP4
 │ ├── GH010861.MP4
 │ ├── GH013974.MP4
 │ ├── GH019910.MP4
 │ ├── GH030423.MP4
 │ ├── GH030451.MP4
 │ ├── GH030861.MP4
 │ ├── GH039910.MP4
 │ └── GH039931.MP4
 ├── cichlids_annotations.csv
 └── cichlids_trajectories.h5
examples/
└── CALMS21/
 ├── optimization/
 │ ├── optimization-results.yaml
 │ ├── optimization-summary.yaml
 │ └── optimization-trials.csv
 ├── features-mice.yaml
 └── results.h5
└── social_cichlids/
 ├── optimization/
 │ ├── optimization-results.yaml
 │ ├── optimization-summary.yaml
 │ └── optimization-trials.csv
 ├── predictions_vs_association/
 │ ├── aggregated_counts-1bl.csv
 │ ├── aggregated_counts-3bl.csv
 │ ├── aggregated_counts-5bl.csv
 │ └── predictions_vs_association.Rmd
 ├── features-mice.yaml
 ├── results.h5
 └── k_fold_predictions_predictions.csv
└── vassi_source.zip
&lt;/pre&gt;