SlowGuess/NeurIPS_2023_OCR huggingface dataset:
LFB (w.NL L
SlowOnly (w.Ctx)
SlowFast (w.Ctx)
GT
Figure A8: Qualitative results of representative methods on the ChimpACT test set on the spatiotemporal action detection task. The ground-truth actions are shown in the last row.
Table A8: Results of spatiotemporal action detection track on ChimpACT test set.
Method	mAP	moving	climbing	sol. obj.	playing	eating	grooming	playing	being	begged from	aggressing	being nursed
ACRN (Sun et al., 2018)	24.4	60.2	23.2	38.2	54.3	7.7	42.9	0.0	0.0	0.0	4.4
LFB (Wu et al., 2019)	22.4	45.3	10.0	34.4	56.3	8.7	51.0	0.4	0.0	0.0	32.1
SlowOnly (Feichtenhofer et al., 2019)	24.5	56.1	31.6	41.0	45.4	10.4	43.0	0.0	0.0	0.0	7.5
SlowFast (Feichtenhofer et al., 2019)	24.5	60.9	37.2	47.3	35.3	10.4	49.2	0.0	0.0	0.0	7.5
Figure A9: More qualitative results of representative methods on the ChimpACT test set on the spatiotemporal action detection task. The ground-truth actions are shown in the last row.
D Data documentation
We follow the datasheet proposed in Gebru et al. (2021) for documenting our ChimpACT and associated benchmarks:
1. Motivation
(a) For what purpose was the dataset created?
This dataset was created to facilitate the study of chimpanzee behaviors, and ultimately advance our understanding of communication and sociality in non-human primates.
(b) Who created the dataset and on behalf of which entity?
This dataset was created by Xiaoxuan Ma, Stephan P. Kaufhold, Jiajun Su, Wentao Zhu, Jack Terwilliger, Andres Meza, Yixin Zhu, Federico Rossano, and Yizhou Wang. Xiaoxuan Ma, Jiajun Su, Wentao Zhu, Yixin Zhu, and Yizhou Wang are with Peking University. Stephan P. Kaufhold, Jack Terwilliger, Andres Meza, and Federico Rossano are with the University of California, San Diego.
(c) Who funded the creation of the dataset?
The creation of this dataset was funded by Peking University and the University of California, San Diego.
(d) Any other Comments? None.
2. Composition
(a) What do the instances that comprise the dataset represent?
For video data, each instance is a video clip regularized from the raw video. Each instance contains video footage focusing on a group of chimpanzees collected in Leipzig Zoo, Germany. For benchmarking, each instance has rich annotations of chimpanzee identities, poses, and actions. See Sec. 3 and Appx. A.
(b) How many instances are there in total?
We have 163 video instances in total.
(c) Does the dataset contain all possible instances or is it a sample (not necessarily random) of instances from a larger set?
No, this is a brand-new dataset.
(d) What data does each instance consist of?
See Sec. 3 and Appx. A.
(e) Is there a label or target associated with each instance?
Yes. See Sec. 3 and Appx. A.
(f) Is any information missing from individual instances?
(g) Are relationships between individual instances made explicit?
(h) Are there recommended data splits?
Yes, we have separated the whole dataset into training, validation, and test set. See Sec. 4.1, Appx. C and the project website for details.
(i) Are there any errors, sources of noise, or redundancies in the dataset?
There are almost certainly some errors in video annotations. We did our best to minimize these, but some certainly remain.
(j) Is the dataset self-contained, or does it link to or otherwise rely on external resources (e.g., websites, tweets, other datasets)?
The dataset is self-contained.
(k) Does the dataset contain data that might be considered confidential (e.g., data that is protected by legal privilege or by doctor-patient confidentiality, data that includes the content of individuals' non-public communications)?
(1) Does the dataset contain data that, if viewed directly, might be offensive, insulting, threatening, or might otherwise cause anxiety?
(m) Does the dataset relate to people?
(n) Does the dataset identify any subpopulations (e.g., by age, gender)?
(o) Is it possible to identify individuals (i.e., one or more natural persons), either directly or indirectly (i.e., in combination with other data) from the dataset?
Not applicable. Our dataset only contains chimpanzees.
(p) Does the dataset contain data that might be considered sensitive in any way (e.g., data that reveals racial or ethnic origins, sexual orientations, religious beliefs, political opinions or union memberships, or locations; financial or health data; biometric or genetic data; forms of government identification, such as social security numbers; criminal history)?
(q) Any other comments?
3. Collection Process
(a) How was the data associated with each instance acquired?
See Sec. 3.2 and Appx. A for details.
(b) What mechanisms or procedures were used to collect the data (e.g., hardware apparatus or sensor, manual human curation, software program, software API)?
We used JVC Everio cameras to collect video footage (Codec H.264). See Sec. 3.2 for details.
(c) If the dataset is a sample from a larger set, what was the sampling strategy (e.g., deterministic, probabilistic with specific sampling probabilities)?
See Sec. 3.3 and Appx. A for details.
(d) Who was involved in the data collection process (e.g., students, crowdworkers, contractors) and how were they compensated (e.g., how much were crowdworkers paid)?
The video data was collected by the authors. The annotations were performed by the workers in BasicFinder CO., Ltd., and the workers were offered a fair wage as per the prearranged contract. See Sec. 3 and Appx. B for details.
(e) Over what timeframe was the data collected?
The data were collected from 2015 to 2018, and labeled in 2022.
(f) Were any ethical review processes conducted (e.g., by an institutional review board)?
Not applicable. The ChimpACT dataset raises no ethical concerns regarding the privacy information of human subjects, as it solely focuses on chimpanzees.
(g) Does the dataset relate to people?
(h) Did you collect the data from the individuals in question directly, or obtain it via third parties or other sources (e.g., websites)?
(i) Were the individuals in question notified about the data collection?
(j) Did the individuals in question consent to the collection and use of their data?
(k) If consent was obtained, were the consenting individuals provided with a mechanism to revoke their consent in the future or for certain uses?
(1) Has an analysis of the potential impact of the dataset and its use on data subjects (e.g., a data protection impact analysis) been conducted?
Yes, see Appx. B.
(m) Any other comments?
4. Preprocessing, Cleaning and Labeling
(a) Was any preprocessing/cleaning/labeling of the data done (e.g., discretization or bucketing, tokenization, part-of-speech tagging, SIFT feature extraction, removal of instances, processing of missing values)?
Yes, see Sec. 3.
(b) Was the "raw" data saved in addition to the preprocessed/cleaned/labeled data (e.g., to support unanticipated future uses)?
Yes, we provide the raw data on our project website.
(c) Is the software used to preprocess/clean/label the instances available?
No. The annotation software is the private labeling platform provided by BasicFinder CO., Ltd. However, existing open-source annotation software such as DeepLabCut (Mathis et al., 2018) could also be used to preprocess/clean/label the instances.
(d) Any other comments?
5. Uses
(a) Has the dataset been used for any tasks already?
No, the dataset is newly proposed by us.
(b) Is there a repository that links to any or all papers or systems that use the dataset?
Yes, we provide the link to all related information on our project website.
(c) What (other) tasks could the dataset be used for?
This dataset could be used for other research topics, including but not limited to pose tracking, few-shot learning, and transfer learning.
(d) Is there anything about the composition of the dataset or the way it was collected and preprocessed/cleaned/labeled that might impact future uses?
We propose to annotate the keyframe every 10 frames for the pose track and action detection track. For tracking track, we label all the frames.
(e) Are there tasks for which the dataset should not be used?
The usage of this dataset should be limited to the scope of understanding chimpanzee/non-human primate behaviors.
(f) Any other comments?
6. Distribution
(a) Will the dataset be distributed to third parties outside of the entity (e.g., company, institution, organization) on behalf of which the dataset was created?
Yes, the dataset will be made publicly available.
(b) How will the dataset be distributed (e.g., tarball on website, API, GitHub)?
The dataset could be accessed on our project website.
(c) When will the dataset be distributed?
The dataset will be released by the end of 2023 on our project website.
(d) Will the dataset be distributed under a copyright or other intellectual property (IP) license, and/or under applicable terms of use (ToU)?
We release our benchmark under CC BY-NC 4.0 license.
(e) Have any third parties imposed IP-based or other restrictions on the data associated with the instances?
(f) Do any export controls or other regulatory restrictions apply to the dataset or to individual instances?
(g) Any other comments?
7. Maintenance
(a) Who is supporting/hosting/maintaining the dataset?
Xiaoxuan Ma is maintaining.
<https://paperswithcode.com/datasets/licenses>
(b) How can the owner/curator/manager of the dataset be contacted (e.g., email address)? maxiaoxuan@pku.edu.cn
(c) Is there an erratum?
Currently, no. As errors are encountered, future versions of the dataset may be released and updated on our website.
(d) Will the dataset be updated (e.g., to correct labeling errors, add new instances, delete instances')?
Yes, if applicable.
(e) If the dataset relates to people, are there applicable limits on the retention of the data associated with the instances (e.g., were individuals in question told that their data would be retained for a fixed period of time and then deleted)?
Not applicable. The dataset does not relate to people.
(f) Will older versions of the dataset continue to be supported/hosted/maintained?
Yes, older versions of the benchmark will be maintained on our website.
(g) If others want to extend/augment/build on/contribute to the dataset, is there a mechanism for them to do so?
Yes, please get in touch with us by email.
(h) Any other comments?