Title: From Real to Synthetic: A Fine-grained Dataset and High-fidelity Biomechanical Model for Animal Behavior Understanding
Primary area: datasets and benchmarks
Keywords: Animal dataset, Biomechanical model, Synthetic data generation, Behavioral uncertainty quantification, Video understanding

Abstract: Rat behavior research contributes to the exploration of human disease mechanisms. However, existing datasets are scarce and cover limited behavior types, hindering the analysis and modeling of complex behavior patterns. We constructed ActionRat, a new multi-view rat behavior dataset that, for the first time, captures diverse actions during free exploration and brain-computer interface (BCI) control. It combines real and synthetic sequences with fine-grained keypoint annotations and atomic action sequences, supporting broader behavior analysis tasks. To efficiently generate synthetic data for dataset expansion, we developed OpenRatEngine, a high-fidelity 3D virtual biomechanical model. This model integrates anatomical priors from computed tomography (CT) scans, kinematic constraints, and lifelike appearance, reducing the domain gap between synthetic and real data. Equipped with pose control, OpenRatEngine generates synthetic sequences with accurate 3D keypoint annotations. We evaluated behavioral uncertainty quantification and animal pose estimation tasks on the ActionRat dataset, and demonstrated the outstanding synthetic data generation capability and realism of OpenRatEngine. Extensive experiments across deep learning models confirmed the effectiveness and value of both real and synthetic data.

TL;DR: A Fine-grained Benchmark Dataset and High-fidelity Biomechanical Model for Animal Behavior Understanding