Pose Splatter: A 3D Gaussian Splatting Model for Quantifying Animal Pose and Appearance
Authors: Jack Goffinet
Youngjo Min
Carlo Tomasi
David Carlson
Venue: NeurIPS 2025 poster ()
Keywords: animal pose
gaussian splatting
neuroscience
Primary area: neuroscience_and_cognitive_science

Abstract: Accurate and scalable quantification of animal pose and appearance is crucial for studying behavior. Current 3D pose estimation techniques, such as keypoint- and mesh-based techniques, often face challenges including limited representational detail, labor-intensive annotation requirements, and expensive per-frame optimization. These limitations hinder the study of subtle movements and can make large-scale analyses impractical. We propose *Pose Splatter*, a novel framework leveraging shape carving and 3D Gaussian splatting to model the complete pose and appearance of laboratory animals without prior knowledge of animal geometry, per-frame optimization, or manual annotations.
  We also propose a rotation-invariant visual embedding technique for encoding pose and appearance, designed to be a plug-in replacement for 3D keypoint data in downstream behavioral analyses.
  Experiments on datasets of mice, rats, and zebra finches show *Pose Splatter* learns accurate 3D animal geometries. Notably, *Pose Splatter* represents subtle variations in pose, provides better low-dimensional pose embeddings over state-of-the-art as evaluated by humans, and generalizes to unseen data.
  By eliminating annotation and per-frame optimization bottlenecks, *Pose Splatter* enables analysis of large-scale, longitudinal behavior needed to map genotype, neural activity, and behavior at high resolutions.

Pose Splatter: A 3D Gaussian Splatting Model for Quantifying Animal Pose and Appearance
Jack Goffinet .1, Youngjo Min Carlo Tomasi', David E. Carlson 1,2,3
Department of Computer Science
Department of Biostatistics and Bioinformatics
Department of Civil and Environmental Engineering Duke University
Durham, NC 27705
{jack fack.goffinet, youngjo.ming min, tomasi,
@duke, edu,
Abstract
Accurate and scalable quantification of animal pose and appearance crucial for studying behavior. Current 3D pose estimation techniques, such as keypoint- and mesh-based techniques, often face challenges including limited representational de- tail, labor-intensive annotation requirements, and expensive per-frame optimization. These limitations hinder the study of subtle movements and can make large-scale analyses impractical. We propose Pose Splatter, a novel framework leveraging shape carving and 3D Gaussian splatting to model the complete pose and appear- ance of laboratory animals without prior knowledge of animal geometry, per-frame optimization, or manual annotations. We also propose a rotation-invariant visual embedding technique for encoding pose and appearance, designed to be a plug-in in replacement for 3D keypoint data in downstream bchavioral analyses. Experiments datasets of mice, rats, and zebra finches show Pose Splatter learns accurate 3D animal geometries. Notably, Pose Splatter represents subtle variations in pose, provides better low-dimensional pose embeddings over state-of-the-art as evalu- ated by humans, and generalizes to unseen data. By eliminating annotation and per-frame optimization bottlenecks, Pose Splatter enables analysis of large-scale, longitudinal behavior needed to map genotype, neural activity, and behavior at high resolutions.
1 Introduction
The study of animal behavior is central focus in neuroscience research, as it provides essential context for understanding neural and physiological processes. In particular, accurate capture of 3D pose allows researchers to study important elements of animal behavior including walking, balance, and interaction with the environment (38). These elements are essential for detecting small behavioral changes associated with neurological diseases or therapies [62]
Deep learning methods originally developed for 3D human pose estimation from images have inspired and driven dramatic advances in 3D animal shape and pose reconstruction as well These methods are most often based on either keypoints or meshes.
46
21 83
45 25
are straightforward to implement and relatively efficient computationally. They triangulate set of anatomical points (e.g., joints, wing edges, tail tips) across multiple camera views to reconstruct their 3D positions. However, each keypoint must be
Keypoint- based methods [45
25 12 1
"These authors contributed equally to this work.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Grissin
Panunnes
Shape Carved Volume
6.View Capture
Mask Inference
Stacked U.Net
Rendered View
a
(b)
Figure I: (a) Pose Splatter pipeline. Multi-view images and their corresponding masks carved into a coarse voxel shape, which a stacked U-Net converts into de-voxelized 3D Gaussian parameters that are finally rendered through Gaussian splatting. The entire process runs in only 2.5 GB of GPU memory (VRAM) compared to 10-20 GB for competing methods. (b) Shape-carving concept. Silhouettes from each camera back- pack-projected into shared voxel grid (yellow cones), removing voxels outside the visual hull. The green intersection marks the rough volumetric prior fed to the network.
located accurately for training, leading to labor-intensive annotation. Additionally, these landmarks are too sparse to capture the body's 's full geometry, let alone the color and texture of its surface.
do reconstruct the animal's complete surface geometry by fitting a parameterized 3D model to observed data, and thereby facilitate in- depth analyses of body shapes and deformations. However, inferring this richer information requires specialized and time-intensive mesh fess-fitting routines for each imput frame. Inference often also depends on an accurate template model (e.g.. SMAL [83) and may fail to generalize to postures that are not well-represented by the original template, or to altogether different species (e.g., mice, rats).
Mesh-based methods 83
82
66
80
33
54
To address these challenges, we propose Pose Splatter, a feed-forward model based on 3D Gaussian splatting (3DGS) that reconstructs 3D shapes of different animal species accurately and in scalable way. 3DGS 26 is a 3D scene rendering technique that has recently become popular due to its extremely fast rendering times and high visual fidelity. represents scene collection of Gaussian particles, each with its own position, covariance, and color parameters. While earlier models optimized these parameters on a seene-by-scene basis, recent efforts have introduced feed- forward variants [T0 [1] designed to perform single-step inference after training. However, unlike Pose Splatter, these methods assume ample inter-view overlap and therefore struggle with sparse-view 3D animal reconstruction, where such overlap is minimal.
Pose Splatter captures each scene with a small set of calibrated cameras (4-6 in this work) and generates foreground masks for each view using SAM2 [5I]. A shape-carving process [31] 30] back- sack-projects these multi-view masks into 3D visual cones, whose intersection yields rough estimate of the voxels inside the animal's shape. A stacked 3D U-Net then refines this coarse volume, and compact multi-layer perceptron (MLP) processes the voxel-level features to produce the parameters for the 3D Gaussian splats, which are rendered with 3DGS. The network is trained end-to-end to minimize image-based losses. We additionally introduce a visual embedding technique that relies on the ability to render the scene from novel viewpoints and provides informative low-dimensional descriptor pose and appearance for use in downstream analyses.
Our framework addresses several limitations of existing techniques. Unlike keypoint-based ap- proaches, which use a sparse representation of body and require extensive landmark annotation, our framework recovers the complete 3D posture of the animal without any manual labeling, In contrast mesh-based methods, which often demand per-frame optimization and accurate template models, our network performs inference via single forward pass and requires no species-specific templates. Our quantitative and qualitative experiments demonstrate that our model achieves robust 3D reconstructions and novel view renderings of multiple animal species despite its simplicity. Our experiments show that the proposed visual embedding captures subtle variations in animal pose and serves useful descriptor for behavioral analysis. By removing manual annotation and computational bottlenecks, our approach opens the door large-scale and high resolution behavioral analysis, facilitating deeper understanding of the genetic and neural underpinnings of behavior.

---

2 Related Work
Keypoint-basee Pose Estimation Keypoint-based pose estimation research first concentrated on 2D)
keypoint estimate for human pose [8]
49
74
16
55
56
58 77]. where models aimed to localize
joints in 2D images to infer human poses. The field moved toward 3D keypoint estimation [41 52 3
29 6 21 50J. using multi-view information to represent human poses in more realistic, spatially aware manner. This evolution was mirrored in animal pose estimation, with early studies primarily lackling 2D keypoint estimation [47
19
42
32
73 40
[3] and more recent studies shifting to 3D
12]. alliging for more accurate representation of animal space. However, these models capture only a sparse representation and depend on manually annotated training data. Pose Splatter, by contrast, reconstructs the animal's complete 3D geometry and requires no manual labels.
keypoint estimation (45, 2
76
75
poses in three-dimensionn
Mesh-based Pose Estimation Mesh-based 3D pose estimation provides a more complete rep- resentation of body shape and surface characteristics compared to keypoint methods by fitting a parametric 3D model to observed data. Mesh models have been extensively applied in human pose
estimation 34 20 24 28 13] to capture fine-grained details, such as muscle contours and body surface deformations, thereby enabling them especially in applications requiring great accuracy, like
biomechanics and animation. Mesh-based techniques [83] 81 71 69 66 5 59 64| [54] have recently been used in animal pose estimation, providing more detailed representation of the diverse anatomies and movements of different species compared to keypoint methods. Unlike mesh methods, Pose Splatter does not require species-specific template or per-frame optimization routines.
3D Gaussian Splatting (3DGS) 3DGS [26) has emerged as a powerful technique for novel view synthesis and 3D reconstruction due to its exceptional rendering speed and quality. Most initial approaches [26 35
68 relied on per-scene optimization routines and abundant multi-view images, which can limit broader applicability. Recently, researchers have begun to investigate feed-forward pipelines (10, [T] that, once trained, can generate novel views from sparse input images in a single inference step. However, to the best of our knowledge, there is currently no 3DGS framework specifically tailored for sparse-view 3D animal reconstruction.
15
63
36
3 Method
At high level, Pose Splatter uses masked images captured from a small number of calibrated cameras. We apply shape carving techniques to create a voxelized "rough" representation of single animal's pose and appearance. Then the rough volume passes through a stacked 3D U- Net architecture to produce a "clean" volume with an occupancy channel and additional feature channels. The occupancy channel determines whether to render a Gaussian for each voxel, while the remaining feature channels for each rendered voxel are independently mapped through a small MLP to determine 3D displacement vectors, which de-voxelizes the representation, along with covariance and appearance features needed to render the Gaussian. Finally, the scene is rendered given camera parameters by Gaussian splatting. Image-based losses are then used to propagate gradients through the model parameters. The model a simple, lightweight framework which uses only about 2.5 GB of GPU memory (VRAM). Note that Pose Splatter captures animal's pose at given instant not in the sense that it estimates joint angles or positions, but the more general sense of capturing the whole geometry of the animal. The overall framework is illustrated in Figure Tp. See Appendix A for a brief discussion of camera considerations.
Mask Generation We generate mask videos used for training using a pre-trained Segment Anything Model (SAM2) (5T). We first prompted mask on the first frame using SAM2 image mode, and propagate the mask through the video using SAM2 video mode. We chose not to fine-tune SAM2 on our video datasets to how well our pipeline could perform without manual annotation, but it is also possible to fine-tune the model, which would likely improve the results.
Determining Animal Position and Rotation The model quantifies animal pose independent of the 3D position and azimuthal orientation (about the vertical axis) of the animal. We determine these quantities without relying on 3D keypoints, thereby avoiding the time-intensive process of creating a training dataset for a keypoint detection network. To this end, a robust triangulation of the center coordinates of the mask in each image yields rough 3D center. Shape carving, described below, provides a rough estimate of the animal mass, which is summarized as a 3D Gaussian distribution with mean vector and covariance matrix. The mean vector is taken as the animal's 's position, while
3

---

the principal axis of the 3D covariance matrix is tracked smoothly over time and projected onto the X/Y plane to estimate the horizontal rotation. Additional details can be found in Appendix [
Shape Carving Procedure Shape carving centers a 3D cubic grid of voxels at the estimated animal center and rotates it according the estimated azimuthal rotation angle. By back-projecting masks from each camera view into space, we determine which voxels lie outside the visual hull of the animal and which ones are inside. We assign colors the inside voxels based on their visibility from each view. This approach assumes minimal occlusion and relies on precise camera calibration to ensure accurate alignment of projections. In particular, this precludes a direct application multi-animal recordings, where occlusions are common. Figure D provides an intuitive illustration of the shape carving process. See Appendix C for more details on our shape carving procedure.
Stacked U-Net Architecture The stacked 3D U-Net architecture consists of three U-Net modules arranged sequentially, designed to progressively refine the rough volume. This stacked configuration, inspired by the refinement capability of stacked hourglass networks, allows for iterative enhancement of feature quality and spatial coherence. Each module contains four downsampling and upsampling blocks, with skip connections to facilitate feature preservation. All layers employ ReLU activations, and the last U-Net module outputs 8 channels. The U-Nets are initialized approximate the identity function by initializing filters near the Dirac delta filter and relying on the first skip connection to propagate the image through the U-Net. We find that no pretraining is needed with this initialization, unlike standard initialization schemes.
Gaussian Splatting To render the animal using Gaussian splatting, we first decide which Gaussians to render, interpreting the first channel of the volume as a probability of rendering a Gaussian for a given voxel. For each rendered voxel, we pass all 8 channels through small MIP to produce Gaussian parameters. Crucially, the mean parameter is taken by adding the M..--predicted displacement to the coordinate of the voxel in 3D space, thereby de- de-voxelizing the animal shape representation. standard splatting is then performed, taking Gaussian and camera parameters and outputting a rendered RGBA image. We use the gsplat library (72) to perform splatting. In standard splatting, each Gaussian particle described by a location parameter ju E IR and spatial covariance matrix Σ E 3 in addition to color and opacity parameters, density of the Gaussian particle at location x is given by
R3 and a E [0, 1], respectively. The
G(x) = exp (-3 xx
As described in [84]. using camera transformation matrix W and the Jacobian matrix J of an affine approximation to the projective transformation, the resulting 2D covariance matrix is given by >' JWEWT JT. Lastly, the color at a given pixel in the image plane is given by
MN c (1 (1-a;)
where the particles are ordered by decreasing depth (along the j subscript) relative to the camera.
Loss Terms We employ two standard image-based losses to train the network. First, we calculate an L1 color loss to encourage accurate color rendering. as LI losses are more robust to changes in colors caused by non-Lambertian effects than L2 losses: Leotor Eg -2. Fzyl 3 En, may where f is the predicted image, is the ground truth image with the rendering background color, outside of the masked region, modificd to be white, and m is the input mask. Sccond, we use an intersection- over-union loss to compare an input mask m to the transparency channel of the rendered image in to encourage accurate silhouettes: LioU = (ff()))))))) Eiiii///E,(( Ei,(m + m - fmm); where products are elementwise. The total loss = LIoU + Ccotor where Acolor is a hyperparameter.
3.1 A Visual Embedding Technique
In addition to producing collections of Gaussian particles that capture the 3D geometry and appearance of the animal, it is often useful distill these particles into moderate-dimennsonnl descriptor of pose and appearance for subsequent behavioral analyses. For example, 2D and 3D keypoint coordinates in
an egocentric reference frame have greatly advanced the study of animal behavior [61 27]. Designing a comparable descriptor directly from the Gaussian particle parameters is complicated by two factors. First, the number of particles can vary from frame to frame. Second, Gaussians that are completely inside the surface layer of the animal volume do not materially affect appearance. To bypass these issues, we propose method that relies on rendered appearance of the particles rather than on the raw particle parameters while maintaining invariance to animal position and azimuthal rotation.

---

(a)
why M a maw
0.6
PC3
0.4
Query
KP
NN
VE Win Rate
Time (51
(b)
(c)
Figure 2: (a) Example renderings for visual embedding. 32 virtual cameras, distributed on a sphere
centered the animal, produce appearance-only renderings used to build visual embedding. (b)
Nearest-neighbor preference study. Nearest-neighhor retrieval with the visual embedding (VE) is favored over 3D-keypoint (KP) baseline in 102-person study (54 % vs. 50 %; p 1.5 10-5,
two-sided (-test, = 40). The query pose (left) and its two candidate nearest neighbors illustrate that the visual embedding preserves subtle leftward head tilt that the keypoint method misses. (c) Visual embedding tracks subtle movements. Two image rows are shown-the upper row contains ground-trutt frames and the lower row the corresponding Pose Splatter renders, illustrating from left to right a typical pose, slight feather expansion, and a head-shaking bout. Beneath them, the first five principal components (PCI-PC5) plotted through time reveal these behaviors: thin grey lines indicate head reorientations that coincide with changes in PCs dark-grey bands mark brief head-shaking bouts that stand in PCs 5, and the surrounding light-blue interval captures slow feather expansion and compression, reflected in the low-frequency trends of PCs and
Overview of the Embedding We place a virtual camera at a set of viewpoints covering the sphere centered on the animal's 3D center. At each viewpoint (0, Φ), where [0, ক| the polar angle from the positive z-axis down and á E [0, 2n] is the azimuthal angle about the z-axis, we render 224 x 224 RGB image of the Gaussian particles as seen looking inward toward the animal (Figure[p)) Because internal Gaussians occluded in all views, the resulting set of images captures only the visually relevant features (shape, silhouette, color, etc.).
Latent Encoding Rather than working directly with the RGB images, each 224 x 224 rendering is passed through a pretrained convolutional encoder (ResNet-18) that outputs 512-dimensional feature vector. Denote this resulting function on sphere by /(0, Φ) E R-12 Each component Ja (0,0)) +) (for l = 1,...,512) encodes some learned feature of appearance or geometry across the sphere.
Spherical Harmonic Expansion and Quadrature To produce a rotation-invarian descriptor, we expand each component fü in truncated spherical harmonic basis Yem (0, Φ) of bandwidth L,
o2 2 E Din- fu.em Yem (0 Ø).
We estimate each coefficient fe via spherical quadrature:
fr EN EN= W4. fa(O,, di.) Yin (0,, 6.),
5

---

where (0), Φ:) range over a suitable sampling grid on the sphere, and wy. are the corresponding quadrature weights. We use Gauss-Legendre quadrature with L 3, which avoids evaluating the points 0 E (0, ). where there is no well-defined vertical camera orientation.
Ensuring Invariance Horizontal Rotations Our goal is invariance to rotations of the animal about the vertical (z) axis i.. a shift in Φ. Under such a rotation, ft.mm picks up a phase factor e By taking the squared magnitude fl emll?, we eliminate that phase dependence. Consequently, we form our final descriptor by collecting all such terms for each latent dimension l: and each e, m:
{ [1fm.emll) k12..512, t= m=-E.(
This yields fixed-size feature vector whose entries remain the same if the animal is rotated in the horizontal plane.
However, we found that these feature vectors still strongly encoded the azimuthal angle of the animal, possibly due uneven lighting conditions across views. To remove this effect, we employ an adversarial formulation of principal components analysis (PCA) to find a 50-dimensional subspace of the feature vectors that contains large portion of the variance of the input vectors and can predict only small portion of the variance of the sine and cosine components of the azimuthal rotation angle [9]. We take these 50-dimensional pose descriptors as the visual embedding. See Appendix Dfoor more details.
4
Experiments
4.1 Datasets
Our first dataset consists of six synchronous 30-minute videos taken from cameras with known parameters of a freely-moving mouse in a 28 cm diameter plastic cylinder, which is released as publicly available dataset accompanying this paper (17) (CCO 1.0). The RGB videos are captured at resolution of 1536 x 2048 at frame rate of 30 FPS, for total of 324000 frames. In the video, the mouse engages in a variety of behaviors including walking, rearing, grooming, and resting. Consecutive thirds of the video are used for training, validation, and testing. A second dataset of freely-moving zebra finch is obtained in the same manner with a 20 minute duration. Both datasets downsampled spatially by factor of 4 and temporally by factor of 5 in the following results.
third dataset is Ra77M, which contains videos of a freely moving rat a cylindrical arena captured from 6 camera angles (CC BY 4.0) [39]. This video is more challenging to mask due to occlusions of the feet and tail by the bedding material, additional occlusion on the side of the arena in one of the views, and uneven lighting conditions across the views. We present results frames. Appendix contains additional results from a subset of the 3D-POP dataset consisting of a single freely moving pigeon in large room captured from 4 camera angles (CC BY 4.0) [4 43)
subset of 135,000
4.2 Training Details
The loss hyperparameter was tuned by hand encourage realistic renderings on the training set of the first mouse video. We set Acotor = 0.5 for all experiments. We trained Pose Splatter with a single Nvidia RTX A4000 GPU along with 32 CPU cores used for data fetching. Our model uses only 2.5 GB of GPU memory (VRAM), compared to 10GB for Gaussian Object (68) and 20GB for PixelSplat and MVSplat (10 I... thanks to its simple architecture. Training runs varied between 2 and 12 hours, depending on the number of frames and camera views used. The learning rate was fixed at 10-1 for all experiments. The number of epochs was chosen so to minimize validation set loss, and ranged from 40 to 75 epochs across all experiments. A single forward pass through the model takes about 30 ms during inference. Full details are in Appendix F Project code is available at http:///tttuu.coooo http://////cccoofoee////eeessses
4.3 Rendering Metrics
We report four metrics to compare the quality of rendered images to ground truth. First, intersection over union (IoU) computes the ratio of the intersection of the binarized predicted mask and the ground truth mask to their union, measuring how well the predicted geometry aligns with the true object silhouette. Second, the average L1 distance between predicted and ground truth colors, normalized by the ground truth mask area, quantifies the accuracy of surface appearance. Third, the peak signal-to-noise ratio (PSNR) evaluates the overall pixel-wise fidelity of the rendered image, with higher values indicating lower reconstruction error relative to the dynamic range of image intensities.

---

S00S
Posaippr
MYopii
Ours
MigiePasy
Mousc
InpurGT
inpur View
Otber Views
(catueod)
Finch
Durs cera)
1DGS
PSGS
GO
Ours
View
View
View
Viewna
View
View
View
View
View
View 4
View 5
View
Facch
(c)
Figure 3: (a) Single-view reconstruction. Against single-view baselines, MagicPony and 3D Fauna collapse when the camera departs from the input view, failing to recover a plausible mouse geometry. Pose Splatter, by contrast, reconstructs accurate shapes from all viewpoints of unseen time step the test set. (b) Sparse-view 3DGS comparison. Most sparse-view 3DGS baselines reproduce the white background well but fail to reconstruct the given subject. Consequently. their quantitative scores appear high even though the rendered animals lack detail. See Table lh for quantitative scores. (c) Comparison with per-sceme-optimizee 3DGS (4 view). Some methods post good metrics yet still fail to reconstruct the given subject. See Table D for metrics. Only GaussianObject and Pose Splatter deliver comparable, visually convincing foreground reconstructions.
Lastly, the structural similarity index measure (SSIM) assesses the perceptual similarity between the predicted and ground truth images by comparing local luminance, contrast, and structural information, providing more perceptually aligned quality score than pixel-wise metries.
4.4 Results
Pose Splatter accurately learns animal geometry and appearance. We com- pared Pose Splatter with the strongest sparse-view 3D Gaussian-splatting (3DGS) baselines. Because the highest- performing methods to date still rely on scene-specific optimization, first selected three per-scene optimiza- tion pipelines: 3DGS [26).
Method
Mouse
Finch
loU?
L11
PSNRT
SSIM1
1oUf
L11
PSNR
SSIMT
Per-Scene
3DGS
0.502
0,742
25.9
0.969
0.513
0.689
26.4
0.975
Optinization
D.923
0.777
34.1
PixelSplar
0.424
0.921
25.2
0.988
0.488
0.858
26.2
0.911
Fend-Forward
MVSplat
0.417
0.887
25.5
0.966
0.461
0.893
25.9
0.970
Ours
0.760
0.632
0,982
0.848
0.345
34.5
0.992
(a) Comparison with sparse-view 3DGS methods.
Method
Mouse cam)
Finch cam)
loUt
L11
PSNR1
SSIM
1oU1
L11
PSNR1
SSIM1
3DGS
0.447
0.786
25.8
0.967
0.459
0.754
26.1
0.973
FSGS
0.414
0.982
24.9
0.974
0.423
0.891
25.4
0.980
FSGS (79), and GaussianOb-
GO
0.706
0.745
28.5
0.981
0.725
0.657
30.4
0.985
ject (GO) (68). For every test scene, each model was optimized from scratch on the same five input views, leaving the remaining single view unseen for evaluation, and we retained the authors' default hyperparameters throughout. As summarized in Table Dh and illustrated in Figure Bp. Pose Splatter outperforms every baseline on both evaluation datasets. The original 3DGS, which is designed for dense multi-view input, degrades noticeably when given only sparse cameras (see Figure 3p); its scores remain moderate largely because it reproduces the uniform white background well, which inflates pixel-wise similarity even as the animal's geometry collapses.
Ours
0.721
0.753
28.2
0.982
0.731
0.685
29.0
0.981
(b) Comparison with per-scene optimization methods.
Table 1: (a) Sparse-view 3DGS benchmark. We compare our method with three per-scene optimization 3DGS baselines and two feed-forwar alternatives. Higher values for loU, PSNR, and SSIM and lower values for LI indicate better performance. The best score in each column is set in bold; the second-bes, is underlined. See Figure 3p for qualitative results. (b) Additional benchmark. We conducted additional evaluation by limiting the input to just views and benchmarking against the pntiiizttion-asser 3DGS baselines. See Figure 3 for qualitative results.
7

---

GT
Finch model
Mouse model
model
GT
UNet
carn
cam
Figure 4: (a) Cross-species renderings. (b) Renderings given different numbers of input views. The rendered views are novel for the 5- and 4-camera models.
FSGS behaves much the same: it reproduces the white background well and sketches a coarse shape, so its overall metrics stay reasonable, yet-as Figure B shows-it fails to recover the correct body shape. GaussianObject (GO) achieves the strongest numbers among the optimization methods. By coupling pretrained diffusion priors with iterative depth-guided refinements, GO produces markedly cleaner surfaces and sharper texture, as both the table and figure confirm. The trade-off is runtime: each scene of GO requires roughly one hour of test-time-optimilee time-optimization, as opposed to the roughly 30 ms test-time forward pass of Pose Splatter (over 100,000x faster).
We now turn feed-forward baselines. In line with each author's recommendations, we trained both PixelSplat and MVSplat with two input views and treated one of the remaining views as the test view, leaving all other hyperparameters at their defaults. Both models post moderate scores because, like the other methods, they reproduce the white background. However, Figure 3b shows that neither network reconstructs the animal itself with meaningful accuracy. Both authors note that their pipelines depend on substantial overlap between input views to establish reliable cross-view correspondences. Our datasets provide minimal overlap, leaving few shared features to match, and this limitation prevents either model from capturing the true 3D geometry. We experimented with varying the number of input views during testing, but observed no noticeable changes the results (see Appendix (G). In addition, we provide comparisons with several large, pretrained, and generalizable feed-forward 3D reconstruction models Appendix G including Hun Yuan 3D-2 788. TRELLIS (65). VGGT (60),
and AnySplat [23]
We further evaluated the optimization-baseee baselines and Pose Splatter using sparser view setting, with each method trained on only four views and assessed on the remaining two. Results are in Table D and Figure 3.. Under this split, GaussianObject retains a slight edge. but Pose Splatter matches it within a small margin on every metric. Figure 3P shows that both methods capture comparable geometry and texture, whereas the original 3DGS and FSGS do not perform well. 3DGS and FSGS achieve good numerical scores because they accurately reconstructed the white background, but qualitative experiments make clear that fine detail is lost.
Additionally, because multi-view animal datasets are scarce, the current state of the art in 3D animal reconstruction still relies single-image mesh predictors. We therefore compared Pose Splatter with two leading single-view models: MagicPony (64) and 3D Fauna (33 We trained these two models on all six reference views and evaluated them on random view from an unseen time-step. Pose Splatter used the same six training images but was tested on all six views of the unseen time step. As illustrated in Figure 3h, the single-view networks accurately reproduce their input view yet fail to maintain shape coherence once the mesh is rotated, whereas Pose Splatter preserves plausible anatomy from every angle. The difference stems from the fact that single-imag pipelines never observe the six views simultaneously, making it difficult to resolve self-occlusions and silhouette ambiguities in complex animal poses. We also present the results of another single-view animal reconstruction model, BANMo (69). on our dataset in Appendix G Similar to MagicPony and 3D Fauna, BANMo fails to accurately reconstruct the shape from unseen views. See Appendix H for additional novel and ineut-view Pose Splatter renderings.
8

---

Methed
Miuse
Finch
Rad
loUt
L14
PSNR1
SSIM
1.11
PSNIRT
SSIMT
lnt
L.1
PSNKT
SSIM
ToUT
1.1
PSNKT
XSIM
6can
8865
0.317
53.5
0.989
0.913
0.23
36.8
0.991
0.797
0,658
26-9
0.975
Mouse -> Rat
0.658
1.014
25.1
0.972
6cam
01825
0:380
32.2
0.987
0.570
0.30%
34.5
01590
0.664
0.549
25.5
0.771
Finch + Rat
0.545 1.200
24.0
0.972
Siam
19.0
0.982
0.548
0,34
0.794
0,68
27.6
0.911
5ca
29.8
1383
0.558
D.421
0.688
1.16
24.0
0.570
Mouse Finch
0.719
0.625
31.1
0.988
RJ61
Finch + Mouse 0.736 0.609
29.3
0.982
28.4
0.579
0.555
(a) Pose Splatter ablation study
(b) 5-camera cross-species generalization
Table 2: (a) Ablation study. The table shows how Pose Splatter responds to fewer input views and to the removal of the stacked U-Net refinement (methods annotated with a superscript minus). (b) 5-camera cross-species generalization. Models trained on one species are evaluated on the single held-out view of another, revealing how well the learned representation transfers across animals.
Keypoint Prediction
Behavior Prediction
1.0
Keypsints
0.9
Visual Embeddinn
0.75
0.50
0.6
0:25
0.4
Wark
Head Up
soll
Gream
Head Left
Tall Down
Rel up
Mouse Finch
Head
Mouse
Finch
Figure 5: Left R² values of predicting egocentric 3D keypoints from visual embeddings. Each scatterpoint represents a single manually annotated keypoint. Right: Accuracies logistic regression models predicting different manually annotated behaviors using egocentric 3D keypoints (gray) versus visual embeddings (purple). Six of eight behaviors are better predicted by the visual embedding.
Pose Splatter generalizes across different species We evaluated cross-species transfer by applying models trained on one animal to single held-out view of another. As Table shows, accuracy trails the in-species baselines (cf. the camera results in Table yet the decline modest; indeed, the Finch-to- Mouse model matches the Mouse-only 5-camera model. Also, the stronger Mouse-to- Rat performance, compared with Finch-to-Rate suggests that morphological similarity may ease transfer. Qualitative results in Figure 4 confirm that, despite the domain shift, all three models still recover the novel animal's overall shape and appearance with only minor degradation.
Visual embedding produces preferred nearest neighbors We have seen that Pose Splatter is able to accurately model the pose and appearance of animals. Now we turn our attention to whether the proposed visual embeddings provide an informative description of animal pose. First, we reasoned that a good description of animal pose should provide meaningful nearest neighbors. To test this, we trained a supervised 2D keypoint predictor (SLEAP, (48) ) using 1000 hand-labeled images of mouse poses with 16 keypoints, which required roughly 15 hours of manual annotation time. We then performed a robust triangulation to create 3D keypoints for each frame. The keypoints are shifted and rotated in the X/Y plane into an egocentric coordinate system, a 48-dimensional representation of mouse pose. We additionally calculated our proposed visual embedding, a 50- dimensional representation, which requires no manual annotation phase. To gauge the relative quality of nearest neighbors produced by both feature sets, we had 102 participants chose the more similar of two poses 40 randomly sampled query poses. We chose Euclidean metric calculate nearest neighbors in both feature spaces and excluded nearby frames in time (within 500 frames). We additionally randomized the order of presentation of the two candidate answers. Example queries and answers are shown in Figure 2b. demonstrating the high degree of similarity produced by both feature sets. Participants showed slight but statistically significant preference for the visual embedding nearest neighbors (54.0 + 0.8% of visual embedding nearest neighbors preferred, mean + SEM, p = 1.5 x 10-5, two-sided sampl (-test, n = 40). Thus, our findings provide strong evidence of genuine preference for the visual embedding neighbors over the keypoint-based neighbors. This is despite the fact that Pose Splatter requires no manual annotations and the 3D keypoints were tested
in the optimistic condition where the manual annotations were taken from the same dataset. See

---

Appendix ! for an investigation of the triangulated 3D keypoint quality and Appendix D for additional survey details.
Visual embedding captures subtle movements To test whether the visual embedding could encode subtle variations in pose, took a closer look 5-second clip in the test portion of the finch video in which the finch's feathers slowly but subtly expand in a way not seen in the training footage. Interspersed in this footage are two brief bouts of head shaking. Figure 2 shows three stills from the clip in addition to the same three rendered frames. Below, we plot the first 5 principal components of the visual embedding over time and observe that they correspond very well with the annotated features of the video, including the subtle expansion of feathers.
Visual embedding vs. keypoints We next assessed whether the visual embedding implicitly encodes 3D information by predicting egocentric 3D keypoints from the visual embedding using a 5-nearest- neighhor regressor (see Appendix (D). The visual embedding explains the majority of variance for most mouse keypoints and all finch keypoints, even though it was not trained explicitly for this purpose (Fig. [ left). We then evaluated a common downstream application application--bhaviiee classification- by training logistic regression models to predict manually annotated behaviors from either egocentric 3D keypoints visual embeddings (see Appendix D for details). The visual embedding outperforms the keypoint-ban basee features for six of the eight annotated behaviors, despite requiring no manual supervision (Fig.) right).
We measured how Pose Splatter performs when fewer input cameras are available and when the stacked U-Net refinement is removed. In the "shape-carving only" variant which bypasses the U-Net and maps the carved voxel grid directly to Gaussian parameters we flag each row with superscript minus in Table 2 (e.g., 6 cam"). We observe that quantitative scores decline as views are removed yet the 5- and 4-camera models still recover the animals' overall shape and appearance well as shown in Figure 4p. Also, every setting that retains the stacked U-Net outperforms its counterpart without it. Qualitative results in Figure 4p clearly show this trend: compared with the full 6-camera model, the 6 cam ("6 cam w/o U-Net") volume is noticeably noisier, leading to blurrier renders and loss of fine detail.
Ablation study
5 Discussion & Conclusion
In this work, we introduced Pose Splatter, a novel approach leveraging 3D Gaussian splatting and shape carving to reconstruct and quantify the 3D pose and appearance of laboratory animals. Unlike existing keypoint- and mesh-based methods, which either require extensive manual annotations or rely predefined template models, Pose Splatter operates in feed-forward manner without the need for per-frame optimization and requires no manual annotation. Our method successfully captures fine-grained postural details and provides compact, iavarinninvariant visual embedding that can be seamlessly integrated into downstream behavioral analyses.
Although Pose Splatter performs well with a single subject, some neuroscience experiments involve multiple interacting animals. Such settings introduce prolonged and severe occlusions beyond our current benchmarks. While Pose Splatter effectively resolves most self-occlusions, addressing these more challenging scenarios remains an open problem for future work. Additionally, while our visual embedding provides a powerful descriptor for capturing pose and appearance, further exploration is needed to improve interpretability and facilitate direct comparisons across species.
Acknowledgments
We thank Tim Dunn, Kafui Dzirasa, and Kathryn Waride-Chhissteneen for helpful conversations, which greatly improved the work. We are grateful to Katherine Kaplan and Rich Mooney for help obtaining zebra finch recordings and to Kathryn Walder-Christena obtaining mouse recordings. Lastly, we thank Kyle Severson for assistance with video acquisition software. Research reported in this publication was supported by the National Institute Of Mental Health of the National Institutes of Health under Award Number RO ROIMH125430, The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
and Hannah Soliman for help
10

---

References
[1] Liang An, Jilong Ren, Tao Yu, Tang Hai, Yichang Jia, and Yebin Liu. surface motion capture of multiple freely moving pigs using MAMMAL", In: Nature Commu- nications 14.1 (2023), p. 7727.
[2] Praneet C Bala, Benjamin R Eisenreich, Seng Bum Michael Yoo. Benjamin Y Hayden, Hyun Soo Park, and Jan Zimmermann. "A "Automated markerless pose estimation in freely moving macaques with OpenMonkeyyStudio
In: Nature communications 11.1 (2020), p. 4560.
[3] Kristijan Bartol, David Bojanić, Tomislav Petković, and Tomislav Pribanic. "Generalizable human pose triangulation". In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, pp. 11028-11037.
[4] Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, and Michael J Black. "Keep it SMPL: Automatic estimation of 3D human pose and shape from single image". In: Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14. Springer. 2016, pp. 561-578.
[5] James P Bohnslav, Mohammed Abdal Monium Osman, Akshay Jaggi, Sofia Soares, Caleb Weinreb, Sandeep Robert Datta, and Christopher D Harvey. " "ArMo: An Articulated Mesh Approach for Mouse 3D Reconstruction" In: bioRxiv (2023), pp. 2023-02.
[6] Yanlu Cai, Weizhong Zhang. Yuan Wu, and Cheng Jin. "FusionFormer: A Concise Unified Feature Fusion Transformer for 3D Pose Estimation".. In: Proceedings of the AAAI Conference Artificial Intelligence. Vol. 38. 2. 2024, pp. 900-908.
[7] Jinkun Cao, Hongyang Tang, Hao-Shu Fang, Xiaoyong Shen, Cewu Lu, and Yu- Wing Tai. "Cross-domain adaptation for animal pose estimation". In: Proceedings of the IEEE/CVF international conference on computer vision. 2019, pp. 9498-9507.
[8] Joao Carreira, Pulkit Agrawal, Katerina Fragkiadaki, and Jitendra Malik. "Human pose estima- tion with iterative error feedback". In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 4733-4742
[9] William E Carson IV, Austin Talbot, and David Carlson. " "AugmentedPCA: A Python Package of Supervised and Adversarial Linear Factor Models". In: arXiv preprint arXiv:2201.02544 (2022).
[10] David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. "pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction" In: Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, pp. 19457- 19467.
[11] Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. "Mysplat: Efficient 3d gaussian splatting from sparse multi-view images". In: European Conference on Computer Vision. Springer. 2024, pp. 370-386.
[12] Timothy W Dunn, Jesse D Marshall, Kyle S Severson, Diego E Aldarondo, David GC Hilde- brand, Selmaan N Chettih, William L Wang, Amanda Gellis, David E Carlson, Dmitriy Aronov, et al. "Geometric deep learning enables 3D kinematic profiling across species and environments". " In: Nature methods 18.5 (2021), pp. 564-573.
[13] Sai Kumar Dwivedi, Nikos Athanasiou, Muhammed Kocahas, and Michael J Black. "Learning to regress bodies from images using differentiable semantic rendering", In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021, pp. 11500-1229.
[14] Aghileh S Ebrahimi, Patrycja Orlowska-Feuerr Qian Huang, Antonio G Zippo. Franck P Martial, Rasmus $ Petersen, and Riccardo Storchi. "Three-dimensional unsupervised proba- bilistic pose reconstruction (3D-UPPER) for freely moving animals". In: Scientific Reports 13.1 (2023), 155.
[15] Zhiwen Fan, Kevin Wang, Kairun Wen, Zehao Zhu, Dejia Xu, and Zhangyang Wang. "Light- gaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps"" In: arXiv preprint arXiv:2311. 17245 (2023).
[16] Mihai Fieraru, Anna Khoreva, Leonid Pishchulin, and Bernt Schiele. "Learning to refine human pose estimation". In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2018, pp. 205-214.
11

---

[17] Jack Goffinet, Kate Kaplan, Hannah A. Soliman, Kathryn K.
Youngjo Min, Richard Mooney, Kafui Dzirasa, and David E. Carlson. Data from: Pose Splatter: 3D) Gaussian Splatting Model for Quantifying Animal Pose and Appearance. Duke Research URL: https: //do::///oi../ org/ 7924/
Data Repository, 2026. DOI: 10. 77924/r42323H/
r42323k2c
[18] Adam Gosztolai, Semih Günel, Victor Lobato-Ríos Marco Pietro Abrate, Daniel Morales, Helge Rhodin, Pascal Fua, and Pavan Ramdya. "LiftPoseDD, a deep learning-based approach for transforming two-dimensional to three-dimensional poses in laboratory animals". In: Nature methods 18.8 2. pp. 975-981.
[19] Jacob M Graving, Daniel Chae, Hemal Naik, Liang Li, Benjamin Koger, Blair R Costelloe, and lain D Couzin. "DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning". In: Elife 8 (2019), e47994.
[20] Yinghao Huang. Federica Bogo, Christoph Lassner, Angjoo Kanazawa, Peter V Gehler, Javier Romero, Ijaz Akhter, and Michael J Black. "Towards accurate marker-less human shape and pose estimation over time". In: 2017 international conference on 3D vision (3DV). IEEE. 2017, pp. 421-430.
[21] Karim Iskakov, Egor Burkov, Victor Lempitsky, and Yury Malkov. "Learnable triangulation of human pose". In: Proceedings of the IEEE/CVF international conference on computer vision. 2019, pp. 7718-7727.
[22] Boyuan Jiang, Lei Hu, and Shihong Xia. "Probabilistic triangulation for uncalibrated multi- view 3d human pose estimation". In: Proceedings of the IEEÉ/CVF International Conference Computer Vision 2023, pp. 14850-14866
[23] Lihan Jiang. Yucheng Mao, Linning Xu, Tao Lu, Kerui Ren, Yichen Jin, Xudong Xu, Mulin Yu, Jiangmiao Pang. Feng Zhao, et al. "AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Views". In: arXiv preprint arXiv:2505. 23 (2025).
[24] Angjoo Kanazawa, Michael Black, David W Jacobs, and Jitendra Malik. "End-to-end recovery of human shape and pose". In: Proceedings of the IEEE conference computer vision and pattern recognition. 2018, pp. 7122-7131.
[25] P Karashchuk, KL. Rupp, ES Dickinson, S Walling-Bell, E Sanders, E Azim, BW Brunton, and JC Tuthill. Anipose: A toolkit for robust markerless 3D pose estimation. Cell Rep. 36, 109730. 2021.
[26] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. "3D Gaussian Splatting for Real-Time Radiance Field Rendering". In: ACM Transactions on Graphics 42.4 (July 2023). URL: https: //repo sam inria. fr / fungraph / 3d gaussian splatting//
[27] Ugne Klibaite, Tianqing Li, Diego Aldarondo, Jumana Akoad, Bence P Ölveczky, and Timothy W Dunn. "Mapping the landscape of social behavior". In: bioRxiv (2024). [28] Nikos Kolotouros, Georgios Pavlakos, Michael J Black, and Kostas Danilidis. "Leariin to reconstruct 3D human pose and shape via model-fitting in the loop". In: Proceedings of the IEEE/CVF international conference on computer vision. 2019, pp. 2252-2261.
[29] Jogendra Nath Kundu, Siddharth Seth, MV Rahul, Mugalodi Rakesh, Venkatesh Babu Rad- hakrishnan, and Anirban Chakraborty. "Kinematic-structt -treerrttcuuur-e--eeeddd representation for unsupervised 3d human pose estimation". " In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34. 07. 2020, pp. 11312-11319.
[30] Kiriakos N Kutulakos and Steven M Seitz. "A theory of shape by space carving". In: Interna- tional journal of computer vision 38 (2000), pp. 199-218.
[31] Aldo Laurentini. "The visual hull concept for silhouette-based image understanding". In: IEEE Transactions on pattern analysis and machine intelligence 16.2 (1994), pp. 150-162.
[32] Chen Li and Gim Hee Lee. "From synthetic to real: Unsupervised domain adaptation for animal pose estimation". In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021, pp. 1482-1491.
[33] Zizhang Li, Dor Litvak, Ruining Li, Yunzhi Zhang, Tomas Jakab, Christian Rupprecht, Shangzhe Wu, Andrea Vedaldi, and Jiajun Wu. "Learning the 3d fauna of the web". In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, pp. 9752-9762.
12

---

[34] Matthew Loper, Naureen Mahmood, Javier Romero. Gerard Pons-Moll, and Michael J Black. "SMPL: a skinned multi-person linear model". In: ACM Transactions on Graphics (TOG) 34.6 (2015), pp. 1-16.
[35] Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. "Dynamic 3d gaus- sians: Tracking by persistent dynamic view synthesis". In: arXiv preprint arXiv:2308.09773 (2023).
[36]
Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. *Dynamic 3d gaus- Tracking by persistent dynamic view synthesis". In: 2024 International Conference on 3D Vision (3DV). IEEE. 2024, pp. 800-809.
sians:
[37] Jesse D Marshall, Ugne Klibaite, Amanda Gellis, Diego E Aldarondo, Bence P Ölveczky, and Timothy W Dunn. "The pair-r24m dataset for multi-animal 3d pose estimation". In: bioRxiv (2021). pp. 2021-11.
[38] Jesse D Marshall, Tianqing Li, Joshua H Wu, and Timothy W Dunn. "Leaving flatland: Advances in 3D behavioral measurement". In: Current Opinion in Neurobiology 73 (2022),
p. 102522.
[39] Jesse D. Marshall, Diego Aldarondo, William Wang, Bence P. Ölveczky, and Timothy Dunn. Rat 7M. httpps://doi.... org/ 10. 6084/m9 www.hare..ccaai.... 2021.
v3 figshare. Collection.
[40] Alexander Mathis, Pranav Mamidanna, Kevin M Cury, Taiga Abe, Venkatesh N Murthy, Mackenzie Weygandt Mathis, and Matthias Bethge. "DeepLabCut: markerless pose estimation of user-defined body parts with deep learning". In: Nature neuroscience 21.9 (2018), pp. 1281- 1289.
[41] Rahil Mehrizi, Xi Peng. Zhiqiang Tang, Xu Xu, Dimitris Metaxas, and Kang Li. "Toward marker- free 3D pose estimation in lifting: A deep multi-view solution". In: 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018). IEEE. 2018,
pp. 485-494
[42] Jiteng Mu, Weichao Qiu, Gregory D Hager, and Alan L. Yuille. "Learning from synthetic animals". In: Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition. 2020, pp. 12386-12395
[43] Hemal Naik, Alex Hoi Hang Chan, Junran Yang. Mathilde Delacoux, D. lain Couzin, Fumihiro Kano, and Mate Nagy. 3D-POP - An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture. Version V6. 2023. DO1: 10. 17617/633133 / HPBBC7 URL: hhtpps:/ : //doi. org/ 17617 /3, HPBBC7
[44] Hemal Naik, Alex Hoi Hang Chan, Junran Yang, Mathilde Delacoux, Iain D Couzin, Fumihiro Kano, and Máté Nagy. "3D-POP: an automated annotation approach to facilitate markerless 2d-3d tracking of freely moving birds with marker-based motion capture". In: Proceedings of the IEFF/CVF conference on computer vision and patlern recognition. 2023, pp. 21274-212 284.
[45] Tanmay Nath, Alexander Mathis, An Chi Chen, Amir Patel, Matthias Bethge, and Mackenzie Weygandt Mathis. "Using DeepLabCut for 3D markerless pose estimation across species and behaviors". In: Nature protocols 14.7 (2019), pp. 2152-211211 2176.
[46] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed AA Osman, Dimitrios Tzionas, and Michael J Black. "Expressive body capture: 3d hands, face, and body from a single image". In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019, pp. 10975-10985.
[47] Talmo D Percira, Diego E Aldarondo, Lindsay Willmore, Mikhail Kislin, Samuel S-H Wang, Mala Murthy, and Joshua W Shaevitz. "Fast animal pose estimation using deep neural net- works". In: Nature methods 16.1 (2019), pp. 117-125.
[48] Talmo D Pereira, Nathaniel Tabris, Arie Matsliah, David M Turner, Junyu Li, Shruthi Ravin- dranath, Eleni S Papadoyannis. Edna Normand, David S Deutsch, Z Yan Wang, et al. "SLEAP: A deep learning system for multi-animal pose tracking"' In: Nature methods 19.4 (2022), pp. 486-495.
[49] Tomas Pfister, James Charles, and Andrew Zisserman. "Flowing convnets for human pose estimation in videos". In: Proceedings of the IEEE international conference on computer vision. 2015, pp. 1913-1921.
[50] Haibo Qiu, Chunyu Wang. Jingdong Wang, Naiyan Wang, and Wenjun Zeng. "Cross view fu- sion for 3d human pose estimation". In: Proceedings of the IEEF/CVF international conference computer vision. 2019, pp. 4342-4351.
13

---

[51] Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan. Kalyan Vasudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer. SAM 2: Segment Anything in Images and Videos. 2024. arXiv: 2408. 00714 [cs. CV] URL: htttps:///aaiiiii org/
[52] Edoardo Remelli, Shangchen Han, Sina Honari, Pascal Fua, and Robert Wang. "Lightweight multi-view 3D pose estimation through camera-disentangle representation". In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020, pp. 6040-6049.
[53] Helena Russello, Rik van der Tol, and Gert Kootstra. "T-LEAP: Occlusion-robuse pose es- timation of walking cows using temporal information". . In: Computers and Electronics in Agriculture 192 (2022), P. 106559.
[54] Remy Sabathier, Niloy J Mitra, and David Novotny. "Animal avatars: Reconstructing animat- able 3D animals from casual videos". In: European Conference on Computer Vision. Springer. 2025, pp. 270-287.
[55] Xiao Sun, Jiaxiang Shang, Shuang Liang, and Yichen Wei. "Compositional human pose regression". In: Proceedings of the IEEE international conference on computer vision. 2017,
pp. 2602-2611.
[56] Xiao Sun, Bin Xiao, Fangyin Wei, Shuang Liang, and Yichen Wei. "Integral human pose regression". In: Pruceedings of the European conference on computer vision (ECCV). 2018,
pp. 529-545.
[57] Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. "Dreamgaussian: Genera- tive gaussian splatting for efficient 3d content creation". In: arXiv preprint arXiv:2309.1660 (2023).
[58] Alexander Toshev and Christian Szegedy. "Deeppose: Human pose estimation via deep neural networks". In: Proceedings of the IEEF conference on computer vision and pattern recognition. 2014, pp. 1653-1660.
[59] Tao Tu, Ming-Feng Li, Chieh Hubert Lin, Yen-Chi Cheng, Min Sun, and Ming- Hsuan Yang. "DreaMo: Articulated 3D Reconstruction From A Single Casual Video". In: arXiv preprint arXiv:2312.02617 (2023).
[60] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. "VGGT: Visual Geometry Grounded Transformer". In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2025.
[61] Caleb Weinreb, Jonah E Pearl, Sherry Lin, Mohammed Abdal Monium Osman, Libby Zhang, Sidharth Annapragada, Eli Conlin, Red Hoffmann, Sofia Makowska, Winthrop F Gillis, et al. "Keypoint-MoSeeg parsing behavior by linking point tracking to pose dynamics". In: Nature Methods 21.7 (2024), pp. 1329-1339.
[62] Alexander B Wiltschko, Tatsuya Tsukahara, Ayman Zeine, Rockwell Anyoha, Winthrop F Gillis, Jeffrey E Markowitz, Ralph Peterson, Jesse Katon, Matthew J Johnson, and Sandeep Robert Datta. "Revealing the structure of pharmacobehavioval space through motion sequencing". In: Nature neuroscience 23.11 (2020), pp. 1433-1443.
[63] Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. "4d gaussian splatting for real-time dynamic scene rendering". In: Proceedings of the IEEÉ/CVF Conference on Computer Vision and Pattern Recognition. 2024, pp. 20310-20320.
[64] Shangzhe Wu, Ruining Li, Tomas Jakab, Christian Rupprecht, and Andrea Vedaldi. "Mag- icpony: Learning articulated 3d animals in the wild". In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, pp. 8792-8802.
[65] Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. "Structured 3d latents for scalable and versatile 3d generation". In: Proceedings of the Computer Vision and Pattern Recognition Conference. 2025, pp. 21469- 21480.
[66] Jiacong Xu, Yi Zhang, Jiawei Peng, Wufei Ma, Artur Jesslen, Pengliang Ji, Qixin Hu, Jichua Zhang, Qihao Liu, Jiahao Wang. et al. "Animal3d: A comprehensive dataset of 3d animal pose and shape". In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023, pp. 9099-9109.
14

---

[67] Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang. Shenghua Gao, and Ying Shan. "In- stantmesh: Efficient 3d mesh generation from a single image with sparse- view large recon- struction models". In: arXiv preprint arXiv:2404.077
(2024).
[68] Chen Yang, Sikuang Li, Jiemin Fang. Ruofan Liang, Lingxi Xie, Xiaopeng Zhang, Wei Shen, and Qi Tian. "GaussianObject High-Quality 3D Object Reconstruction from Four Views with Gaussian Splatting". In: ACM Transactions on Graphics 43.6 (2024).
[69] Gengshan Yang. Minh Vo. Natalia Neverova, Deva Ramanan, Andrea Vedaldi, and Hanbyul Joo. "Banmo: Building animatable 3d neural models from many casual videos". In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, pp. 2863-2873, [70] Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. "De- formable 3d gaussians for high-fidelit monocular dynamic scene reconstruction", In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024,
pp. 20331-20341.
[71] Chun-Han Yao, Wei-Chih Hung, Yuanzhen Li, Michael Rubinstein, Ming-Hsuan Yang, and Varun Jampani. "Lassie: Learning articulated shapes from sparse image ensemble via 3d part discovery". In: Advances in Neural Information Processing Systems 35 (2022), pp. 15296- 15308.
[72] Vickie Ye, Ruilong Li, Justin Kerr, Matias Turkulainen, Brent Yi, Zhuoyang Pan, Otto Seiskari, Jianbo Ye, Jeffrey Hu, Matthew Tancik, and Angjoo Kanazawa. "gsplat: An Open-Source Library for Gaussian Splatting". In: arXiv preprini arXiv:2409.0677 06765 (2024), arXiv: 2409
06765 CV] URL: httpps:/ //ar://axiiv org/ hts//ba//200 aba/24/a//244////////////////a 06765
[73] Hang Yu, Yufei Xu, Jing Zhang, Wei Zhao, Ziyu Guan, and Dacheng Tao. "Ap-10k: A benchmark for animal pose estimation in the wild". In: arXiv preprint arXiv:2108. 126.11177 (2021).
[74] Dingwen Zhang, Guangyu Guo, Dong Huang, and Junwei Han. "Poseflow: A deep motion representation for understanding human behaviors in videos". In: Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition. 2018, pp. 6762-6770.
[75] Libby Zhang, Tim Dunn, Jesse Marshall, Bence Olveczky, and Scott Linderman. "Animal pose estimation from video data with a hierarchical von Mises-Fisher Gaabs--n model". In: International conference
artificial intelligence and statistics. PMLR. 2021, pp. [76] Yilun Zhang and Hyun Soo Park. "Multiview supervision by registration". In: Proceedings of the IEEE/CVE Winter Conference on Applications of Computer Vision. 2020, pp. 420-428.
[77] Yuexi Zhang, Yin Wang, Octavia Camps, and Mario Sznaier. "Key frame proposal network for efficient pose estimation in videos". In: European Conference on Computer Vision. Springer. 2020, pp. 609-625.
[78] Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang. Yifei Feng, Mingxin Yang. Sheng Zhang, Xianghui Yang, et al. "Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation". In: arXiv preprint arXiv:2501. /2202 (2025).
[79] Zehao Zhu, Zhiwen Fan, Yifan Jiang, and Zhangyang Wang. "Fsgs: Real-time few-shot view synthesis using gaussian splatting". In: European conference on computer vision. Springer. 2024. pp. 145-163.
[80] Silvia Zuffi and Michael Black. "AWOL: Analysis WithOut Synthesis Using Language". In: European Conference on Computer Vision. Springer. 2024, pp. 1-19.
[81] Silvia Zuffi, Angjoo Kanazawa, Tanya Berger-Wolf and Michael J Black. "Three-D Safari: Learning to Estimate Zebra Pose, Shape. and Texture from Images" In the Wild"". In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision. 2019, pp. 5359- 5368.
[82] Silvia Zuffi, Angjoo Kanazawa, and Michael J Black, "Lions and tigers and bears: Capturing non-rigid, 3d, articulated shape from images". In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2018, pp. 3955-3963.
[83] Silvia Zuffi, Angjoo Kanazawa, David W Jacobs, and Michael J Black. "3D menagerie: Modeling the 3D shape and pose of animals". In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, pp. 6365-6373.
[84] Matthias Zwicker, Hanspeter Pfister, Jeroen Van Baar, and Markus Gross, "Surface splat- ting". In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques. 2001, pp. 371-378.
15

---

A Camera Considerations
Pose Splatter requires a minimum of roughly four calibrated cameras to operate, but performs best with 5 or more cameras, seen in Figure Ap Additionally, even lighting conditions and well-spread camera orientations facilitate rotation- inttriinnnvvaaantt visual embeddings and higher quality shape-carver volumes, respectively. These camera requirements may hinder application in some outdoor wild settings, but are becoming more common in laboratory settings where precise behavioral tracking is needed (e.g.
22 12 (.
B Center and Rotation Estimation
To estimate the animal's orientation at each time point t. we modeled its shape as Gaussian distribution (u, 2) by matching the 3D moments of shape-carved voxel grid. We then extracted the principal axis by computing the eigenvector corresponding to the largest eigenvalue of E, This eigenvector is normalized to unit length and called v,. Because an eigenvector can flip sign from one time point to the next, we introduced sign-con gonsisssst procedure to best ensure smooth progression of the eigenvectors. Specifically, after computing V at time t ++ 1, we used a Wasserstein-2 optimal transport map to track a reference point from the distribution at time to that at time / + 1. We then compared the distance of this transported point to the two possible orientations (/4++1 + v*+1) and chose the orientation that preserved the local consistency (i.e., whichever was closer to the transported point).
Finally, we enforced a global orientation consistency by making use of the following heuristic: animals move on average in the direction they face, and not the opposite direction. We operationalize this heuristic by checking if the cumulative displacement of the mean positions μr from start end correlated positively with the sequence of principal axes {vr}. If the overall dot product was negative, we flipped all axes to align with the general direction of motion. This procedure yielded temporally consistent set of principal axes {v:) that reliably tracked the animal's 's primary orientation over time.
C Shape Carving Details
Voxel Occupancy We first start with 112 x 112 x 112 spatial grid with equal-sized edges. The z-axis is aligned with the third axis of the grid, while the first axis the grid is aligned with the estimated azimuthal heading direction of the animal (see Appendix (B. For each point in the grid and each camera, we determine whether its projection onto the image plane of the camera corresponds with masked (animal) or unmasked (background) point. Voxels that correspond with masked regions in at least N cameras are considered occupied.
Voxel Colors To estimate the color of each voxel in the reconstructed volume, first determined its visibility from multiple camera viewpoints using a ray-casting procedure. For each camera, the voxels were sorted by their distance from the camera center, ensuring that the closest voxels were processed first. These voxels were then projected into the image plane using a standard pinhole camera model. To track occlusions, a depth buffer was maintained at each pixel location. If voxel mapped to a pixel that already contained a closer voxel, it was marked as occluded. This process produced a visibility mask indicating whether each voxel was directly observable from each camera.
We then computed each voxel's color by sampling the corresponding pixel values from the camera images. All voxels, whether fully visible or occluded, were projected into the camera views, and their colors were retrieved from the respective images. To integrate these sampled colors into a single representative value per voxel, we applied weighted averaging scheme. Fully visible voxels contributed with a weight of one, while occluded voxels were assigned a reduced weight (0.25) to reflect the uncertainty introduced by occlusion. The final voxel color was obtained by normalizing and summing these weighted contributions across all cameras, allowing us to account for occlusions while still incorporating partial information from less reliable viewpoints.
Determining Volume Dimensions
To save on RAM and GPU memory during training, we truncated our original 112 112 112 volumes based on voxel usage in the animal's training frames. We first calculated sum over all the voxel occupancies and then manually set thresholds and determined volume slice indices to balance
16

---

voxel use and memory considerations. The following table shows the final dimensions of the volumes for each animal.
Mouse
96 x 80 x 64
Finch
96 64 80
Rat
96 x 80 x 64
Table 3: Volume dimensions
(dz X dy x dz) for each animal.
Final Steps Our volumes consist of channels: one binary occupancy channel and 3 color channels. To produce our final volumes, we produce two volumes independently, one with an occupancy threshold of C, where C the number of cameras, and one with the threshold C 1. We then average the two volumes together. Note that the C' --- threshold produces coarser visual hull, with generally more occupied voxels. Initial tests showed that fine body parts such as mouse tails were often not represented in the shape-carved volume with just threshold at C, especially when masks from different views disagree on the boundaries of the animal. We found that adding the C- - 1 threshold and averaging resulted in much better coverage of fine body parts.
D Visual Embedding Details
The feature extractor used in the visual embedding is a pre-trained ResNet 18, trained on the ImageNet dataset. We pass rendered images through all but the last layer of the network, producing 512- dimensional vector per image. 32 of these vectors are used spherical routine to produce the norms of 16 spherical harmonic coefficients, independently for each feature dimension. The resulting coefficients are then flattened to 8192-dimensional vectors.
To produce our visual embeddings, we reduce the feature dimension in two steps. First, we perform PCA on dataset of feature vectors, reducing the feature vectors to 2000 dimension. Then we collect the estimated azimuthal angle for each pose and concatenate the sine and cosine components of this angle together to be used as concomitant data the adversarial PCA routine [9]. Adversarial PCA was then performed to reduce the feature dimensionality to 50 from 2000. The regularization strength parameter j was set to the smallest integer power of 10 such that the adversarial PCA reconstructions of the sine and cosine components produced an average R² value less than 0.05, indicating the sines and cosines are not readily linearly decodable from the 50-dimensional features. We call these features the visual embedding.
To predict egocentric 3D keypoints from visual embeddings (Figure 5 80/20 train/test split, fit 5-nearest neighbor regressor to the training data, and report a uniform average of the R²2 scores over the 3 spatial dimensions on the test set. We also predicted egocentric 3D keypoints from the 8192-dimensional visual features, which are processed to produce the 50- dimensional visual embeddings to test how much usable information lost this process. We used ridge regression with 5-fold cross validation on the training set to select a model and then report a uniform average of the R² scores over the spatial dimensions on the test set. Figure 6 shows moderately better performance using the visual features than the visual embedding to predict egocentric 3D keypoints.
left), we used a consecutive
To classify behavior given egocentric 3D keypoints and visual embeddings, we used random 60/40 train/test split using all available frames. We used 5-fold cross validation to determine the L2 regularization strength the train set, targeting balanced accuracy metric. Accuracies for each class are then reported on the test set. Mouse behavior was classified using one 4-way prediction (Walk vs. Head Up vs. Still vs. Groom) and finch behavior was classified using two two-way predictions (Head Left vs. Head Right and Tail Down vs. Tail Up).
E
Application to 4- View Pigeon Data
To complement the 6-view mouse, finch, and Rar7M datasets, we applied Pose Splatter to subset (Sequence 8) of the 4-view 3D-POP dataset (44). which contains video of single pigeon in large room.
17

---

Finch Keypoints
Mouse Keypoints
0.8
0.6
0.7
0.4
0.6
Visual Embedding (kNN)
0.2
Visual Embedding (kNN)
Visual Features (Ridge)
Visual Features (Ridge)
0.5
0.0
beak tip
foot left-
tail tip
back
knee left
knee right
head
nose
car left
ear right
paw front left
paw front right
paw rear left
paw rear right
head
back
base
tail middle
tail end
side left
side right
tail base middle middle end
foot
tail
Figure 6: Predicting egocentric 3D keypoints from visual embedding (d 50) and visual features (d = 8192). We observe good predictive ability as measured by R values using both visual embeddings (green) and visual features (purple) to predict held-out finch (left) and mouse (right) keypoints. Note the different vertical axes the two subplots.
Compared to the three datasets tested previously (mouse, finch, and Ra the cameras in the 3D-POP data are located further apart relative the size of the animal. For this reason, we found that the shape carving procedure with the provided camera parameters sometimes failed to detect overlap among the back-projected masks. To correct for this, applied adaptive frame-by-frame adjustment to the intrinsic camera parameters.
Briefly, first find 2D mask centroids in each view and triangulate rough 3D center of the animal using these centers and the provided camera parameters. We then re-project this 3D center point onto the 2D image planes and calculate discrepancy between the reprojected point and the 2D mask centroids. Lastly, the camera center parameters (c₂ and Cy) are updated to remove the discrepancy.
More specifically, we assume intrinsic matrix of the form
Jn
0
[y
Cy
(I)
1
We then project the 3D center point Fworrd into camera coordinates: Feam Re + t, where [R; t] are the camera's 's extrinsic parameters. Lastly, we update the intrinsic center parameters for each camera:
-- u fu (2
cam
Cy
+ 0"
(a /
(3) cam
(2)
where u* and u* are the image coordinates of the reprojected 3D center and (rrams Feam Team) are the three coordinates of the 3D center in camera coordinates.
(2)
(3)
IoU1
L1
PSNRI
SSIM
0.622
1.16
24.8
0.982
Table 4: Pose Splatter metrics on sequence from the 3D-POP single pigeon, 4-view dataset.
Apart from this intrinsic parameter modification aspects of the pipeline remain unchanged for the pigeon data. Table 4 shows performance metrics the test set (c.f. Table 2a 4 cameras). Qualitatively, the renderings are lower quality than the mouse, finch, and rat 4-camera renderings, expected from the lower PSNR, but still adhere to the overall shape of the pigeon (Figure 7).
18

---

Observed
Reconstructed
Figure 7: Representative test set pigeon reconstructions from sequence 8 of the 3D-POP dataset.
Shape Carving
U-Nets
ML.P
Shift & Rotate
Splatring
Train FPS
Inference FPS
Finch
7.0 ms
4.6 ms
14.3 ms
6.0 ms
0.2 ms
7.0
7.4
Mouse
7.0 ms
4.7 ms
13.6 ms
3.5 ms
0.3 ms
5.8
6.5
Rat
7.1
5.1
14.3 ms
2.6
0.3 ms
9.7
13.1
Table 5: Median times for the stages of the Pose Splatter forward pass and overall frame rates for training and inference.
F Training and Network Details
Training was with single Nvidia RTX A4000 GPU along with 32 CPU cores used for data fetch- ing. Our model uses only about 2.5GB of GPU memory (VRAM), thanks to its simple architecture. Training runs varied between and 12 hours, depending the length of the dataset. We estimate that the experiments presented in this paper required days of compute time to complete.
The learning rate was fixed 10 for all experiments. The number of epochs was chosen to minimize validation set loss, and ranged from 40 75 epochs across all experiments.
In experiments with fewer than 6 camera views, we found that this center and rotation estimation procedure produces less reliable results. While it would be useful to develop more robust procedure for camera systems with fewer cameras, this was not a primary aim of our work. Therefore, we used centers and orientations inferred with 6 cameras for all experiments.
Tables 5 and show timing information and the number of Gaussians rendered for all three datasets. Additional details may be found in the supplemental code.
Mcanwhile, our additional training details for the comparison baselines were as follows. For the per-scene optimization methods (3DGS [26]. FSGS [79]. and GaussianObject (68)). the original Revisited (COLMAP)." In typical cettooscciene--oiiennte animal-behavior studies, however, one may work with hundreds of thousands of video frames; running COLMAP on every frame would be prohibitively slow. We therefore assume no COLMAP ground-truth point cloud is available. Instead, follow GaussianObject's COLMAP-fre protocol: we seed optimization with a point cloud created from "DUSt3R: Geometric 3D Vision Made Easy (DUSt3R)," feeding the calibrated poses directly into DUSt3R (because the camera intrinsics and extrinsics are known) so that its reconstruction as clean as possible. The GaussianObject paper shows that this DUSt3R initialization maintains state-of-the-an performance, a result that our experiments confirm. All remaining hyperparameters mirror the "best" settings recommended in each method's s official repository. Additionally, per-scene optimization methods require varying amounts of time depending on the approach, ranging from a few minutes (3DGS, FSGS) to about an hour (GaussianObject) per scene. As such, optimizing tens thousands of frames is impractical. Therefore, we randomly sampled 150 images from the test set and conducted experiments using this subset. For the feed-forward baselines, PixelSplat (10) and MVSplat (II), we likewise adopt the authors' recommended hyperparameters. Finally, the single-view animal reconstruction models MagicPony (64] and 3D Fauna (33] are trained with their prescribed data preprocessing pipelines and hyperparameters.
pipelines initialize optimization with point cloud created from "Structure-from-Mootie
19

---

Before Training
After Training
Finch Mouse
13.9k
13.8k
8.4k
8.5k
Rat
4.8k
4.8k
Table 6: Average number of Gaussians rendered before training (at initialization) and after training.
G Additional Model Comparisons
G.1 PixelSplat & MVSplat
We report how the feed-forward baselines behave the number context views changes (see Fig-
ure According to the papers and authors' code repositories, both PixelSplat [T0) and MVSplat (Ti] were trained with two input views, and adding more cameras does not guarantee improvement the output. Our results confirm that increasing the number of context views does not appreciably improve the performance either baseline.
Target
PixelSplat
MVSplal
Target
PixelSplat
MVSplat
inpul views
4 input views
Figure 8: Results for the two feed-forward baselines with varying numbers of input views.
G.2 TRELLIS & Hun
As an additional point of comparison, we tested the ability of two recent large pretrained 3D object models to render consistent and realistic mice: Hun Yuan3D-2 (78) and TRELLIS (65) Both models pretrain VAEs on large collections of 3D assets and use flow matching to generate latents from conditioning images. These latents can then be decoded into meshes, Figure ground truth conditioning views alongside the meshes produced by the both models when conditioned on these views. We note that the meshes display crisp yet often inaccurate geometric details. For example, note the third Hun Yuan3D --gnerrttdd mesh, which has legs (5 visible from rendered view) and the sixth TRELLIS -Reeett-generated mesh, which has unrealistic head shape. Furthermore, the meshes clearly inconsistent with one another, a clear limitation of single-view conditioned models.
Conditioning Image
HunYuan3D
TRELLIS
Figure 9: Hun Yuan3D-2 and TRELLIS produce crisp yet often inaccurate geometric detail that are inconsistent across conditioning views.
20

---

Figure 10: InstantMesh produces unrecognizable shapes given multi-view image input. Three views of the output mesh are shown. The conditioning images are shown on the first row of Figure
GT
Rotated camera views
Figure 11:
When the mesh generated by BANMo is rotated, it fails to reconstruct proper shape from unseen views.
G.3
InstantMesh
InstantMesh is large pre-trained image to mesh pipeline that first maps a monocular image to multi-view image using pretrained diffusion model and then the multi-view images construct textured mesh [67]. We bypassed the first part of the pipeline and sent the multi-view images displayed in Figure (top row) directly the InstantMesh model, taking care to match the expected preprocessing steps and formats exactly. Figure [ shows three views of the predicted mesh, which not recognizably mouse. We suspect that InstantMesh is unable to reconstruct meaningful shape because of the tightly controlled range of camera parameters seen by the model during training. Specifically, the camera positions are equidistant from the object centers, which does not apply to images of moving animal relative to stationary cameras. This design choice is reasonable for InstantMesh, which trained on easily-man maniilllale synthetic 3D data, but has disadvantages for translating to messier real-world data, even in relatively well-controlled laboratory conditions.
G.4 BANMo
We also tested BANM 669 a well-known model for monocular 3D animal reconstruction. BANMo is template-free free approach that reconstructs animatable 3D models from monocular videos through per-video optimization. However, each sequence requires separate optimization, making the method computationally expensive and unsuitable for large-scale datasets. Moreover, its reliance single- view input limits reconstruction accuracy, especially for occluded or rarely visible body parts. Figure I illustrates that BANMo struggles to reconstruct accurately under unseen views.
G.5 VGGT & AnySplat
We tested two recent generalizable, feed-forward 3D reconstruction models, VGGT (60 and AnyS- plat [23), to check whether large pre-trained models could outperform complement our lightweight model.
VGGT reconstructs point clouds and depth maps from multi-view inputs but does not produce renderings. Due to the absence of ground-truth depth or 3D meshes in our dataset, quantitative evaluation not feasible. Qualitatively, VGGT failed to produce coherent 3D surfaces animal data, which features minimal overlap between views. Instead, it generated misaligned, layered point clouds, characteristic "onion-peel" artifact also visible in elongated parts such as tails. The result shown in Figure 12
AnySplat, which uses VGGT as its geometric backbone for Gaussian Splatting, exhibited similar issues as shown Figure While supports direct rendering and quantitative evaluation, its zero- shot performance on our dataset was significantly lower than Pose Splatter. Fine-tuning AnySplat yielded only marginal improvements across all metries (IoU, I11, PSNR, SSIM) and did not resolve the misalignment artifacts. The quantitative result is displayed in Table 7
21

---

GT
Rotated camera views
Figure 12: Due to inaccurate camera estimation, VGGT produces point cloud that shows an "onion- peel" layered appearance when reconstructed.
GT
GT
GT
GT
Figure 13: Similar to the results of VGGT (Figure 12 AnySplat exhibits an "onion-pede peel"" artifact and produces inaccurate reconstruction results.
These findings indicate that current generalizable models like VGGT and AnySplat struggle to handle sparse, low-overlap multi-view animal data. In contrast, lightweight model trained from scratch, such as Pose Splatter, provides more accurate and efficient reconstructions under these challenging conditions.
H Additional Renderings
Figure 14 shows randomly sampled test set renderings for the three 6-camera models from observed views.
Figure 14: Representative renderings from observed views.
Figure 15 shows randomly sampled test set rendering for the three 6-camera models from unobserved views.
Despite the ve-voxelization step in Pose Splatter, we observed regular grid-like patterns in many renderings. For the mouse and finch datasets we trained models with twice the spatial resolution in the voxelization step, resulting in 8 times as many voxels. As shown in Figure [6] we see a similar visual quality but apparent regular grid patterns.
I Quality of 3D Keypoints
After training SLEAP model 48 to predict 2D keypoints from images, we perform robust triangulation across views using the known camera parameters estimate 3D keypoints. To assess the quality of these 3D keypoints, we calculated the distribution of reprojection errors (in pixels) for
22

---

Dataset
Views
Model
Setting
IoUt
L11
PSNR
SSIM
AnySplat
Zero-Shot
0.403
1.132
25.1
0.952
5
AnySplat
Tune-Tuned
0.431
1.088
25.5
0,959
Pose Splatter
0.760
0.632
29.0
0.982
Mouse
AnySplat
Zero-Shot
0.392
1.227
25.6
0,961
AnySplat
Tune-Tuned
0,415
1.051
26.3
0,965
Pose Splatter
0.721
0.753
28,2
0.982
AnySplat AnySplat
Zero-Shot
0,365
1.359
24.7
0.968
5
Fine-Tuned
0,412
1.174
25,4
0.971
Pose Splatter
0.848
0.345
34.5
0.992
Finch
AnySplat AnySplat
Zero-Shot Tuned
0.421
1.057
24.2
0.962
4
0,459
1.012
24.9
0.967
Pose Splatter
0.731
0.685
29.0
0.981
Table 7: Quantitative comparison between AnySplat and Pose Splatter on Mouse and Finch datasets with cameras.
Figure 15: Representative renderings from unobserved views.
Regular Models
Large Models
Figure 16: Visual comparison between regular and large shape carving volumes. The renderings on the left come are from models using "standard" volume size of 96 80 x 64 for the mouse and 96 x 64 x 80 for the finch (the volume sizes used in the rest of the paper). The renderings on the right come from models that 192 x 128 x 160 for the finch. Note that the regular grid artifacts seen on the left are not visible on the right. Best viewed zoomed in.
larger volume size of 192 160 128 for the mouse and
both the mouse and finch datasets. As seen in Figure 17 we find higher quality 3D keypoints for the
23

---

finch dataset. We believe this results from the higher quality 2D keypoints for this dataset. This may also explain the better performance predicting 3D keypoints from visual embeddings on the finch dataset (Figure 5 left). Representative examples of SL.AAP- 2D keypoints and reprojected 3D keypoints are shown Figure
3D Keypoint Reprojection Errors
50
Finch 3D Keypoint Reprojection Erturs
40
30
Reprejcttonn
mise
our
ear right
new froet fort
naw frent right
pow rear foft
paw 1603
back
tas Base
tall middle
tai end
latt
urte right
fall blass mitdle
midsle
fost
tad
Knte Aef
rpiit
Figure 17: Summary of per-keypoint reprojection errors for a) mouse and b) finch datasets. Violin plots for each keypoint show the distribution of errors between SLEAP- -rreeceddd 2D keypoints and the 2D projections of per-frame robustly triangulated 3D keypoints. Units are Euclidean distance in pixels, for full 1536 x 2048 images. Median errors are marked.
50
50 px
Figure 18: Frames showing CLEAA--predicte 2D keypoints (blue) and corresponding projected 3D keypoints (red). Black lines connect corresponding 2D and 3D keypoints and 50-pixel bars are shown for scale. Note the high accuracy of the triangulated finch keypoints (left) and some 2D keypoint errors for the mouse pose such as front right versus back left paw (right).
J Survey
To ensure transparency and enable replication of the nearest neighbor survey, with results presented in Figure 2b, provide the survey instructions and illustrative sereenshots of the user interface. The complete wording of the participant instructions is reproduced verbatim below, while Figure 19 depicts the layout of an individual survey question.
Please read these instructions carefully before beginning the survey.
In this study, you will judge how closely two candidate poses resemble the body posture of reference ("query")) mouse.
Each question laid out in columns: the left column shows the query pose, while the center and right columns show Option and Option 2. Every column contains two synchronized camera views (top and bottom) captured the same instant, giving complementary angles single pose.

---

Your task is to decide which option or 2 better matches the query. Focus only on the configuration of body parts such as the head, torso, limbs, and tail, and disregard background, lighting, or color differences. Beneath the images are two radio buttons; click the button under the option you believe is closer the query.
There are 40 questions in total, and there is no time limit. By continuing, you confirm that you consent to your anonymous responses being used for academic research on animal-pose representations.
Question 1
Query
Option 1
Option
Option
Option
Figure 19: Example survey question
25

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
All claims in the abstract and introduction are demonstrated in the experiments section.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss limitations such as the requirement of several camera views and the inability of the model handle severe occlusions.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
26

---

Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The method description in the main body and appendices are sufficient to reproduce the experimental results. We also provide project code on GitHub and publicly release the mouse and finch datasets to facilitate experimental reproduction.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (c.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
27

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Code implementing the method has been publibly released GitHub: https //github. com/ inckgoofinet//oosespppppp
The mouse and finch datasets have been publicly released: httpss://dol. 1ttp:///111.99////2/33222
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
httpeeeseeeooeeebbbaasssoooooeoo for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
y /nips. /nips.cc//ppppp http:ppbliccggideeeeeeee
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question:
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
These details are presented in the main text and a supplementary section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: A single statistical claim is made (the human evaluation of nearest neighbors) and the necessary details are clearly stated.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
28

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: These details are reported supplemental section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer:
[Yes]
Justification:
The paper conforms all aspects of the ethics guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
This work has no immediate societal impacts, positive or negative.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
29

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not pose these risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The Rat7M and 3D-POP datasets are properly credited and the CC BY 4.0 licenses are noted.
Justification:
Guidelines:
The answer NA means that the paper does xnot use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
30

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/dataseta
datasets.papersstt
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [Yes]
Justification: The 6-camera mouse and finch behavior datasets have been publicly released, along with descriptions of the data: Ittts:://doot httpddoi..rgg/go 10. 700.79924//4 r42323ke
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification:
We have included this in Appendix.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (TRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: While this paper does have human preference study, this is not considered research with human subjects by our IRB.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
31

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were not used in essential or non-standard ways.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
32


Reviews:
[{"review_id": "a92t2FRL9B", "reviewer": "Reviewer_peFs", "summary": "The article presents a novel approach, pose splatter, for computing 3D pose and appearance using shape carving and 3D Gaussian splatting. The method is novel in terms of predicting 3D pose without knowledge of animal body geometry or manual annotation. The paper also introduces a novel technique for encoding pose, which is designed to replace keypoint based pose information. This step will help in behaviour analysis based on pose of the animal. The method performs better than state of the art in terms of capturing all the subtle details of the pose, at the same time the pose computation time is reduced because per-frame optimization is not needed, similarly the method is helpful for large scale deployment because annotation of keypoint is not necessary.", "questions": "- The authors should explain limitations very clearly (refer weakness above and limitations section below)\n- There are 3 multi-view datasets (not cited) with ground truth on 3D posture, (ref wekness) why they are not suitable for this approach?\n- The metrics have to be clearly explained in the paper, in a way that a biologist would be able to use the method, if the method is intended for use of biologists. Anyhow, current effort makes it unclear for any reader even in the AI community.\n- If there are camera limitations or volume limitations, what type of experiments would be suitable for this method or currently the method is an improvement over existing method but not ready to be deployed or tested in context of behavior experiments.\n- While pose estimation is requirement for modern experiments, all datasets used are with 6 cameras, please explain if this is common setup for neurological experiments? Currently, the setup looks customized for the experiment. This is not a limitation but clarification would be helpful for readers with non-biology background.\n- The two datasets seem to be introduced in the paper, or they are publicly available? If not, do authors intend to release them? If yes, the datasheet for dataset (a common practice at NeurIPS datasets) should be compiled and added, in order to have clarity over how the data is collected and what is it’s relevance to proposed downstream applications.", "limitations": "The authors do not have a clear limitation section, this makes it difficult to understand the limits of the given method e.g. how many cameras are best to use, does the method work with more than one animal (it is fine if it does not but still good to know what authors think), does it work with sequences in the wild?, Why existing multiview datasets are not sufficient? \n\nThe authors wrote in the checklist that camera based limitations are mentioned but there is not clear statement about it in the paper. Please clarify these details since they are important for practical consideration. \n\nAlso, the claim that keypoint based methods require intensive labor is true but if the existing method required X no of cameras, where as classical stereo methods can do the job with 2 cameras the users have a choice to make, especially in wild settings where multiple calibrated cameras is a problem. This is also a problem where animals have to be studies in a large volume, if the dependecies of method is in a certain volume it has to be mentioned. Currently, all of these details remain a bit unclear.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength : \n\n- The paper picks up on an interesting problem and it is challenging, animal posture computation is highly sought after tool for automating behavior experiments. Posture is important information for extracting behaviors or activities.\n- The proposed method brings as new dimension to 3D pose estimation of animals, traditionally used with either keypoint triangulation or mesh based approached. This approach goes away from it, which simply makes it easier for researchers to deploy the solutions without providing initial information or worrying about species specific geometry.\n- The authors have tested the method with multiple datasets, and show the power of method in terms of accuracy in prediction of behavior and show that it is an improvement over keypoint based methods in most cases for behavior prediction.\n- The authors have captured related work for the most part and explained the method in detail.\n\nWeakness: \n\n- The introduction is a bit unclear about following aspects in terms of being specific, the number of cameras required for the approach is not mentioned ( “A small set of calibrated cameras”).\n- While related work in the field is mentioned extensively, including the outdoor / wild applications, there are three datasets which are fitting for the purpose but not mentioned or covered 3D-POP (pigeons in indoor), 3D-MuPPET( pigeons in wild) and 3D-SOCS (great tits in semi-wild). Both contain 3D, keypoint based pose for birds (pigeons, tits) with multi-view cameras. The sequence also shows one or more birds, these two datasets seem to be ideas for such method since both have multiple cameras. The authors have not mentioned them, moreover, it would be interesting to know if the method can be applied to these datasets since all of these are specifically designed for behavior studies with 3D posture (i.e. gaze, etc.), also some are validated with motion capture with very high accuracy. Are these not suitable because of camera requirements of the proposed method? \nThe same with OpenMonkeyStudio (cited by authors), it is unclear why these datasets are not useful?\n- The metrics used for evaluation of the results are not explained clearly. Only acronym is mentioned, this is assuming that all readers of the paper have in depth knowledge of the field. Keeping the broad scope of the paper explanation is necessary for a biologist in need of using the method in future. Also, some metrics e.g. R2, are not explained, unclear what the term.\n- In addition, it would be helpful to specify how is the 3D position measured, a basic 3D prediction would mean XYZ based result, it is unclear to understand what is the final format of the result. This is crucial for users because in terms of behavioral experiments understanding pose sequence for behavior is one aspect but real world movement is also necessary in order to quantify movement of the animal in space (unless it is a head fixed animal), it is unclear if this can be obtained from the method?\n- It seems the method, may not support multiple individuals at the same time. This is something crucial for multi-species experiments (indoor or outdoor), deserved to be mentioned in limitations (which are not explicitly detailed in the paper)", "was_revised": true, "final_justification": "The authors did sufficient work to resolve my queries. The editing of text is something we have to take their word on it but they have demonstrated that their work does scale to other experimental scenarios relevant for fundamental science within and outside domain of neurology. Involving such methods is great for NeurIPs community as such. \n\nAddition of the pigeon dataset made the paper much broader. Event if results are not great, camera based setups are more commonly being used commonly for behaviour studies. In this context, 3D pose is becoming much more crucial for high resolution behaviour analysis. From this perspective, I believe the paper must be given support.  \n\nI have not updated the score because the impact is strong on application domain than technical domain but I still maintain that reasons to accept are more than rejecting, the field of application is extremely important and difficult subject of animal pose does deserve more attention.", "rebuttal": "Thank you for the detailed feedback and for highlighting the key strengths of our work. We appreciate your comment that we are tackling an important and challenging problem, and that our method offers an easier-to-adopt alternative to traditional keypoint or mesh approaches without requiring any annotations or species-specific information. Thank you also for noting our detailed explanation and the thorough validation across multiple datasets. We have addressed specific points in your review below.\n\n**Number of Cameras:** You are right that our use of the phrase “a small set of calibrated cameras” in the introduction is vague. While we provide the specifics in the experiments section, we will clarify in the introduction of our final version that our method was tested using 4 to 6 cameras and that we achieve best results with 5 or more cameras, as seen in Figure 5b.\n\nWe chose a 6-camera configuration because it aligns with the standard established by Rat7M, which is one of the most widely-used benchmarks for this task. A 6-view arrangement is a common standard in modern neuroscience for achieving precise 3D kinematic tracking (e.g. Weinreb, C., Pearl, J. E., Lin, S., Osman, M. A. M., Zhang, L., Annapragada, S., ... & Datta, S. R. (2024). Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics. _Nature Methods_, 21(7), 1329-1339.). We will add a sentence to the paper to provide this important context.\n\nRegarding two-camera stereo methods, our experiments with state-of-the-art models like PixelSplat and MVSplat highlight their core limitation for this task: they require substantial overlap between views to function effectively. This makes it difficult to reconstruct a complete 3D shape, as large portions of the animal's body are often occluded from any given pair of cameras. As our results show, their performance degrades significantly on our datasets where view overlap is sparse. Figure 12 further demonstrates that their performance does not improve even when given 3 or 4 views.\n\nFinally, regarding the practicality for large-scale experiments, a 4-6 camera system does not present a major barrier in a laboratory setting. Once the cameras are calibrated and fixed, they can collect longitudinal data for long durations and across different subjects with minimal effort. In outdoor or wild contexts, however, setting up this number of calibrated cameras may be prohibitive. We will add the following text to describe these limitations:\n\n> _Pose Splatter_ requires a minimum of roughly four calibrated cameras to operate, which may hinder its use in some outdoor or wild settings.\n\n**Multi-Animal Support:** You've correctly identified that our current model is designed and benchmarked for single-animal scenarios. As we note in the discussion, handling the \"prolonged and severe inter-animal occlusions\" in multi-animal scenes is a significant challenge that we have not yet benchmarked. While we mentioned this briefly as an area for future work, we will state it more explicitly in a limitations section in our revised manuscript:\n\n> Additionally, the model has no mechanism to handle occlusions of the animal, which can occur in complex environments and multi-animal scenarios.\n\n**Metrics (L1, IoU, PSNR, $R^2$, etc.):** We apologize for not defining the evaluation metrics more clearly. We omitted detailed explanations as they are standard in 3D reconstruction literature, but we agree that providing them would make the paper more accessible. We will add a new subsection to the revised manuscript defining each one.\n\nFor your convenience, we provide a brief overview of the benchmark metrics used in our evaluation tables. To compare performance against other methods, we use four standard metrics. The **L1** metric measures the average absolute pixel-wise difference between our rendered image ($\\hat{x}$) and the ground truth ($x$), evaluating color and texture accuracy, where a lower value is better. The **IoU (Intersection-over-Union)** score evaluates shape accuracy by calculating the overlap between the rendered silhouette ($\\hat{m}$) and the ground-truth mask ($m$), where a higher score is better. We also report **PSNR** (peak signal-to-noise ratio) and **SSIM** (structural similarity index measure), which are standard image quality metrics where higher scores indicate a better, more perceptually similar reconstruction. It is worth noting that the L1 and IoU metrics also form the basis for our color loss ($L_{color}$) and silhouette loss ($L_{IoU}$), respectively, which are minimized during training.\n\nIn our keypoint prediction experiments, we use the $R^2$ score, or the Coefficient of Determination. This metric measures the proportion of variance in the 3D keypoint coordinates that is predictable from our visual embedding. While the interpretation of what constitutes a ``good'' $R^2$ score is highly context-dependent, for complex and noisy biological data like animal behavior, a score above 0.5 generally indicates a strong predictive relationship. As shown in Figure 5 (left), our visual embedding proves to be a strong predictor for the animal's pose. For the finch, the $R^2$ values are consistently high (most > 0.75), indicating that our annotation-free embedding has captured the pose information almost as well as the manually derived keypoints themselves. For the mouse, we also see strong predictive power for most keypoints, demonstrating that our method successfully encodes quantifiable information about the animal's posture. This result is significant because it shows our embedding can serve as a rich, quantitative replacement for labor-intensive keypoint annotation in downstream analyses.\n\n**Other multi-view Datasets (3D-POP, 3D-MuPPET, 3D-SOCS):** Thank you for suggesting these valuable datasets. It seems like the single pigeon videos from 3D-POP could be a valuable addition to the paper. We also agree that adding experiments using the OpenMonkeyStudio data would be useful. We have begun processing this data and hope to add updates during the Author/Reviewer discussion period.\n\n\n**3D Position Output:** As stated in our methods section, the direct output of our network is a collection of 3D Gaussian particles representing the animal’s complete shape and appearance, which can be projected onto a 2D plane to create a rendered image from any given camera view. From a practical standpoint, the mean of each of these individual particles provides a real-world XYZ coordinate, and the full set of these points forms a representation of the animal's 3D shape that can be used to quantify movement.\n\nOur focus in the paper is to quantify the pose of the animal without reference to animal location or orientation (azimuthal angle), both of which are estimated in a pre-processing step described in Appendix A. _Pose Splatter_ uses these estimated values to produce predicted real-world 3D positions of the Gaussian particles that make up the animal.\n\n**Dataset Release and Documentation:** You are correct that the mouse and finch datasets are new contributions of this work. We are fully committed to open science, and as stated in the paper, the datasets will be released upon publication. We will include videos, mask videos, camera parameters, estimated 3D center and azimuthal angles, visual embedding time series, example code, and documentation. We are planning to release the data on the DANDI archive in Neurodata Without Borders (NWB) format. The DANDI archive is data repository geared toward enabling reproducible science within the neuroscience community, where each dataset is assigned a DOI."}, {"review_id": "MDnyWLKBtI", "reviewer": "Reviewer_P9fo", "summary": "This paper proposes Pose Splatter, which leverages shape carving and 3D Gaussian splatting to model animal pose and appearance.  Specifically, the foreground masks for multi-view images are back-projected to generate a rough shape estimation, followed by a stacked 3D U-Net to refine the coarse volume and a MLP to predict Gaussian parameters.  The whole network is trained with image based and mask based losses. Moreover, a visual embedding technique is also introduced to provide descriptors of pose and appearance for downstream analysis tasks.", "questions": "Please refer to the weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The paper tackles an interesting direction of leveraging 3D GS to model animal pose and shape information for downstream analysis. \n2. The proposed method works well on the mouse and finch dataset, and achieves cross-category generalization \n\nWeaknesses\n1. The technical contribution of the proposed method is limited. The pipeline consists of shape-carving based coarse estimation and 3D U-Net based refinement, these are quite standard and lack novelty\n2. For quantitative comparison, the proposed Pose splatter only compares with PixelSplat and MVSplat, which are designed for general scene. More relevant baselines such as generalizable human gaussian [A, B] should be included in the comparison. Moreover, both PixelSplat and MVSplat uses two views, which is a bit unfair.\n\nA. Junjin Xiao et al. RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Images. CVPR 2025\n\nB. Youngjoong Kwon et al. Generalizable Human Gaussians for Sparse View Synthesis. ECCV 2024.\n\n3. The description of both the visual embedding technique and the corresponding experimental design is not very clear.", "was_revised": true, "final_justification": "I raise my score to borderline accept after discussing with other reviewers. The authors are encouraged to claim more on the contribution to the animal behavior domain instead of technical contribution if the paper gets accepted.", "rebuttal": "We thank the reviewer for the feedback. We are glad you found the direction of our work interesting and noted our strong performance on the animal datasets, including cross-category generalization.\n\nWe understand your concerns regarding technical contribution, experimental comparisons, and clarity. We believe these points may stem from a misunderstanding of our paper's core objective and the specifics of our evaluation. We would like to clarify these points below.\n\n**Standard Components:**\nWe respectfully disagree with the assessment that our work has limited technical novelty. While the individual components like shape carving and U-Nets are established, the core innovation of our work lies in their integration into a new, end-to-end framework that solves a problem existing methods do not: the annotation-free 3D reconstruction of diverse animals. Our primary contribution is not a new network layer, but a complete system that works \"without prior knowledge of animal geometry, per-frame optimization, or manual annotations.\" This is a fundamental departure from methods that require species-specific templates or costly per-scene optimization. Additionally, the standard components in our work enable a fast, lightweight, high-performing solution that requires only 2.5 GB VRAM, compared to about 20 GB VRAM for PixelSplat and MVSplat. Furthermore, we introduce a novel pose quantification tool—the visual embedding—which we demonstrate outperforms traditional 3D keypoints in both human preference studies and downstream behavior prediction tasks. The other reviewers (peFs, r2qH) recognized the novelty and impact of both our overall system and the visual embedding, affirming that it is a significant contribution to the field of animal behavior analysis.\n\n\n**Choice of Comparison Baselines:**\nWe would like to clarify our choice of baseline comparisons, as this seems to be a point of misunderstanding. The suggested baselines, such as RoGSplat and Generalizable Human Gaussians, are explicitly designed for human reconstruction and rely heavily on human-specific priors like the SMPL body model, which are unavailable for our animal subjects. In fact, to even attempt a comparison, one must first check if it's possible to fit SMPL parameters to the animal data. We tried this, but the SMPL model failed to converge on our animal datasets, optimizing into distorted shapes that did not reflect the animals' true forms. Also, the central premise of Pose Splatter is its ability to work without any species-specific templates; therefore, comparing our general animal model to a human template-based model would be an inappropriate comparison. We chose PixelSplat and MVSplat precisely because they are generalizable, feed-forward models not tied to a specific category. While these models are generally used for reconstructing whole scenes, they are also known for reconstructing individual objects in the scene well and represent the current state-of-the-art for feed-forward, generalizable 3D Gaussian splatting.\n\nAlso, regarding the use of two views for these baselines, this was not an arbitrary or unfair limitation. We were following the original authors' recommendations, as stated in our experiment result section (Section 4.3.): \"In line with each author's recommendations, we trained both PixelSplat and MVSplat with two input views.\" Furthermore, our experiments in Appendix G show that providing these models with more views did not improve their performance. These models are designed for high-overlap stereo pairs and struggle in sparse-view settings with minimal overlap like ours, regardless of the number of input views, as stated in Section 4.3.\n\nWe respectfully ask the reviewer to consider that our paper's primary goal is not to achieve a new state-of-the-art performance in general 3D reconstruction, but to propose a novel and practical system for animal pose quantification. The model comparison serves as a reference point, but our main contribution is twofold: first, achieving annotation-free 3D reconstruction (a task not addressed by previous animal pose models) and second, introducing the visual embedding as a new tool for behavioral analysis that outperforms sparse keypoint methods. \n\n**Unclear Description of Visual Embedding:**\nAs other reviewers did not raise this concern, we would be happy if you could point to any specific areas of confusion so we can address them directly in our revision. In the meantime, we hope a concise summary below helps clarify our method, which is also detailed across the main text and appendices.\n\nThe process involves rendering images of the 3D model from 32 virtual cameras on a sphere, passing them through a pre-trained ResNet-18 encoder to get 512-dimensional features, applying a Spherical Harmonic expansion and taking the squared magnitude of the coefficients to achieve rotation-invariance, and finally using adversarial PCA to produce the final 50-dimensional embedding. The experiments validating this embedding, including the human preference study and the behavior prediction task, are detailed in Section 4.3, with additional specifics in Appendices C and H."}, {"review_id": "u2wH3W9ZJk", "reviewer": "Reviewer_mWtm", "summary": "This paper proposes Pose Splatter, a method that leverages shape carving and Gaussian splatting to reconstruct laboratory animals, such as mice. The authors extract rotation-invariant visual embeddings based on Pose Splatter for animal behavior analysis. Experiments demonstrate the reconstruction capability and validate that the extracted visual embeddings can be used to understand animal behaviors.", "questions": "Please see weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. The paper is well-structured and easy to follow.\n2. The idea of extracting visual embeddings based on a reconstruction model for animal behavior analysis is reasonable and well-motivated. The pipeline is clearly constructed and logically justified.\n\nWeaknesses\n1. The authors primarily compare their method with scene-level reconstruction approaches, such as MVSplat and PixelSplat. However, since the focus is on modeling individual animals, it would be more appropriate to include comparisons with object-level feed-forward models, such as GRM [1], TRELLIS [2], and HunYuan3D [3]. These models do not require species-specific templates and are known for their strong generalization capabilities.\n\n2. The dataset used in this study is quite limited, incorporating only two types of animals: mice and finches. This raises concerns about the generalizability and overall effectiveness of the proposed method. Moreover, the observed improvement over the per-scene optimization method (GO) appears marginal, which further questions the necessity and impact of employing a feed-forward model in this setting. \nExisting feed-forward models are typically designed for large-scale datasets and are valued for their strong generalization capabilities across diverse samples. Given the limited data available in this study, it is worth carefully reconsidering whether training a feed-forward model from scratch is truly justified. \nIn fact, leveraging the priors inherent in well-established base models may yield better performance and efficiency than training a new model from scratch on such a small dataset. Additionally, expanding the dataset to include a broader range of animal species would substantially strengthen the validation of the proposed approach and better demonstrate its potential for wider applicability.\n3. For understanding animal behavior, action recognition models that process video or image sequences are also widely used. The paper does not clearly explain the advantages of the proposed method over these established action recognition approaches. Including additional comparative experiments or analyses would be helpful to clarify the unique contributions and practical benefits of the proposed method in behavioral analysis.\n\n[1] GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation\n\n[2] TRELLIS: Structured 3D Latents for Scalable and Versatile 3D Generation\n\n[3] Hunyuan3D: https://github.com/Tencent-Hunyuan/Hunyuan3D-2", "was_revised": true, "final_justification": "In the first stage, my major concerns are:\n\n1. Comparison with object-level reconstruction methods (InstantMesh, TRELLIS).\n\n2. Limited improvements compared with GaussianObject.\n\n3. Comparison with action recognition models.\n\n4. Comparison with feed-forward methods based on VGGT.\n\n5. Limited training and testing data, restricted to mice, pigeon, and finches.\n\nAfter the rebuttal, concerns 1–4 have been mostly addressed. My remaining concern is the limited data. \nThe authors say they will release this dataset as part of their contribution, which I consider valuable given the difficulty of obtaining such data.\nAs a result, I am inclined to raise my score to 4.", "rebuttal": "We sincerely thank the reviewer for the positive feedback on our paper's structure, motivation, and logical pipeline. We are glad you found the idea of using visual embeddings for behavior analysis to be reasonable and well-motivated.\n\nWe would like to address the weaknesses you raised, as we believe they may stem from a misunderstanding of our work's primary task, goals, and contributions.\n\n**Comparisons to Object Models (GRM, TRELLIS, HunYuan3D):**\nThank you for suggesting these powerful recent models. Unfortunately, the code for GRM has not been publicly released, making a direct comparison impossible. Regarding TRELLIS and HunYuan3D, while they are indeed powerful, they are designed for a fundamentally different task. These are generative models trained on massive datasets of 3D objects to create novel 3D objects from inputs like **text or a single image**. In contrast, our method is a reconstruction model designed for scientific applications that require the model to _accurately_ recover the specific pose and appearance of an animal from a sparse set of views, _where no ground truth 3D structure is available_.\n\nDespite this task mismatch, we took your suggestion and tested these models on our data. The results highlighted a critical issue: these models tend to hallucinate occluded parts rather than reconstruct them. For instance, they might invent a symmetrical backside for an animal when it's not visible in the input view. This leads to the generation of a plausible 3D animal, but not an accurate reconstruction of that animal's true size, shape, and posture in that specific moment. Because our goal is the precise 3D reconstruction needed for scientific behavioral analysis, this tendency to invent data makes these otherwise powerful generative models unsuitable for our specific application.\n\n**Comparison with GaussianObject:**\nRegarding the comparison with GaussianObject (GO), while our method does not offer a dramatic improvement in final rendering quality, the motivation for our feed-forward approach was never about marginal gains in image metrics; it was about enabling scalability for large-scale scientific analysis. As we report in the paper, GO requires roughly one hour of optimization per scene, while our method's inference time is just 30 milliseconds—a speedup of over 100,000 times. For the behavioral neuroscience domain, where datasets often contain hundreds of thousands of frames, this difference is critical. Per-scene optimization is computationally prohibitive at this scale, making a fast, feed-forward model the only viable path for practical, large-scale analysis.\n\n**Training from Scratch vs. using a Pre-Trained Model:**\nWe appreciate the reviewer’s suggestion to use a pre-trained model, but a suitable “well-established base model” for our specific task does not yet exist. The models mentioned, such as TRELLIS and HunYuan3D, are designed for generating objects from single views or text and are not optimized for reconstruction from the kind of sparse, low-overlap multi-view data we use. In the absence of a pre-trained foundation model for our task, training from scratch is the only viable option. \n\nFortunately, our model is computationally lightweight, requiring only about 2.5GB of VRAM and a few hours of training time per dataset, making this approach highly accessible to researchers. Despite this efficiency, our model demonstrates strong cross-species generalization, as shown in Figure 3a and our quantitative results in Section 4.3. We therefore believe our approach is a practical and effective solution until large-scale foundation models for this specific task are developed.\n\nRegarding the request to add more species beyond the three in the current work, we agree this would strengthen the paper and it is a goal for future work. As suggested by reviewer peFs, we are currently working to apply _Pose Splatter_ to additional multiview videos with pigeons and monkeys. However, we wish to emphasize that publicly-available synchronized, multi-view data of freely behaving animals is relatively rare. We believe our current contribution of two new datasets that will be released publicly, is already a significant and impactful contribution to the field.\n\n**Video vs. Image Models**\nThis is an insightful question that gets to the heart of our contribution. Traditional action recognition models are fundamentally temporal: they process a sequence of frames to classify an action that unfolds over time. In contrast, our method is static: its primary function is to create a high-fidelity 3D representation of an animal's posture at a single, specific timestep. The advantage of our approach is the superior quality of this per-timestep representation, which serves as a better building block for any downstream temporal analysis. Additionally, the static model gives us the flexibility to perform filtering or smoothing as required by specific scientific questions.\n\nTherefore, the unique contribution of our work is not to replace action recognition models, but to provide them with a richer, more informative input. By accurately describing **how** the animal is posed at every moment, we enable more powerful downstream models to determine **what** action is occurring over time."}, {"review_id": "BMYrSgseaw", "reviewer": "Reviewer_r2qH", "summary": "This paper proposes a method for estimating the 3D pose and appearance of subjects from a sparse set of views, with a focus on animal pose estimation. The approach leverages Gaussian splatting to model the 3D geometry of animals, enabling efficient feed-forward rendering of novel views. The authors introduce a low-dimensional embedding to jointly encode pose and appearance. They evaluate their method on three datasets (mice, rats, and zebra finches) demonstrating improved performance over existing Gaussian splatting techniques for 3D reconstruction and rendering, as well as over 3D landmark tracking for behavior analysis.", "questions": "Please address (1)-(4) above.", "limitations": "Fine.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "I like this paper. It tackles an interesting and challenging problem and proposes a method that seems well-suited for lab setups aimed at studying animal behavior and has potential for meaningful positive impact. The paper itself is very well written and organized, and overall a pleasure to read. The pipeline is explained clearly, with both motivation and technical detail provided for each component. The work appears reproducible, and the authors provide code (which I haven’t tested) that will be made publicly available, along with two of the three datasets used in the experiments.\n\nBelow are a few concerns I would encourage the authors to address in their rebuttal, and which I believe could be resolved with a minor revision:\n\n*(1) Overstatements:*\nSome claims in the paper strike me as overstated. For example, the sentence “Pose Splatter enables analysis of large-scale, longitudinal behavior needed to map genotype, neural activity, and micro-behavior at unprecedented resolution” seems exaggerated. While improvements are demonstrated clearly, “unprecedented resolution” is misleading. The summary in lines 343–349 is more balanced and appropriate.\nSimilarly, claims of high-quality or high-fidelity reconstruction are made repeatedly. While the method improves upon other GS-based techniques, the visual results still appear relatively coarse due to low-resolution visual hull estimates and the inherent roughness of Gaussian splats, especially when compared to reconstructions using higher-complexity 3D representations.\nAnother example concerns the claimed position and azimuthal invariance. Although these properties are observed, they stem from standard PCA alignment and the phase-invariance of norm in the embedding. I recommend that the authors tone down such statements to more accurately reflect the technical contributions.\n\n*(2) Pose Estimation:*\nI have some reservations about the use of the term “pose.” In the context of animal studies, pose estimation typically involves a semantic understanding of how body parts relate to one another. The proposed method does not offer such semantic structure. Rather, it performs implicit 3D reconstruction and enables novel view synthesis. The embedding computed from these reconstructions encodes a combination of pose and appearance features that are shown to be effective for downstream tasks. However, it does not amount to pose estimation in the traditional sense. For some applications, this limitation may reduce interpretability or fail to capture the specific information needed (e.g., a small set of body landmarks might be more suitable in such cases).\n\n*(3) Specificity to Animals:*\nWhat exactly makes this approach specific to animals? Although the paper is framed in that context, I was left wondering whether the method is in fact general and could apply just as well to other settings, such as human subjects or other objects, where methods like 3DGS, FSGS, or GO have been successfully used. If the method is truly general, it would be useful to clarify this. If there are aspects that make it particularly well-suited for animals, those should be highlighted more explicitly.\n\n*(4) Limitations and Practical Use:*\nAlthough the paper mentions limitations in the conclusion, I am concerned that the method may be restricted to sterile, highly controlled environments, limiting its applicability to real lab settings. The paper does not provide enough detail on practical conditions required for the approach to work: How many views are needed? What lighting conditions are assumed? How must the cameras be arranged spatially? I encourage the authors to elaborate on these aspects to help readers understand the practical constraints of deploying the method.\n\n---\n\n*Additional minor issues and questions:*\n\n* *Line 129:* Why exactly three U-Net modules? Please clarify.\n* *Line 159:* Is the training loss computed over all input views, or only a subset? Please clarify.\n* *Figures:* Some figure elements (especially 2(b)) are not explained until much later in the paper and feel out of context. Consider reordering or revising.\n* *Line 177:* What \"pretrained convolutional encoder\" is used?\n* *Lines 191–196:* This description is vague; more detail would help, even if included in the appendix.\n*Line 219:* To better support the claimed advantage that \"our model uses only about 2.5GB of GPU memory (VRAM),\" it would be helpful to contextualize this number by referencing the memory requirements of competing models.\n* *Figure 5 (left panel):* The content is unclear, and I couldn’t interpret what it shows. Please explain.\n* *Lines 305–307:* \"Example queries and answers are shown in Figure 2b, demonstrating the high degree of similarity produced by both feature sets.\". The example queries and corresponding KP NN and VE NN are all quite different and I find it hard to evaluate the quality and conclusions of this experiment.\n* *Lines 321–322:* The process of predicting \"egocentric 3D keypoints from the visual embedding using a 5-nearest neighbor regressor\" is not clear, please provide more details.", "was_revised": true, "final_justification": "I’m positive about this work and would be happy to see it published with the revisions proposed by the authors. The authors provided comprehensive answers to most concerns raised in my and other reviews.", "rebuttal": "We are grateful for your thoughtful, detailed, and positive review. Thank you for the encouraging feedback. We are especially pleased that you found the paper well-written and a pleasure to read, and that you recognized our work as a well-motivated and reproducible contribution to the study of animal behavior.\n\nYour concerns and suggestions are meaningful, and we agree that addressing them will improve the manuscript. We address each of your points below.\n\n**Overstatements:** \nThank you for this valuable feedback. You are right that some of our claims, such as \"unprecedented resolution,\" are overstated. Our intention was to highlight the potential for large-scale analysis enabled by our method's efficiency, but we agree the language should be more measured and precise. We have revised the last sentence of the abstract as follows:\n\n> By eliminating annotation and per-frame optimization bottlenecks, *Pose Splatter* enables analysis of large-scale, longitudinal behavior needed to map genotype, neural activity, and behavior at high resolutions.\n\nAdditionally, we have revised our description of the visual embedding in the abstract:\n\n> We also propose a rotation-invariant visual embedding technique for encoding pose and appearance, designed to be a plug-in replacement for 3D keypoint data in downstream behavioral analyses.\n\n\n**Pose Estimation:**\nWe agree that our method does not perform \"pose estimation\" in the traditional, semantic sense of identifying and structuring body parts. It is more accurately described as implicit 3D reconstruction, from which we derive a holistic descriptor of pose and appearance. We use the term \"pose\" in a broader sense to refer to the animal's overall physical configuration. In our revision, we will clarify this distinction, explicitly stating that our method provides an implicit or holistic representation of pose. \n\n\n**Specificity to Animals:**\nYou are correct that our core pipeline is, in principle, general and could be applied to other non-rigid objects. However, it was originally designed for animal behavior because its key strengths directly solve the most significant challenges in that domain. Specifically, our method is template-free, making it immediately applicable to diverse species that lack standard 3D models like those used in human-centric research. Furthermore, it is annotation-free, bypassing the major bottleneck of defining and annotating animal keypoints. Therefore, while the algorithm is general, and could probably be used for other types of objects, its impact is probably most pronounced in the animal domain. \n\n**Limitations and Practical Use:**\nThank you for these important practical questions. We agree that a clearer discussion of the method's settings is crucial for readers, and we will add a dedicated section on these considerations in our revised appendix. Our goal was to design a system that is robust and flexible enough for real-world lab settings. For the number of views, our method is adaptable to various multi-camera setups, performing well with a sparse set of 4 to 6 views. The approach is also robust to the kinds of non-ideal lighting found in many lab settings and does not require specialized studio lighting. This robustness is largely thanks to our reliance on a powerful foundation model (SAM2) for segmentation; because such models perform well in varied lighting, the prerequisite of a reasonable foreground mask is achievable under a wide range of typical lab conditions. Finally, the camera arrangement follows the standard best practice for any high-fidelity multi-view 3D reconstruction task: placing cameras to provide comprehensive coverage and minimize self-occlusion.\n\n**Additional minor issues and questions:**\n\n* **Three U-Net Modules (Line 129):**\nThe choice of three U-Net modules was determined empirically in early experiments. We observed that performance improved incrementally as we stacked more modules, probably due to the iterative refinement of the coarse visual hull, where each module improves upon the output of the previous one. We found that three modules offered a good trade-off between performance gains, memory consumption, and forward pass time.\n\n* **Training Loss Computation (Line 159):** To clarify, the training loss is defined between the rendered output and a single, randomly selected input view for each training sample. We will add a sentence to the \"Loss Terms\" section to state this explicitly in the revised manuscript.\n\n* **Figure 2(b):** Thank you for this suggestion. We will revise the caption of Figure 2 and the surrounding text to provide clearer context for the nearest-neighbor study upon its first appearance, so it does not feel out of place.\n\n* **Pretrained Encoder (Line 177):** We apologize for this omission in the main text. As detailed in Appendix C, the encoder is a pre-trained ResNet-18. We will add this important detail to the main text for clarity.\n\n* **Vague Embedding Description (Lines 191-196):** The section in the main text is a summary of the final step of our embedding process, with the complete technical details of the Spherical Harmonic expansion and adversarial PCA provided in Appendix C. We have revised the main text to offer a more intuitive explanation of the process, ensuring the core concepts are easier to grasp without needing to immediately refer to the appendix:\n\n> However, we found that these feature vectors still strongly encoded the azimuthal angle of the animal, possibly due to uneven lighting conditions across views. To remove this effect, we employ an adversarial formulation of principal components analysis (PCA) to find a 50-dimensional subspace of the feature vectors that contains a large portion of the variance of the input vectors and can predict only a small portion of the variance of the sine and cosine components of the azimuthal rotation angle. We take these 50-dimensional pose descriptors as the visual embedding. See Appendix C for more details.\n\n* **Contextualizing the 2.5GB VRAM Usage (Line 219):** This is a great suggestion to better highlight our model's efficiency. As we state in the paper, our model is very lightweight, requiring only about 2.5GB of GPU memory (VRAM). For context, competing feed-forward models like PixelSplat and MVSplat consume at least **20GB** of GPU memory, depending on the dataset and batch size, while per-scene optimization models such as Gaussian Object require over **10GB**. We will add these numbers to the revised manuscript to make the practical advantages of our efficient architecture much clearer.\n\n* **Figure 5 (left panel):** This panel shows the R2 (Coefficient of Determination) score for predicting the 3D locations of each individual animal keypoint using the visual embedding. Each dot represents a keypoint (left rear paw, beak tip, etc.), and its position on the y-axis indicates the percentage of that keypoint's positional variance in an egocentric coordinate system that the visual embedding can explain using a 5-nearest neighbor regressor. See Appendix C for details. We find that the visual embedding is sufficient to explain a large portion of the positional variance of the 3D keypoints.\n\n* **Lines 305–307:** The example was chosen to illustrate a subtle difference (a head tilt to the left) that our method captured but the baseline missed. The position of the mouse in the arena is different between the Query, KP NN, and VE NN columns and so the nearest neighbor retrievals are visually distinct. However, note the leftward tilt of the head apparent in the bottom Query image. This tilt is absent in the top KP NN image, in which the mouse's head is pointed straight ahead, but visible in both VE NN images. This is the visual distinction we intended to highlight. We will improve the figure's annotations (e.g., with arrows and zoom-ins) to make the key difference more obvious.\n\n* **On the 5-Nearest Neighbor Regressor (Lines 321-322):** Thank you for pointing this out. The process is detailed in Appendix C, but we had mistakenly dropped a reference to the appendix in the part of the main text that describes this figure. To clarify: for a given test sample's visual embedding, we find the 5 training samples with the most similar embeddings (via Euclidean distance). The final predicted 3D keypoints are then simply the average of the ground-truth 3D keypoints from those 5 neighbors. We will add this explanation to the main text.\n\n* **On the Reference Formatting:** Thank you for catching this. We will expand all \"et al.\" citations to include the complete author lists in the final version, adhering to the conference's formatting guidelines."}]

Author rebuttal:

Meta-review: This paper introduces Pose Splatter, a feed-forward framework for reconstructing 3D animal pose and appearance from sparse multi-view recordings using Gaussian splatting. Unlike keypoint- or mesh-based pipelines, the method requires no annotations, templates, or per-video optimization, making it more scalable for large behavioral datasets. The authors also propose a rotation-invariant visual embedding derived from these reconstructions, which can replace 3D keypoints in downstream analyses. Results across mice, finches, rats, and pigeons show strong reconstruction quality and biologically meaningful embeddings, with clear efficiency gains over optimization-based methods.

Reviewers agreed that the core contribution lies less in novel network components and more in building a lightweight, practical system that works for challenging animal data where existing tools fail. The rebuttal was convincing in showing why large generalizable object or human-centric models (e.g., AnySplat, InstantMesh, TRELLIS) underperform in this setting, often hallucinating occluded geometry. Additional experiments on new datasets, including pigeons, strengthened the case for generalization, and clarifications around camera requirements, evaluation metrics, and limitations (single-animal support, multi-camera setups) improved accessibility. The release of code and datasets was also seen as a valuable community contribution.

While there was some debate about overstated claims and whether “pose estimation” is the right framing, the consensus is that this work addresses a timely and important challenge in neuroscience and animal behavior. It provides a practical tool that removes annotation bottlenecks and enables scalable longitudinal analyses. On balance, the reviewers converged toward acceptance: the paper is technically solid, well motivated, and makes a meaningful applied contribution that justifies inclusion at NeurIPS. I recommend acceptance.
Decision: Accept (poster)