Temporally-consistent 3D Reconstruction of Birds
Authors: name: Johannes Hägerlind; name: Jonas Hentati-Sundberg; name: Bastian Wandt
Published: 2024-08-24 17:12:36

Abstract: This paper deals with 3D reconstruction of seabirds which recently came into
focus of environmental scientists as valuable bio-indicators for environmental
change. Such 3D information is beneficial for analyzing the bird's behavior and
physiological shape, for example by tracking motion, shape, and appearance
changes. From a computer vision perspective birds are especially challenging
due to their rapid and oftentimes non-rigid motions. We propose an approach to
reconstruct the 3D pose and shape from monocular videos of a specific breed of
seabird - the common murre. Our approach comprises a full pipeline of
detection, tracking, segmentation, and temporally consistent 3D reconstruction.
Additionally, we propose a temporal loss that extends current single-image 3D
bird pose estimators to the temporal domain. Moreover, we provide a real-world
dataset of 10000 frames of video observations on average capture nine birds
simultaneously, comprising a large variety of motions and interactions,
including a smaller test set with bird-specific keypoint labels. Using our
temporal optimization, we achieve state-of-the-art performance for the
challenging sequences in our dataset.