Automatic Individual Identification of Patterned Solitary Species Based on Unlabeled Video Data
Authors:   Department of Computer Science, University of Applied Sciences Darmstadt, Darmstadt, Germany, Suessle Vanessa , Arandjelovic Mimi ,   Max Planck Institute for Evolutionary Anthropology (MPI EVAN), Leipzig, Germany,   German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig, Leipzig, Germany, Kalan Ammie K. ,   Department of Anthropology, University of Victoria, Victoria, Canada, Agbor Anthony ,   Max Planck Institute for Evolutionary Anthropology (MPI EVAN), Leipzig, Germany, Boesch Christophe ,   Max Planck Institute for Evolutionary Anthropology (MPI EVAN), Leipzig, Germany, Brazzola Gregory ,   Max Planck Institute for Evolutionary Anthropology (MPI EVAN), Leipzig, Germany, Deschner Tobias ,   Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany, Dieguez Paula ,   German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig, Leipzig, Germany, Granjon Anne-Céline ,   Max Planck Institute for Evolutionary Anthropology (MPI EVAN), Leipzig, Germany, Kuehl Hjalmar ,   German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig, Leipzig, Germany,   Senckenberg Museum of Natural History Goerlitz, Goerlitz, Germany,   International Institute Zittau, Technische Universität Dresden, Zittau, Germany, Landsmann Anja ,   Zooniverse Citizen Scientist, c/o Max Plank Institute for Evolutionary Anthropology, Leipzig, Germany, Lapuente Juan ,   Max Planck Institute for Evolutionary Anthropology (MPI EVAN), Leipzig, Germany, Maldonado Nuria ,   Max Planck Institute for Evolutionary Anthropology (MPI EVAN), Leipzig, Germany, Meier Amelia ,   Max Planck Institute for Evolutionary Anthropology (MPI EVAN), Leipzig, Germany, Rockaiova Zuzana ,   Zooniverse Citizen Scientist, c/o Max Plank Institute for Evolutionary Anthropology, Leipzig, Germany, Wessling Erin G. ,   Department of Human Evolutionary Biology, Harvard University, Cambridge, Massachusetts, USA,   School of Psychology & Neuroscience, University of St. Andrews, St. Andrews, Scotland, Wittig Roman M. ,   Ape Social Mind Lab, Institute for Cognitive Sciences Marc Jeannerod, UMR 5229 CNRS / University of Lyon 1, Bron, France,   Taï Chimpanzee Project, Centre Suisse de Recherches Scientifiques, Abidjan 01, Côte d'Ivoire, Downs Colleen T. ,   School of Life Sciences, University of KwaZulu-Natal, Scottsville, Pietermaritzburg, South Africa, Weinmann Andreas ,   Department of Mathematics, University of Applied Sciences Darmstadt, Darmstadt, Germany, Hergenroether Elke ,   Department of Computer Science, University of Applied Sciences Darmstadt, Darmstadt, Germany
Published: July 2023

The manual processing and analysis of videos from camera traps is time-consuming and includes several steps, ranging from the filtering of falsely triggered footage to identifying and re-identifying individuals. In this study, we developed a pipeline to automatically analyze videos from camera traps to identify individuals without requiring manual interaction. This pipeline applies to animal species with uniquely identifiable fur patterns and solitary behavior, such as leopards (Panthera pardus). We assumed that the same individual was seen throughout one triggered video sequence. With this assumption, multiple images could be assigned to an individual for the initial database filling without pre-labeling. The pipeline was based on well-established components from computer vision and deep learning, particularly convolutional neural networks (CNNs) and scale-invariant feature transform (SIFT) features. We augmented this basis by implementing additional components to substitute otherwise required human interactions. Based on the similarity between frames from the video material, clusters were formed that represented individuals bypassing the open set problem of the unknown total population. The pipeline was tested on a dataset of leopard videos collected by the Pan African Programme: The Cultured Chimpanzee (PanAf) and achieved a success rate of over 83% for correct matches between previously unknown individuals. The proposed pipeline can become a valuable tool for future conservation projects based on camera trap data, reducing the work of manual analysis for individual identification, when labeled data is unavailable.