# An active ensemble classifier for detecting animal sequences from global camera trap data

Tommi Mononen, Bess Hardwick, Sandra Alcobia, Adrian Barrett, Gergin A Blagoev, Stéphane Boyer, Paula Gonçalves, Brigitte Gottsberger, Elli Groner, Chris C Y Ho, et al.

► To cite this version:

Tommi Mononen, Bess Hardwick, Sandra Alcobia, Adrian Barrett, Gergin A Blagoev, et al. An active ensemble classifier for detecting animal sequences from global camera trap data. Methods in Ecology and Evolution, 2025, 16 (10), pp.2500-2516. 10.1111/2041-210X.70144. hal-05520816

HAL Id: hal-05520816

https://hal.inrae.fr/hal-05520816v1

Submitted on 20 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# An active ensemble classifier for detecting animal sequences from global camera trap data

Tommi Mononen¹ ID

Bess Hardwick¹

Sandra Alcobia²,³

Adrian Barrett⁴

Gergin A. Blagoev⁵

Stéphane Boyer⁶

Paula Gonçalves³

Brigitte Gottsberger⁷ ID

Elli Groner⁸

Chris C. Y. Ho⁵

Marketa Houska Tahadlova⁹,¹⁰

Andrea Kaus-Thiel¹¹

Irmgard Krisai-Greilhuber⁷

Valerie Levesque-Beaudin⁵

Carlos Lopez-Vaamonde⁶,¹²

Sebastien Moreau⁶

Anna Mrazova⁹,¹⁰

Mikko Pentinsaari⁵

Adrian Pinder⁴

Kirsty Quinlan⁴

Wolfram Remmers¹³

Inês T. Rosário³ ID

Katerina Sam⁹,¹⁰ ID

Margarida Santos-Reis³

Lucas Sire⁶,¹⁴

Elise Sivault⁹,¹⁰ ID

Stefan Stoll¹³,¹⁵

Patrick Strutzenberger⁷ ID

Lauren Vanderlingen¹⁶

Fabrice Vannier⁶

Tereza Vlasata⁹,¹⁰

Otso Ovaskainen¹,¹⁷

**Correspondence**

Tommi Mononen

Email: tommi.mononen@helsinki.fi

Otso Ovaskainen

Email: otso.t.ovaskainen@jyu.fi

**Funding information**

Research Council of Finland, Grant/Award

Number: 336212 and 345110; Czech

Science Foundation (GACR), Grant/Award

Number: GACR 22-17593M; H2020

European Research Council, Grant/Award

Number: 856506

Handling Editor: Xingfeng Si

**Abstract**

1. Camera traps can generate huge amounts of images, and thus reliable methods for their automated processing are in high demand: in particular to find those images or image sequences that actually include animals. Automatically filtering out images that are empty or contain humans can be challenging, as images can be taken in different landscapes, habitats and light. Weather and seasonal conditions can vary greatly. Most of the images can be empty, because cameras using passive infrared sensors (PIR) trigger easily due to moving vegetation or rapidly varying shadows and sunny spots. Animals in images are often hiding behind vegetation, and camera traps will see them from previously unseen angles. Therefore, conventional animal image detection methods based on deep learning need huge training sets to achieve good accuracy.

2. We present a novel background removal approach based on movement masked images computed using sequences of images. Our deep vision classifier uses these movement images for classification instead of the original images. Additionally, we apply a deep active learning (active learning for deep models) for collecting training samples to reduce the number of annotations required from the user.

3. Our method performed well in singling out image sequences that actually include animals, thus filtering out the majority of images that were empty or contained humans. Most importantly, the method performed well also for backgrounds

<figure><img src="image_2.png" /></figure>

<footnote>For affiliations refer to page 2514.</footnote>

<footnote>This is an open access article under the terms of the [Creative Commons Attribution License](https://creativecommons.org/licenses/by/4.0/), which permits use, distribution and reproduction in any medium, provided the original work is properly cited.</footnote>

<footnote>© 2025 The Author(s). Methods in Ecology and Evolution published by John Wiley & Sons Ltd on behalf of British Ecological Society.</footnote>
and animal species not seen in the training data. Active learning brought good separation between classes already with small training sets, without the need for laborious large-scale pre-annotation.

4. We present a reliable and efficient method for filtering out empty image sequences and sequences containing humans. This greatly facilitates camera trapping research by enabling researchers to restrict the task of animal classification to only those image sequences that actually contain animals.

KEYWORDS

active learning, background removal, camera trap images, deep learning, ensemble learning, movement detection

# 1 | INTRODUCTION

Camera traps have become a widely used method for monitoring animal populations, especially mammals (Delisle et al., 2021). In densely populated regions and public parks, the images taken can contain humans rather than animals. Additionally, if camera traps are set to a sensitive mode for detecting smaller animals as well, the number of false triggers (empty images) can be large. Reliable automated methods for detecting those images that contain animals and filtering out the remaining images are thus needed.

One reason why it is not trivial for deep learning models to detect animals from images is the heterogeneity of the images: the appearance of natural scenery changes throughout a day. Shadows change their places, wind moves vegetation and the weather can be different over the course of a day and from 1 day to another. During the night, there is very little light and camera traps use flashes, which again makes everything look completely different. Animals are hiding in a landscape, or they might be too close or only partially visible in images. Changing seasons bring additional challenges via the changing look of natural scenes. Despite all these challenges, we need methods that are able to separate animal images from non-animal images.

There are many deep learning solutions for the task that report high accuracy in classification (Fennell et al., 2022; Tabak et al., 2020). Most of them have huge training sets (Leorna & Brinkman, 2022; Norman et al., 2023; Norouzzadeh et al., 2018; Yang et al., 2021), because large and diverse training material leads to better accuracy with convolutional neural networks (CNN), given that the material is compatible with a classification target. However, two grand obstacles still remain: there might not be a large enough number of training images available, and classification may not work very well with unseen and unexpected backgrounds despite a large training material (Beery et al., 2018; Norman et al., 2023; Schneider et al., 2020). The latter is mainly caused by the fact that models have not seen enough data to completely rule out backgrounds of different habitats, or that the backgrounds they have seen were too repetitive. Thus, the models are utilising the background information in classification, or get confused by seeing details of unseen habitats. Even if Machine Learning for Wildlife Image Classification (MLWIC) and

MLWIC2 classifiers are trained on about 3.3 and 2.6 million images, their performance decreases when applied to new unseen cameras in other countries and sites (Tabak et al., 2019, 2020; Wei et al., 2020). One exception seems to be Microsoft's MegaDetector (Fennell et al., 2022; Mitterwallner et al., 2023; Vélez et al., 2023), which is frequently updated with new annotated datasets and is designed to detect humans, animals and vehicles in images. The training set size is on the order of millions according to the github page (agentmorris/ MegaDetector), and version 5a uses, for example, iNaturalist 2017 data with thousands of species. The huge diversity of seen species, habitats and weather conditions helps the model to generalize well.

One way to enhance the generalization ability is to remove habitat information. Some methods apply background estimation (Bouwmans et al., 2019) to separate animals from their backgrounds, thereby helping neural networks to recognize relevant pixels more easily. In Yousif et al. (2019) images were divided into smaller blocks and, by comparing computed feature histograms over co-located blocks in time, the system inferred a possible presence of a moving target. Sherlock classifier (Penn et al., 2023) relies on comparing the difference between a target image and a rolling mean background, where the background is computed over temporally close-by images. However, the method works best on very open landscapes like moors. In a windy forest scene, where the direction of the sun changes throughout the day, these kinds of good mean images are challenging to construct. Another good way of removing disturbing backgrounds is the use of bounding boxes (Norman et al., 2023), which is a more suitable approach for species classification.

Our approach for background removal is to use movement information computed from image sequences (also called multi-shots or events). A similar approach has been used previously in Zilong classifier (Wei et al., 2020). IFLA assistant also shows movement images to help a user to manually find animals (Xi et al., 2021). Camera trap multi-shots are short sequences of images taken within a short time frame after a trigger event. Faster photographing speeds are enabled by measuring exposure only at the beginning—before the first shot—and in these cases exposure stays constant throughout a single image sequence. Therefore, comparing successive images is easy, and scenery stays relatively constant with respect to light and other conditions in most cases—compared to longer time intervals.
<figure><img src="image_3.png" /></figure>

Larger changes are most likely caused by animals or rapid vegetation movement.

A good way to train a classification model from scratch without pre-annotated data is to use active learning. In active learning (Ren et al., 2021), a model can acquire correct classifications of a small set from a user in each round (see Section 'Visualization of the active learning process' in *Supporting Information*). This selected set is such that the model gains new information with respect to the classification task. Then the model is trained again using an updated training set, which includes the new annotated images. Via this iterative process, the model will eventually learn to classify images. Active learning has been successfully applied previously in the camera trap context, and it has been found to greatly reduce manual classification effort, while at the same time the results are comparable to large fully annotated training sets (Norouzzadeh et al., 2021).

In ensemble learning, the predictions of several experts (models) are combined. Ensemble averaging has been reported to perform well against single models (Rougier, 2016). This is important especially with tiny training sets with short training runs, where randomness has a larger role (see section 'Benefits of ensemble averaging' in *Supporting Information*). The ensemble approach has been applied with deep learning models (Mohammed & Kora, 2023). For example, a model ensemble of three different model architectures (Yang et al., 2021) improved empty image detection compared to single architecture results. Studies combining active learning with model ensembles are rarer, although this combination has been shown to improve image classification results (Beluch et al., 2018).

In Table 1 we summarize the main approaches that are utilized in our classifier. In this paper, we will first introduce a pixel variance transformation for image sequences (background removal). It will bring out the movement in each image relative to other images in a sequence. In this representation, the role of image backgrounds is greatly diminished. The core of our classifier is formed by two filters

(detectors), each being an ensemble of deep networks trained with image augmentation (transfer learning and ensemble learning). The filters are built using active learning, where in each training round a user is asked to annotate a new set of images. Unlike most of the other solutions for animal image detection (Vélez et al., 2023), our classifier does not try to learn phenotypical features of all available animal species. It will expect any detected unknown movement, not caused by a human or vegetation, to be caused by an animal. We hypothesize that this indirect approach instead of direct animal detection enables our classifier to detect new and previously unseen animal species with a minimal amount of training data. Finally, the approach will be expanded to classify image sequences.

## 2 | MATERIALS AND METHODS

### 2.1 | Study sites and data

LIFEPLAN is a global research project studying the current state of biodiversity and ecological interactions on a global scale. Field teams across the world collect spore, soil, arthropod, audio, and image data. Each sampling site had five identical camera traps (trail cameras) during 2021–2022 (Custom model ‘4.0C’ from Wildlife Monitoring Solutions B.V., Netherlands) (Rogers et al., 2023). Four of the cameras are in the corners of an approximately 100×100m plot, and the fifth is located at the centre. They are taking colour photos during daytime and infrared (IR) flash photos during nighttime. The cameras are in a multi-shot mode and they take five photos in a row with 1s intervals whenever they are triggered. The trigger mechanism is a passive infrared (PIR) sensor reaching to 20m with large mammals and 3–4m with medium-sized birds (e.g. *Turdus merula Linnaeus*; common blackbird). The total number of collected 12 megapixel true colour photos is currently over 17 million (taken in

<table><caption>TABLE 1 The main methods that are used in our classifier to deal with very limited training data without pre-annotated training data.</caption><thead><tr><td>Method</td><td>Purpose</td><td>Effect</td></tr></thead><tbody><tr><td>Transfer learning</td><td>Re-training a deep learning model for a new classification task</td><td>Enabling classification with otherwise too small training sets</td></tr><tr><td>Ensemble learning</td><td>Generate variability among a set of classifiers to prevent overfitting</td><td>Better accuracy with small training sets</td></tr><tr><td>Active learning</td><td>Dynamically asking for classifications from a user</td><td>Better classification accuracy due to on-demand training set selection</td></tr><tr><td>Background removal (based on movement)</td><td>Reduction of irrelevant information</td><td>Better generalization over different cameras</td></tr><tr><td>Image augmentation</td><td>Create new samples similar to the existing training set data</td><td>Better generalization achieved by larger simulated data</td></tr><tr><td>Parallel computing</td><td>Running models of an ensemble in parallel</td><td>Reduction in training and classification times</td></tr></tbody></table>

Note: Transfer learning and image augmentation are the most used methods in small data context.
2020-2025) corresponding to 3.4 million image sequences. The PIR sensors are purposely set to a sensitive mode. This choice allows the cameras to detect also smaller animals, like mice and voles, close to cameras.

In this study, we will use data collected from eight sites (see example images in Figure 1):

* *Guelph (G), Canada:* an arboretum in the city; lots of animals and some people.

* *Mitzpe Ramon (MR), Israel:* a desert nature reserve; people and animals.

* *Perth (P), Australia:* a nature park; people and animals.

* *Tours (T), France:* a university campus region; people and animals.

* *Vienna (V), Austria:* a botanical garden; lots of people and some animals.

* *Hunsrück-Hochwald (HH), Germany:* a national park; lots of animals.

* *Lisbon (L), Portugal:* a busy forest park; windy site with people and animals.

* *České Budějovice (CB), Czech:* trails near a campus; lots of people and pets, some wild animals.

We manually annotated 1–2 months of camera data per camera from the first five sites (G, MR, P, T, V), and the sixth site (HH) was used as additional training data. These annotated test sets are used only to compute the actual performance of our classifier—not in model training. Data were collected in 2021, except Lisbon data are from 2022. The Lisbon data set (L) is also used in a practical use-case example. Data from České Budějovice (CB) is used to infer

performance differences between daytime colour and low-light IR images. To reduce the amount of manual annotation work, we annotated test data in only 10 cameras (G1, V1, MR1, MR2, P1, P2, T1, T2, L1 and CB1, where the letters indicate the sites). These cameras were selected beforehand, while finding suitable training material for the classifier, without knowing future test performance in these sets. Due to the replacement of broken equipment, Guelph has more than five cameras (G1–G8).

No research permit was required at sites Tours, Vienna, Hunsrück-Hochwald, Lisbon, and Czechia. For site Perth, permit 25000629-c was issued by the Department of Biodiversity, Conservation and Attractions, Government of Western Australia. For site Mitzpe Ramon, permit 43699 was issued by Israel Nature and Park Authority (INPA). For site Guelph, an unnumbered permit was issued by Guelph University Arboretum.

## 2.2 | Overview of methodology

We trained two separate classification models, to be called the human filter and the empty filter, with the idea that the target class (animals) consists of images that are not empty and that do not contain humans (Figure 4). We use the word ‘filter’ instead of ‘detector’ to indicate the categories to be omitted. Both filters were trained with an active learning approach, which generated a separate training set for each of them, but the final filters were applied to common fixed test sets. The training proceeded in rounds, each round involving the manual annotation of around 50 images to human and 50 images to

<figure><img src="image_4.png" /><figcaption>FIGURE 1 Camera views showing different habitat types used in training and tests. Thickness of vegetation varies within sites and from one season to the other. In the top row, the leftmost and rightmost images show the two cameras (G1 and V1) used in the initial training of two filters.</figcaption></figure>
<figure><img src="image_5.png" /></figure>

non-human classes for the human filter. Similarly, for the empty filter, we annotated around 50 images to empty and 50 images to non-empty classes. For testing the generality of the results, the training data was initialized from a single camera, and then data from new cameras and sites was gradually added to the training set. We performed in total 24 training rounds. For the empty filter, the rounds start through the cameras G1-G1-G1-G1-V1 (a sequential training order over cameras), where the first round consisted of initially selected seed images. For the human filter, the rounds start through the cameras V1-V1-V1-V1-G1. Data from two main cameras (G1 and V1) were used during several rounds because these cameras have very rich and versatile image material. After these five rounds, both filters were further trained with cameras G2-G3-G4-G5-G6-V2-V2-G7-G8-L2-L3-L4-L5-HH1-HH2-HH3-HH4-HH5-L1. Each round after the first one was based on active learning, where the new set of training images was selected by the most recently trained filter. The filters form a final classifier.

## 2.3 | Collection of candidate and test sets

A candidate set for training is a set that consists of all possible images that can be used for model training. However, in our approach, we let the active learning algorithm decide which of these images are actually used in model training. A training set is formed of the selected images.

The data were split into fixed candidate and test sets. Candidate data weeks are not overlapping with test data weeks, which means, for example, that we do not have the same people with the same clothes in the candidate and test sets (except possibly regular members of field teams). We selected April and September as test data months and the rest of the year as candidate data (see Table S1). The test sets were manually annotated by investigating shrunken colour images (448×448 pixels). The resolution used in the manual annotation was higher than what the filters see (224×224 pixels). The manual annotation of test sets was completely fair and very strict:

(1) No data were left out, even if we knew that correct classification of some particular examples was impossible for our classifier;
(2) Presence of targets can in many cases be very marginal, meaning that if a target (a human or an animal) was clearly detectable or recognizable only in a single image within a sequence, its presence in other images in the sequence (or other temporally proximate sequences) was still marked in the same class.

The main training sites for the filters are Vienna and Guelph. The main Guelph training camera (G1) has 34% empty images, and 81% of sequences are animal sequences in the test set. The main Vienna camera (V1) has 66% human images, and the human sequence proportion is 74% in the test set. These test set proportions reflect the candidate set percentages for the cameras. Additional statistics are shown in Table 2.

## 2.4 | Image data pre-processing

Image backgrounds contain information which is not needed for detecting animals. Background removal has been shown to improve classification results with a model trained using images taken in captivity and applied to wild images (Curry et al., 2021). To erase backgrounds, we will remove each pixel that does not have enough variation between the different images in a sequence.

### 2.4.1 | Movement revealing image transformation

Temporal differencing is a movement detection approach for videos (Amandeep & Goyal, 2015), where consecutive time steps are subtracted from each other to detect a change in image information. However, many of the sophisticated video targeted methods are not directly applicable in our case, where each sequence has only five images with 1s intervals, and preceding and following sequences may be taken under totally different light conditions. Therefore, we

<table><thead><tr><th>Camera</th><th>Candidate images</th><th>Test sequences</th><th>Animal</th><th>Human</th><th>Empty</th></tr></thead><tbody><tr><td>G1 (Guelph 1)</td><td>11,045</td><td>1286</td><td>1048</td><td>4</td><td>234</td></tr><tr><td>V1 (Vienna 1)</td><td>49,890</td><td>388</td><td>49</td><td>286</td><td>53</td></tr><tr><td>P1 (Perth 1)</td><td>2935</td><td>143</td><td>81</td><td>41</td><td>21</td></tr><tr><td>P2 (Perth 2)</td><td>4010</td><td>377</td><td>271</td><td>57</td><td>49</td></tr><tr><td>T1 (Tours 1)</td><td>3890</td><td>192</td><td>138</td><td>13</td><td>41</td></tr><tr><td>T2 (Tours 2)</td><td>3240<br>(candidate)+1525<br>(test)</td><td>305</td><td>17</td><td>3</td><td>285</td></tr><tr><td>MR1 (Mitzpe Ramon 1)</td><td>2160</td><td>141</td><td>34</td><td>9</td><td>98</td></tr><tr><td>MR2 (Mitzpe Ramon 2)</td><td>3520</td><td>366</td><td>33</td><td>10</td><td>323</td></tr></tbody></table>

Note: Test set counts of sequences (per class) are shown for a set of cameras used in tests. The most common class per camera is highlighted in bold.

<caption>TABLE 2 Candidate image and test sequence counts.</caption>
apply temporal differencing only within a sequence by comparing each image against all other images, not only preceding ones.

We denote the value of a pixel (x, y) at an image i with $p_{channel}(i, x, y)$, where 'channel' refers to a RGB colour channel (red, green or blue) having values from 0 to 255. Discrete variance of a pixel over an image sequence is defined as

$$ \text{var}(p_{\text{channel}}(\cdot, x, y)) = \frac{1}{n-1} \sum_{i=1}^{n} \left( p_{\text{channel}}(i, x, y) - \frac{1}{n} \sum_{j=1}^{n} p_{\text{channel}}(j, x, y) \right)^2, $$

where n is the number of images in a sequence. Additionally, we denote with

$$ \text{var}(p_{\text{channel}}(-i, x, y)) $$

the case where variance is computed over all other images except the target image i. A pixel variance transformation can be computed as

$$ \begin{aligned} T(i, x, y) = & \max(0, w_{\text{red}}[\text{var}(p_{\text{red}}(\cdot, x, y)) - \text{var}(p_{\text{red}}(-i, x, y))] + \\ & w_{\text{blue}}[\text{var}(p_{\text{blue}}(\cdot, x, y)) - \text{var}(p_{\text{blue}}(-i, x, y))] + \\ & w_{\text{green}}[\text{var}(p_{\text{green}}(\cdot, x, y)) - \text{var}(p_{\text{green}}(-i, x, y))] - \text{thres}) \end{aligned} \quad (1) $$

where colour component weights are $w_{red}$, $w_{blue}$ and $w_{green}$. The relative weights of different components make it possible to tune and optimize variance transformation in such a way that it detects some colour variations with increased sensitivity. Here we have only used balanced setting ($w_{red} = w_{blue} = w_{green} = 1$), but in especially homogeneous environments, a different weighting can boost differences between targets and backgrounds. We allow only positive values larger than thres (1/256 in our tests, based on visual investigation), so that negative shapes caused by other images in a sequence will disappear from images. The threshold value can be used to tune detection sensitivity of the transformation. Intuitively, this transformation subtracts pixelwise variances of all other images except the image i from the pixelwise variances of the whole image sequence. This pixel variance based transformation will reveal distinctive movement in a target image [$T(i, x, y) > 0$; white pixels] and completely removes the background [$T(i, x, y) = 0$; black pixels] if there is no movement at all (see Figure 2). To further improve the quality of these movement masks, we apply median filtering (medfilt2 in Matlab) to remove randomly scattered single white and black pixels called salt and pepper noise. Thus, the operation fills tiny holes in white areas and removes lonely white pixels from black areas.

<figure><img src="image_6.png" /><figcaption>FIGURE 2 (a) An example image sequence with the corresponding B&W and colour masks for the first image in the sequence showing only a moving deer. (b) Examples of other movement-filtered images. Each column shows a scaled original image and the corresponding B&W mask. Each of these examples is a selected single image from a complete sequence of five images. Animals behind vegetation will pop out in masks and look the same at different locations and habitats.</figcaption></figure>
## 2.4.2 | Properties of different movement masks

The presented black and white (B&W) movement masks show which pixels have changed compared to other images in the sequence. They describe the shapes of the moving targets. In many cases, this is enough to make the distinction between two animal species or between an animal and a human. Visually, they can separate animals (solid masses) from vegetation movement (scattered masses). Night (IR) and daylight images look exactly the same in this representation (compare the two deer in Figure 2).

If we mask a scaled colour image with the corresponding B&W movement mask, we end up with a much richer representation (see Figure 2a). Surface patterns (e.g. fur or spikes) and colours become visible. However, the colours of ambient and reflected light are not constant in natural environments: a grey wolf is actually reflecting green light when walking through a green meadow. In addition, gray-scale IR images and colour images are well separated by the colour information. Therefore, an animal looks quite different in day and night photos. We will refer to these two masked image categories with convenient short forms: B&W and colour masks.

## 2.4.3 | Stabilizing camera movements

Unfortunately, strong winds can also shake camera traps, not just vegetation. This will cause shifts throughout the whole image area between different images. As a result, backgrounds which we try to avoid become visible and therefore will be included in movement images. To overcome camera shaking, we use a post-process image stabilization method which finds and relocates all immobile objects to

corresponding pixel locations in different images, thereby making the dislocations disappear (Figure 3), which will improve classification results (see Section ‘Image stabilization test’ in Supporting Information). The image stabilization is done by applying a shake-correcting affine transformation removing especially rotation and translation (Matlab/ Computer vision Toolbox: cvexTformToSRT.m). Our algorithm applies image stabilization to the images in a sequence using the third image as an anchor point (a position defining image), as it is closest in time to other images in the sequence. The operation is done for every sequence before computing any movement masks. As the operation is quite heavy, the original full-size images are first scaled to the size 448×448 (bicubic interpolation) for image stabilization. Then masks are computed and finally rescaled to a final size of 224×224 pixels. Image operations done at a higher resolution ensure a higher quality result. In those cases where the stabilization algorithm fails (see Section ‘Image stabilization test’ in Supporting Information), original images are used instead while computing movement masks.

## 2.5 | Image sequence classification

We explain the structure of our classifier (Figure 4) by first explaining filters. Then we explain an active learning approach for training them. Finally, we combine the trained filters and a classification logic.

### 2.5.1 | Residual networks and filtering approach

ResNets (Residual Networks) are used in object recognition due to their relatively good performance and speed (Fauzi et al., 2021;

<figure><img src="image_7.png" /><figcaption>**FIGURE 3** An example of a stabilized camera trap sequence during a gust of wind. In the bottom row images, camera movement has been eliminated with algorithmic post-processing. A dog becomes more visible and there are fewer undesired background pixels. Naturally, the movement of vegetation does not disappear completely.</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>FIGURE 4 A basic diagram of a classifier structure showing the major components of our deep active learning classifier and how it classifies single images and image sequences. On the right, a simplified procedure diagram shows how the active ensemble learning module chooses a new image for user annotation. A distance-sorted image index table is searched row by row to find images to propose. The distances are not shown here, but only indexes of images. Each column corresponding to the outcome of a single ResNet is sorted independently.</figcaption></figure>

Norouzzadeh et al., 2018). We are using an 18-layer version (see Section 'Image information compression and ResNet18' in *Supporting Information*) because it is relatively small and fast, and because our colour masks have less information than the original images and therefore do not need more complex CNNs. Moreover, the Matlab implementation of ResNet18 seemed to have the best trade-off between computation time and accuracy among available models. We apply transfer learning (ImageNet database; more than 1 million images), where the last three layers (fully connected, softmax and classification layer) are replaced and the model is retrained to classify our camera trap training samples.

We let ResNet18 learn and classify colour masks. The movement masked images show two very distinctive movement types: vegetation and human/animal movement. We train two filters: empty filter (vegetation/empty vs. animal/human) and human filter (human vs. animal/vegetation/empty). For example, if there are no moving animals or humans in the image, then the entire mask is declared empty by the empty filter (see Figure 4). We omitted a vehicle class and thereby a vehicle filter because the data in this paper has only some snapshots of moving maintenance vehicles (e.g. ride-on lawn movers). Humans were almost always present in these cases.

2.5.2 | Structure of a filter

Instead of using only a single model (ResNet18) for a filter, we train an ensemble of models. In our tests, we use 24 ResNets that all see a same training set, but different augmentations of it. These random modifications of the training set colour masks consist of horizontal flipping, max ±5% scaling horizontally and vertically and moving masks max ±10 pixels horizontally and vertically.

In classification, we compute the median of class probabilities over the models in an ensemble. The median is used instead of a mean because it is closer to a mode (skewed distribution) and represents a majority vote when models are highly unsure about the correct class. The approach is called soft voting or ensemble averaging, and its main purpose is to provide more reliable results by reducing variance (Naftaly et al., 1997) and avoiding the worst accuracies of single training runs (Christiansen, 2018; Rougier, 2016) (see Section 'Benefits of ensemble averaging' in *Supporting Information*). Even if some of the ResNets fail to give a reasonable classification, most of the models in an ensemble as well as their joint prediction will perform well. The other advantage is that the training of multiple ResNets can be done in parallel, and therefore this approach roughly equals a longer run of a single model because many more augmented images are seen by a model at once.
<figure><img src="image_9.png" /></figure>

### 2.5.3 | Active learning (a single model)

Constructing a good training set from scratch is challenging and time consuming. In LIFEPLAN, we have cameras all over the world and species diversity is huge (mammals, birds, reptiles and insects). Many existing annotated training data sets are collected from geographically selected locations and do not therefore have a global selection of animals. In addition, many of these available annotated training sets have only single images instead of image sequences and also lack empty images, which makes these datasets unsuitable for training our movement approach.

In active learning, a user starts by finding some example images for each class (we picked 20 diverse images per class). The classification model will be trained with these images, and in each round an active learning algorithm asks for correct annotations for a selected set of new images (in our runs at most 50 per class). These user-annotated images will then be added to the training set, and the process continues iteratively.

The images that a model selects for user annotation are different from existing training images. The main advantage of the active learning approach is that a formed training set does not consist of 10s of almost identical images per category, given that candidate set images are diverse. Our active learning method tries to cover the whole image space [a geometric approach with a covering blanket (Beluch et al., 2018)]. There are also numerous other methodological approaches (Ren et al., 2021) for finding annotation sets.

Our current algorithm is classifying 500 randomly chosen colour masks from a candidate set during each step (see Section ‘Visualization of the active learning process’ in Supporting Information). Automatic classification of all images during each active learning step would be computationally too slow. Activations in the fully connected layer (a vector of values) represents information that a model has extracted from a colour mask that was fed in. The vector is a point in a 512-dimensional space (see Section ‘Image information compression and ResNet18’ in Supporting Information), where the vectors of other colour masks also lie. The algorithm chooses 2×50 new masks from the set of 500, so that these new masks are as far from the previous training set masks as possible by Euclidean distance, which we tested as the default choice and found to work well for the purpose. Thus, a distance of a mask vector $M_i$ (in the set $S_{500}$) to the training set mask vectors $S_{training}$ is measured as

$$ dist(M_i, S_{training}) = \min_{M_j \in S_{training}} dist_{eucl}(M_i, M_j) \quad (2) $$

and the greedy algorithm finds masks one by one that maximize the given distance over $S_{500}$. A found distance maximizing mask is added to the training set, and all distances are updated accordingly, before finding the next suitable mask. The model selects an equal amount of colour masks from both classes based on its current classifications. If candidate masks belonging to one class are exhausted, or distances to the training set become 0, fewer images from that class will be passed to the user.

It is important that the number of training set colour masks is quite even between classes; otherwise, models have a tendency to become biased and abandon a smaller class. As an extra precaution, we are additionally using upsampling to ensure the balance. Thus, a smaller class is made to be equally large as a bigger class by randomly sampling additional copies from the training set of the smaller class. Random augmentations during training make two copies of the same colour mask look slightly different.

### 2.5.4 | Active ensemble learning (training filters)

Every submodel in a model ensemble will give a different score to the same colour mask (a probability of an image belonging to a given class). Active learning by an ensemble model is more complex (Figure 4) than the single model version. The main reason is that each submodel/ResNet has its own understanding about distance order over a set of 500 colour masks. To form a common agreement about the order, each submodel forms first its own ordered list, and then a search algorithm starts to go through all the lists in parallel. When it has found the same colour mask from all the lists, it will check if its class matches with the sought class (based on a median probability estimate). If it does, the algorithm will add the image to an annotation set for a user, and if not, it will continue searching. Then the distances of each model are computed again with the assumption that the selected colour mask is now in the training set. The process will be repeated iteratively the required number of times (images).

A classification score histogram over all median probabilities can be used for checking the current state of convergence w.r.t. some camera (Figure 5). When almost all median probabilities are relatively close to 0 or 1, further active learning rounds will change results only marginally. However, it must be kept in mind that even if an ensemble model is confident, it can still classify some set of images incorrectly.

## 2.6 | Rules for single image and image sequence classification

Figure 4 shows final classification procedures in a compact form. After full training of both filters, we can use the ensemble models (filters) for classification by merging the predictions of these two binary filters: If a human is detected (human filter probability ≥50%) an image is classified as a human image. If the image is classified as a non-human image and it is non-empty (empty filter probability <50%) then it is classified as an animal. Otherwise, the image is classified as empty. Essentially, the human filter probability threshold tunes the balance between human and animal classes, and the empty filter probability threshold tunes the balance between important and non-important movement.

An image sequence is classified as a human sequence if any of the images is a human image. If all images are empty, then the sequence is empty; otherwise, it is an animal sequence. Thus, a woman walking a dog is always a human sequence, as is a sequence with
<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /><figcaption>FIGURE 5 A user can follow the convergence of a filter during active learning from the score histograms of 500 randomly sampled images. Left panel: After the initial 2×20 user-selected images (the first round), a filter is fairly unsure about classification, which is reflected by median probabilities far from 0 and 1. Right panel: After five rounds (upsampling; 2×192 images) the filter starts to be more confident about classifications, indicated by a reduction in uncertainty. Sampled image sets are different in the two panels.</figcaption></figure>

people and squirrels. This rule ensures that highly sensitive human images and sequences will be automatically filtered out.

## 2.7 | Practicalities while training filters

To give a good start for active learning, the first 20 example images per class should be as diverse as possible. Unlike in an ordinary annotation process, we cannot just select images based on original colour images, because our ensemble model (filter) sees only colour masks. A user has to ensure that a colour mask represents a given class well. For a human, it is sometimes difficult to interpret the content of background-removed images. We found that the best way to do the annotation is to investigate a colour image, a B&W mask, and a colour mask side by side (see representations shown in Figure 2a). B&W masks are needed because the pixels of a colour mask can be so dark that the mask cover is easier to check from the B&W mask.

We chose initial seed images for filters from two sites following these guidelines:

*Human filter* training is started using botanical garden images from Vienna, 2021. All colour masks with humans are considered to be long to class *human* if the mask covers human pixels well and not much background. If no human is present, the class is *non-human*.

*Empty filter* training is started using arboretum images from Guelph, 2021. All colour masks that have a moving animal and/or human are considered to belong to class *non-empty*, whereas vegetation movement alone falls into class *empty*.

When active learning returns a set of images to be annotated, an additional complication arises. Some images can be too obscure for a human to confidently classify. For example, a target can be

too close or too small. Colour masks can also reveal more background pixels than target pixels when the position of the target partially overlaps in several images. Also, if a target stands still during the whole image sequence (5s in our cameras), it cannot be seen in movement masks (e.g. people sitting on a park bench). The basic rule is that if a user can classify colour masks to correct classes without guessing or hesitation, then they should be annotated. Our interactive algorithm gives a user the option to skip obscure colour masks without annotating them (see Figure S5). They are marked as *unclassified* so that the active learning will not show them to the user again. These unclassified images are then included in a set of known images, which is used when an active learning algorithm searches for new images to show a user. Thus, the algorithm sees them as if they were in a training set; therefore, it will not show them or highly similar images either. These unclassified images are not used for filter training.

# 3 | RESULTS

## 3.1 | Three classifiers

We trained empty and human filters for three classifiers (see below) using active learning (see Section 'Visualization of the active learning process' in *Supporting Information*). Training of each active learning round lasted 10 epochs (passes through training data) with a constant learning rate of 0.00015. Model training was not continued from a previous active learning step, but always started from the beginning in each round (non-sequential) using the training set collected so far, because we wanted to avoid any possible convergence problems. We stored all intermediate training sets, and to produce final plots, we retrained all active
<figure><img src="image_12.png" /></figure>

learning steps of filters using 25 epochs and a slightly slower constant learning rate of 0.0001 to ensure convergence. This learning rate is a suggested initial value for transfer learning in Matlab, and it worked well. The minipatch size of 10 is lower than usual due to high memory consumption caused by multiple parallel runs (NVIDIA V30 with 24 GB of memory).

**12 round classifier:** The empty filter is trained with cameras G1-G1-G1-G1-V1-G2-G3-G4-G5-G6-V2-V2 and the human filter with cameras V1-V1-V1-V1-G1-G2-G3-G4-G5-G6-V2-V2 (see Figure 6). The training set of the human filter ended up having 391 non-human and 497 human colour masks, and the training set of the empty filter had 627 non-empty and 219 empty colour masks.

**15 round classifier:** Starting from the 12 round classifier, both filters are trained for a single round with each problematic camera: MR2-P2-T2.

**24 round classifier:** A base classifier is again the 12 round classifier (Guelph and Vienna cameras). Using active learning for 12 further rounds, both filters are trained using sites Guelph (two cameras), Lisbon (five cameras) and Hunsrück-Hochwald (five cameras) with sequence G7-G8-L2-L3-L4-L5-HH1-HH2-HH3-HH4-HH5-L1. The final human filter is trained with 815 non-human and 946 human training set colour masks. The final empty filter is trained with 1202 non-empty and 420 empty colour masks.

For comparison, we computed MegaDetector results using our sequence classification rules (Section 2.6) for all test sets presented in Table 2. For more information, see Section ‘MegaDetector(v5a) performance with Table 2 datasets’ in Supporting Information.

## 3.2 Performance in familiar habitats and with known animal species

We report accuracies to show improvement during active learning, and precision with recall after 12 rounds for different classes (animal, human, empty). The accuracy curves show that our main training sites achieve 83% (G1) and 86% (V1) accuracies within 12 training rounds (Figure 6a). In comparison, predicting always the most common classes gives accuracies of 81.5% (G1; animal) and 73.7% (V1; human). However, the most common class for each particular data set is not known beforehand; the classifier also performs better than the hypothetical model predicting always the most common class.

At the beginning of training the 12 round classifier, the human filter learns to recognize almost all moving targets as human, because there are not many animal sequences taken in the Vienna city park. This leads to much lower accuracy for the Guelph camera at the beginning (Figure 6a). Gradually, the human filter learns to differentiate also between animal and human as it sees more animal images in the training data (starting from round 5). The empty filter does not have to differentiate between animals and humans, and therefore its accuracy can just increase relatively monotonously (not shown here). The main observation is that data from other cameras improve the classification results of our main cameras. However, adding new data does not always monotonously increase the accuracy for a camera (Figure 6a).

After 12 rounds we can observe that empty and human sequences are classified into correct classes quite reliably (high recall), but the classes include also some sequences from the wrong classes, which is indicated by lower precision (Figure 6b). In contrast, animal sequence precision is high with most cameras, indicating that when the classifier claims something to be an animal sequence it actually is.

<figure><img src="image_13.png" /><figcaption>FIGURE 6 (a) Accuracy in sequence classification improves during the active learning process with two initial training cameras. Blue line segments show accuracy improvements from each new camera used in training—right before and after a training round. The corresponding camera names are marked on the x-axis. Accuracy is still improving between rounds 12 and 24. (b) Shows class-wise precisions and recalls for different cameras from various sites after 12 rounds. Double framed markers correspond to the cameras G1 and V1 of panel A and the legend lists marker shapes (sites) and colour codes (classes). (c) The situation after doubling the training data stays almost the same, except that the precision of the human class has decreased.</figcaption></figure>
## 3.3 | Performance at unseen habitats and with unseen animals

The data of the 12 round classifier comes only from Vienna and Guelph sites. We use as unseen test sites Tours (France), Mitzpe Ramon (Israel) and Perth (Australia). Of these three, the habitat of Tours can be considered to be the closest match to Guelph (Figure 1), but the most common species in Tours, cats and foxes, are almost absent from the training set (empty filter: none; human filter: three cat and one fox colour masks). Habitats and animal species of Mitzpe Ramon and Perth sites are very different compared to the training set images. In Mitzpe Ramon, the landscape is open and vegetation is rare (desert). In Perth, on the contrary, cameras are in a bushland habitat. The classifier seems to perform almost equally well with unseen animal species and sites compared to the training sites (with previously seen animals), because every non-vegetation and non-human movement is interpreted as an animal (Figure 6b).

In windy conditions, vegetation is moving considerably between two images. This movement brings local, site-specific information to colour masks; the location of vegetation brings information w.r.t. backgrounds. Therefore, empty sequence detection is not equally reliable at unseen sites. One solution is to add a small amount of empty images of unseen sites to a training set to make the filter(s) more reliable. This can be done by doing an additional active learning round for that particular camera or site. We test this hypothesis using three cameras from Tours, Mitzpe Ramon and Perth. Initially, using the 12 round classifier, the misclassification rate is quite high with the chosen cameras (Figure 7).

Classifications of three datasets are improved a great deal using the 15 round classifier that has seen local images. Accuracies of the trained classifier are close to perfect in the first two datasets, which is reflected by class sizes getting closer to the truth (Figure 7). The active learning round of T2 was done using the whole set—including the test set (only for this camera, because the only windy days were during test weeks). This reflects an actual use-case experiment, where a user uses complete data. Mitzpe Ramon shows high performance after training the bush movement because the landscape is empty and otherwise immobile. The Perth test set is more complex as vegetation varies throughout different seasons, plants sway in the wind and vegetation is able to hide at least smaller animals.

## 3.4 | Performance degeneration when adding new cameras

Adding data from a new site to a training set can potentially make the results for the previous sites worse when training sets are small. Also, a large amount of overly monotonic material (movement of tree leaves, the movement of a single bush) can decrease the accuracies of other sites. Therefore, it is better to leave out repetitive monotonous extreme cases from the filters

and not include all new training samples. We trained the 24 round classifier following this guideline. When active learning gave too many similar looking vegetation movement images in rounds 13–24, we chose only 5–15 of them and marked the rest as unclassified to keep the training set more heterogeneous. This is, of course, a purely subjective choice.

The accuracies of the 24 round classifier improved compared to the 12 round classifier with respect to our two main training cameras (Figure 6a), while the 24 round classifier learned to classify sequences in the new sites. There are natural trade-offs between recalls and precisions as well as among different classes depending on added material (Figure 6b,c). Only the accuracy of MR1 has decreased from 96% to 87%; because the 24 round classifier started to make more errors with empty sequences. Other unseen sites had essentially the same accuracies (P1 stayed at 82%, P2 increased from 78% to 81%, T1 stayed at 77%), which indicates that after some point, site specific or highly similar images are essential for gaining further improvements in accuracy.

We also compared the results of 12 and 24 round classifiers in the previous three problematic sites (T2, MR2 and P2). With T2 the recall of empty sequences rose from 39% to 53% and the accuracy rose from 42% to 54%. This is probably due to adding the windy Lisbon site. The accuracy of MR2 stayed at 81%. With P2 animal recall rose from 71% to 78% and accuracy from 78% to 81%. Hence, adding new cameras did not, in most cases, cause an accuracy drop compared to 12 round classifier results.

## 3.5 | Performance differences between day and night

Daytime colour masks contain more information than grey-scale IR masks. When a mask has only a small amount of informative pixels, classification might be based purely on a particular colour: for example, green vegetation or a hint of brown fur. Grey-scale IR masks lack this essential information.

We tested day (colour) versus night (grey-scale IR) performance on unseen data from a camera in České Budějovice (CB1). We used the 24 round classifier with a single additional active learning round for the human filter only, trained on a random sample of 500 images from the full 2021 set of images from CB2. Both cameras capture many people moving during the late evening. The additional single active learning round was done because an initial test revealed that the human filter training set should contain more grey-scale human examples (originally only 20 of 946 human example masks).

In daytime, human sequence detection recall was 98% and precision 97% (of 4853 human sequences); with IR sequences, recall was 85% and precision was 99% (of 258 human sequences). Correspondingly, daytime animal detection recall and precision were 80% and 89% with colour sequences (of 837 animal sequences); with IR sequences, recall was 87% and precision was 84% (of 289 animal sequences). It is likely that these results partly reflect uneven IR and colour image proportions in the training sets.
<figure><img src="image_14.png" /></figure>

**FIGURE 7** Sequence classification performance improves after additional training rounds on cameras having initially lower accuracy (T2, MR2 and P2). Example images from these cameras are shown in the leftmost column. Grey bars show the true amount of sequences in each class and green bars show classification results. In Tours, the movement of tree leaves is interpreted as possible humans (compare the bars at the top middle panel). In Mitzpe Ramon, the movement of a bush is interpreted incorrectly as animal movement. In Perth, the 12 round classifier interprets some kangaroo images as humans. After the additional training rounds, the performance increases considerably in each case (the rightmost column). Percentages shown are the recalls of each class; that is, what proportion of correct class sequences were found.

## 3.6 | A practical example of camera filtering

In real cases, active learning is done using the complete data of a camera. We take data from a single camera from the windy Lisbon site (camera L1). In total, there are 44,960 images, which would be very tedious to manually go through. Our 24 round classifier classified 178 of 8992 as animal sequences (the last round of the 24 round classifier was trained using this camera; without it, 770 sequences were classified as animal sequences). By creating a video of this smaller set of sequences, it is very easy to check that the majority of the sequences are still actually empty and all but one human sequence were successfully removed. Manual investigation of this cropped set revealed that only 62 of the 178 sequences

are truly animal sequences. Then we labouriously classified the complete set and found 27 additional animal sequences that our classifier did not find. We show sample images from three of these in **Figure 8**. The bird in image 1 is interpreted as vegetation movement (possibly a tree trunk due to Perth training images) and images 2-3 have some human clothing colours. Training the human filter with a bird-rich camera would help to reduce the classification error. Many individual images within these two sequences were correctly labelled as animal, but our sequence classification rule will make the sequences human sequences. In many other misclassified animal sequences, targets were not moving enough, were too small/distant, or were even difficult to notice from a single image alone (without movement). The bottom row in **Figure 8**
<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /><figcaption>FIGURE 8 Examples of Lisbon camera classifications. Class predictions are shown in the top right corner of each image. On the top row (images 1–3) are the worst cases where the classifier failed to recognize highly visible animals. On the bottom row (images 4–6) are some of the most cryptic examples that our classifier recognized correctly. Zoomed B&W masks on the bottom right corner of each image show the movement pixels. By raising the human probability threshold to 80%, images 2 and 3 become correctly classified.</figcaption></figure>

shows single images of the correctly classified sequences, where our classifier was able to find animals. Each of the images is the most high-confidence animal (a moving non-human target) image in the corresponding sequence.

We compared our results against the results of MegaDetector v5a. To make the comparison fair, we scaled the resolution of images down to 312×224 pixels (1.4 times more pixels than in 224×224 image; maintaining the original aspect ratio). Using the default confidence threshold of 0.2, MegaDetector selected 927 sequences; therefore, we increased the confidence threshold to 0.45, leading to a decent 166 sequences. Of these, 60 sequences were animal sequences. Thus, both classifiers found a roughly equal amount of animal sequences, but the difference in training set sizes is huge: some thousands of images (520 empty, 1202 non-empty, 946 human and 815 non-human colour masks) against millions of images (according to the MegaDetector github site). Of course, our classifier has seen some data from the target camera (72 non-human, 1 human, 82 empty and 5 non-empty colour masks) and MegaDetector none. We also tested MegaDetector with a threshold of 0.2 and full-resolution images (4032×3024) and in this case, it selected 200 sequences, 84 of them being animal sequences (recall 94%, precision 42%). By tuning the confidence threshold to 0.45, MegaDetector achieved 91% recall and 85% precision, matching previously reported performances (Fennell et al., 2022; Mitterwallner et al., 2023; Vélez et al., 2023).

4 | DISCUSSION

In this paper, we combine two different techniques, ensemble learning and movement-based background removal, to build a classifier for animal detection that needs only small training data. The first makes it possible to run a large amount of augmented images in parallel, and the second reduces the complexity of images by removing unnecessary pixels. Active learning minimizes the need for extensive human annotation when building the desired filters by enabling the algorithm to determine on behalf of the user which annotations are truly necessary.

Separate candidate and test sets are needed only for investigating performance measures (accuracy, recall and precision). In practice, two filters can be trained with complete datasets without division into candidate and test sets. In this case, the performance can easily be judged visually by investigating sequence classifications using fast-forward videos of camera trap data. Moreover, it is usually rather easy to see what kind of training material is needed for gaining better performance by investigating colour masks of incorrectly classified images.

The ensemble approach is more important at the very beginning of training, when variance in predictions of submodels is large. The size of the ensemble could be lower later on with larger training data, as variance shrinks (see Section ‘Benefits of ensemble averaging’ in Supporting Information).
Our active learning method is not completely objective, as a user has the ability to decide what images should not be included in either positive or negative classes. Excluded colour masks can be, for example, just too difficult for a user to annotate reliably, show too few target pixels or too much background, or a mask can be too similar to already included ones. All of these are subjective choices, and different users may end up drawing different conclusions.

The movement filtering approach works best with long enough sequences or videos when a target is changing place between images (Figure 3). The most difficult cases for the filters to learn are sequences where the movement of animals or humans is mixed with strong vegetation movement. These images support negative and positive classes at the same time (e.g. the empty filter infers vegetation to be in the positive class and a target in the negative class). Therefore, training has to be built in such a way that these cases will emerge in active learning samples as well, so that a classifier learns to ignore any vegetation movement when there is a moving target. Windy sites with a lot of moving animals and humans are best for this purpose. Moving vegetation was reported to be a problem for the Zilong classifier (Wei et al., 2020), but not for our classifier due to the active learning, as we demonstrated with the practical example.

There exist many movement detection methods (Amandeep & Goyal, 2015; Zhao et al., 2022) and some of them may produce clearer masks than our method. However, there is a trade-off: increased quality of target masks makes movement of vegetation also more visible. Ideally, movement masks should make targets visible enough, but at the same time reduce visibility of vegetation movement.

The current movement masking method can be improved. First, detection seems to work worse with darker objects (e.g. people with black clothes). This might be related to our choice of background colour in colour masks (black), and changing it could fix the problem by increasing contrast. Second, current movement mask thresholds exclude barely visible distant targets in IR images. False triggerings are much more common during daytime, due to higher temperature differences in nature. Therefore, the mask threshold value could be set to greater sensitivity during the night as there is less wind. Finally, we only use low-resolution images. Applying the classifier to subimages, thus zooming in (for example 448×448 divided into four subimages), would more easily reveal also slighter movements. Our very limited comparison with windy Lisbon data showed that MegaDetector could not find more animals than our classifier while using low resolution, but with high resolution it finds more. This observation suggests that a subimage approach could make our classifier more accurate.

There are some non-fixable shortcomings in the movement mask approach. If an animal or human does not move or changes its position only slightly, the classifier does not detect it, because the target's movement is not included in colour masks. On the contrary, MegaDetector seems to find animals best if a hint of animal features is present. Squirrels running in vegetation are not easily detected when they appear as dark blurry shapes. Combined, our classifier and a traditional straightforward classifier based on full colour image classification would detect highly blurry animals as well as animals staying perfectly still.

Some models can locate found targets inside images either by drawing a bounding box around them like MegaDetector (Fennell et al., 2022), or highlighting pixels belonging to them like Detectron2 (Mask R-CNN). This would allow finding multiple classes within the same image, for example, human and animal classes. Our classifier is designed only for data preprocessing and to filter out sensitive and unnecessary images.

We are using a probabilistic approach that can estimate uncertainty in classification via median probabilities. Many species identification systems limit their guesses to sure or sure enough cases. A user can for example use our classifier to investigate most species that visited a site during a year by limiting their search to these high-confidence animal images [e.g. by setting $p_{median}$ (empty) < 0.25 and $p_{median}$ (human) < 0.25].

A user can tune the classifier to better predict for his or her data when using an active learning classifier, because the classifier is not fixed and final. However, our novel movement mask approach demands a little more from a user during the image annotation phases because the user has to ensure that colour masks are informative as well as the corresponding colour images.

## AUTHOR CONTRIBUTIONS

Tommi Mononen conceived the ideas and designed the methodology; all authors except Tommi Mononen and Otso Ovaskainen collected the data; Tommi Mononen analysed the data; Tommi Mononen led the writing of the manuscript. Otso Ovaskainen, Bess Hardwick, Adrian Pinder, Patrick Strutzenberger, Stefan Stoll and Valerie Levesque-Beaudin contributed critically to the drafts, and all authors commented on the manuscript.

## AFFILIATIONS

## ACKNOWLEDGEMENTS

CSC-IT Center for Science Ltd. provided data storage and computational resources. The research was funded by the European

<footnote n="1">¹Faculty of Biological and Environmental Sciences, University of Helsinki, Helsinki, Finland; ²Companhia das Lezírias S.A., Samora Correia, Portugal;</footnote>

<footnote n="3">³Faculdade de Ciências, cE3c - Centre for Ecology, Evolution and Environmental Changes and CHANGE - Global Change and Sustainability Institute, Universidade de Lisboa, Lisbon, Portugal; ⁴Ecosystem Science Program, Biodiversity and Conservation Science, Western Australian Department of Biodiversity, Conservation and Attractions, Kensington, Australia; ⁵Centre for Biodiversity Genomics, University of Guelph, Guelph, Ontario, Canada; ⁶Institut de Recherche sur la Biologie de l'Insecte, UMR7261 CNRS - Université de Tours, Tours, France; ⁷Universität Wien, Wien, Austria; ⁸Dead Sea and Arava Science Center, Mitzpe Ramon, Israel; ⁹Biology Centre of Czech Academy of Sciences, Institute of Entomology, České Budějovice, Czech Republic; ¹⁰Faculty of Sciences, University of South Bohemia, České Budějovice, Czech Republic; ¹¹Nationalparkamt Hunsrück-Hochwald, Birkenfeld, Germany; ¹²INRAE, URZF, Orléans, France; ¹³University of Applied Sciences Trier, Environmental Campus Birkenfeld, Birkenfeld, Germany; ¹⁴Institut de Systématique, Évolution, Biodiversité (ISYEB), UMR7205 Muséum national d'Histoire naturelle, CNRS, Sorbonne Université, EPHE, Université des Antilles, Paris, France; ¹⁵Faculty of Biology, University of Duisburg-Essen, Essen, Germany; ¹⁶Six Nations of the Grand River Elected Council, Ohsweken, Canada and ¹⁷Department of Biological and Environmental Science, University of Jyväskylä, Jyväskylä, Finland</footnote>
Union's Horizon 2020 research and innovation programme (grant agreement no. 856506: ERC-synergy project LIFEPLAN) and the Research Council of Finland (grants no. 336212 and 345110 to Otso Ovaskainen). Team Czechia (Katerina Sam) acknowledges GACR 22-17593M. Open access publishing facilitated by Jyvaskylan yliopisto, as part of the Wiley - FinELib agreement.

## CONFLICT OF INTEREST STATEMENT

The authors declare that they have no conflict of interest.

## DATA AVAILABILITY STATEMENT

Our data cannot be made available as they contain images of people, which is sensitive personal data. Example data and simplified Matlab code are available via doi.org/10.5281/zenodo.13767173 (Mononen et al., 2025).

## ORCID

Tommi Mononen https://orcid.org/0000-0002-3603-0813

Brigitte Gottsberger https://orcid.org/0000-0002-6317-306X

Inês T. Rosário https://orcid.org/0000-0003-1604-1133

Katerina Sam https://orcid.org/0000-0002-3436-0579

Elise Sivault https://orcid.org/0000-0001-8970-1804

Patrick Strutzenberger https://orcid.org/0000-0003-3278-5693

## REFERENCES

Amandeep, & Goyal, M. (2015). Review: Moving object detection techniques. *International Journal of Computer Science and Mobile Computing*, 4(9), 345–349.

Beery, S., Van Horn, G., & Perona, P. (2018). Recognition in terra incognita. In V. Ferrari, M. Hebert, C. Sminchiesescu, & Y. Weiss (Eds.), *Computer vision - ECCV 2018. ECCV 2018. Lecture notes in computer science* (Vol. 11220). Springer.

Beluch, W. H., Genewein, T., Nurnberger, A., & Kohler, J. M. (2018). The power of ensembles for active learning in image classification. In *IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA*. https://doi.org/10.1109/CVPR.2018.00976

Bouwmans, T., Javed, S., Sultana, M., & Jung, S. K. (2019). Deep neural network concepts for background subtraction: A systematic review and comparative evaluation. *Neural Networks*, 117, 8–66.

Christiansen, B. (2018). Ensemble averaging and the curse of dimensionality. *Journal of Climate*, 31(4), 1587–1596.

Curry, R., Trotter, C., & McGough, A. S. (2021). Application of deep learning to camera trap data for ecologists in planning/engineering – Can captivity imagery train a model which generalises to the wild? In *IEEE International Conference on Big Data (Big Data). Orlando, FL, USA*. https://doi.org/10.1109/BigData52589.2021.9671661

Delisle, Z. J., Flaherty, E. A., Nobbe, M. R., Wzientek, C. M., & Swihart, R. K. (2021). Next-generation camera trapping: Systematic review of historic trends suggests keys to expanded research applications in ecology and conservation. *Frontiers in Ecology and Evolution*, 9, 617996. https://doi.org/10.3389/fevo.2021.617996

Fauzi, W. A., Nugroho, S. M. S., Yuniarno, E. M., Anggraeni, W., & Purnomo, M. H. (2021). Multiple face tracking using Kalman and Hungarian algorithm to reduce face recognition computational cost. *JAREE (Journal on Advanced Research in Electrical Engineering)*, 5(1), 64–70.

Fennell, M., Beirne, C., & Burton, A. C. (2022). Use of object detection in camera trap image identification: Assessing a method to rapidly and

accurately classify human and animal detections for research and application in recreation ecology. *Global Ecology and Conservation*, 35, e02104.

Leorna, S., & Brinkman, T. (2022). Human vs. machine: Detecting wildlife in camera trap images. *Ecological Informatics*, 72, 101876.

Mitterwallner, V., Peters, A., Edelhoff, H., Mathes, G., Nguyen, H., Peters, W., Heurich, M., & Steinbauer, M. J. (2023). Automated visitor and wildlife monitoring with camera traps and machine learning. *Remote Sensing in Ecology and Conservation*, 10(2), 236–247. https://doi.org/10.1002/rse2.367

Mohammed, A., & Kora, R. (2023). A comprehensive review on ensemble deep learning: Opportunities and challenges. *Journal of King Saud University*, 35(2), 757–774.

Mononen, T., Hardwick, B., Alcobia, S., Barrett, A., Blagoev, G. A., Boyer, S., Gonçalves, P., Gottsberger, B., Groner, E., Ho, C. Y. Y., Houska Tahadlova, M., Kaus-Thiel, A., Krisai-Greilhuber, I., Levesque-Beaudin, V., Lopez-Vaamonde, C., Moreau, S., Mrazova, A., Pentinsaari, M., Pinder, A., ... Ovaskainen, O. (2025). *Image sequence classification for camera trap data: Matlab code and example data*. Zenodo. https://doi.org/10.5281/zenodo.13767173

Naftaly, U., Intrator, N., & Horn, D. (1997). Optimal ensemble averaging of neural networks. *Network: Computation in Neural Systems*, 8(3), 283–296.

Norman, L. M., Bischoff, P. H., Wearn, O. R., Ewers, R. M., Rowcliffe, J. M., Evans, B., Sethi, S., Chapman, P. M., & Freeman, R. (2023). Can CNN-based species classification generalise across variation in habitat within a camera trap survey? *Methods in Ecology and Evolution*, 14, 242–251.

Norouzzadeh, M. S., Morris, D., Beery, S., Joshi, N., Jojic, N., & Clune, J. (2021). A deep active learning system for species identification and counting in camera trap images. *Methods in Ecology and Evolution*, 12(1), 150–166.

Norouzzadeh, M. S., Nguyen, A., Kosmala, M., Swanson, A., Palmer, M. S., Packer, C., & Clune, J. (2018). Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning. *Proceedings of the National Academy of Sciences of the United States of America*, 115(25), E5716–E5725.

Penn, M. J., Miles, V., Astley, K. L., Ham, C., Woodroffe, R., Rowcliffe, M., & Donelly, C. A. (2023). Sherlock – A flexible, low-resource tool for processing camera-trapping images. *Methods in Ecology and Evolution*, 15(1), 91–102.

Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Gupta, B. B., Chen, X., & Wang, X. (2021). A survey of deep active learning. *ACM Computing Surveys*, 54(9), 1–40.

Rogers, H. M. K., Banelyte, G. G., Farrell, A. M., Hardwick, B., Mononen, T., & Kerdraon, D. (2023). *Lifeplan camera trapping protocol V.3, protocols.io*. https://doi.org/10.17504/protocols.io.q26g7pxp1gwz/v3

Rougier, J. (2016). Ensemble averaging and mean squared error. *Journal of Climate*, 29(24), 8865–8870.

Schneider, S., Greenberg, S., Taylor, G. W., & Kremer, S. C. (2020). Three critical factors affecting automated image species recognition performance for camera traps. *Ecology and Evolution*, 10(7), 3503–3517.

Tabak, M. A., Norouzzadeh, M. S., Wolfson, D. W., Newton, E. J., Boughton, R. K., Ivan, J. S., Odell, E. A., Newkirk, E. S., Conrey, R. Y., Stenglein, J., Iannarilli, F., Erb, J., Brook, R. K., Davis, A. J., Lewis, J., Walsh, D. P., Beasley, J. C., Vercauteren, K. C., Clune, J., & Miller, R. S. (2020). Improving the accessibility and transferability of machine learning algorithms for identification of animals in camera trap images: MLWIC2. *Ecology and Evolution*, 10(19), 10374–10383.

Tabak, M. A., Norouzzadeh, M. S., Wolfson, D. W., Sweeney, S. J., Vercauteren, K. C., Snow, N. P., Halseth, J. M., Di Salvo, P. A., Lewis, J. S., White, M. D., Teton, B., Beasley, J. C., Schlichting, P. E., Boughton, R. K., Wight, B., Newkirk, E. S., Ivan, J. S., Odell, E. A., Brook, R. K., ... Miller, R. S. (2019). Machine learning to classify animal species in camera trap images: Applications in ecology. *Methods in Ecology and Evolution*, 10, 585–590.
<figure><img src="image_17.png" /></figure>

Vélez, J., McShea, W., Shamon, H., Castiblanco-Camacho, P. J., Tabak, M. A., Chalmers, C., Fergus, P., & Fieberg, J. (2023). An evaluation of platforms for processing camera-trap data using artificial intelligence. *Methods in Ecology and Evolution, **14***, 459-477.

Wei, W., Luo, G., Ran, J., & Li, J. (2020). Zilong: A tool to identify empty images in camera-trap data. *Ecological Informatics*, **55**, 101021.

Xi, T., Wang, J., Qiao, H., Lin, C., & Ji, L. (2021). Image filtering and labeling assistant (IFLA): Expediting the analysis of data obtained from camera traps. *Ecological Informatics*, **64**, 101355.

Yang, D.-Q., Tan, K., Huang, Z. P., Li, X. W., Chen, B. H., Ren, G. P., & Xiao, W. (2021). An automatic method for removing empty camera trap images using ensemble learning. *Ecology and Evolution*, **11**, 7591-7601.

Yousif, H., Yuan, J., Kays, R., & He, Z. (2019). Animal scanner: Software for classifying humans, animals, and empty frames in camera trap images. *Ecology and Evolution*, **9**(4), 1578-1589.

Zhao, X., Wang, G., He, Z., & Jiang, H. (2022). A survey of moving object detection methods: A practical perspective. *Neurocomputing*, **503**, 28-48.

Data S1. Image information compression and ResNet18.

Benefits of ensemble averaging.

Visualization of the active learning process.

MegaDetector(v5a) performance with Table 2 datasets.

Image stabilization test.

**How to cite this article:** Mononen, T., Hardwick, B., Alcobia, S., Barrett, A., Blagoev, G. A., Boyer, S., Gonçalves, P., Gottsberger, B., Groner, E., Ho, C. C. Y., Houska Tahadlova, M., Kaus-Thiel, A., Krisai-Greilhuber, I., Levesque-Beaudin, V., Lopez-Vaamonde, C., Moreau, S., Mrazova, A., Pentinsaari, M., Pinder, A., ... Ovaskainen, O. (2025). An active ensemble classifier for detecting animal sequences from global camera trap data. *Methods in Ecology and Evolution*, **16**, 2500-2516.
https://doi.org/10.1111/2041-210x.70144

## SUPPORTING INFORMATION

Additional supporting information can be found online in the Supporting Information section at the end of this article.