# A classification-occupancy model based on automatically identified species data

Ryo Ogawa, Frédéric Gosselin, Kevin F A Darras, Stephanie Roilo, Anna F Cord

► To cite this version:

Ryo Ogawa, Frédéric Gosselin, Kevin F A Darras, Stephanie Roilo, Anna F Cord. A classification-occupancy model based on automatically identified species data. Ecology, 2025, 106 (5), pp.e70086. 10.1002/ecy.70086. hal-05207003

HAL Id: hal-05207003

https://hal.inrae.fr/hal-05207003v1

Submitted on 11 Aug 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A classification-occupancy model based on automatically identified species data

Ryo Ogawa<fnref n="1" />,<fnref n="2" /> ID

Frédéric Gosselin<fnref n="3" /> ID

Kevin F. A. Darras<fnref n="2" />,<fnref n="3" /> ID

Stephanie Roilo<fnref n="1" />,<fnref n="2" /> ID

Anna F. Cord<fnref n="1" />,<fnref n="2" /> ID

**Correspondence**

Ryo Ogawa
Email: ryo.ogawa@uni-bonn.de

**Funding information**

Bundesministerium für Bildung und
Forschung, Grant/Award Number:
03LW0079K; Horizon 2020 Framework
Programme, Grant/Award Number:
817501; Deutsche
Forschungsgemeinschaft, Grant/Award
Number: EXC 2070 - 390732324

**Handling Editor:** Elise F. Zipkin

**Abstract**

Occupancy models estimate a species' occupancy probability while accounting for imperfect detection, but often overlook the issue of false-positive detections. This problem of false positives has gained attention recently with the rapid advancement of automated species detection tools using artificial intelligence (AI), which generate continuous confidence scores for each species detection. Novel occupancy models have been introduced that integrate these confidence scores to identify false positives, but these models require thorough assessments of diagnosis and validation. Here, we propose a new occupancy model based solely on AI-detected species data. We conducted simulations to examine the inferential and predictive accuracies with known true parameters and analyzed AI-detected species data to test the practical usefulness through goodness-of-fit tests and evaluation with external data. Our proposed model mostly outperformed alternative models that ignore imperfect detection or false-positive error probabilities in terms of accuracy in simulation analyses and goodness-of-fit tests in the case study, but not in terms of discrimination metrics based on external data. The proposed occupancy model aids in understanding species-habitat relationships and developing automated biodiversity monitoring workflows by accounting for both false-negative and false-positive errors.

**KEYWORDS**

automated biodiversity monitoring, Bayesian statistics, BirdNET, false positive, model integration, passive acoustic monitoring, probabilistic sampling, species classifier

## INTRODUCTION

The field of occupancy models, which estimate a species' occupancy probability while accounting for imperfect detection (MacKenzie et al., 2002)—that is,

false negatives—has been transformed by technological progress. Traditional methods rely on labor-intensive human surveys, resulting in limited data collection. Modern techniques combined with artificial intelligence (AI), in contrast, allow for extensive and standardized

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

© 2025 The Author(s). Ecology published by Wiley Periodicals LLC on behalf of The Ecological Society of America.

<footnote n="1">¹Agro-Ecological Modeling Group,
Institute of Crop Science and Resource
Conservation, University of Bonn, Bonn,
Germany</footnote>

<footnote n="2">²Chair of Computational Landscape
Ecology, TUD Dresden University of
Technology, Dresden, Germany</footnote>

<footnote n="3">³INRAE, UR EFNO,
Nogent-sur-Vernisson, France</footnote>
data collection with less human effort. These include environmental DNA (eDNA) sampling, which uses polymerase chain reaction (PCR) to detect target species presence (Serrao et al., 2018); camera trapping, combined with automated image processing identifying species from captured images (Lonsinger et al., 2024); and passive acoustic monitoring (PAM) coupled with audio recognition models identifying sound-producing animals (Wood et al., 2024). These advancements have deepened our understanding of species-habitat relationships and can help us make a significant step toward automated biodiversity monitoring (Besson et al., 2022).

The advent of automated species detection tools (hereafter called species classifiers) has also revealed a challenge previously overlooked in occupancy models: false positives (i.e., species misidentifications) can lead to an overestimation of species' occupancy estimates, which in turn can cause inaccurate variable inference. While occupancy models accounting for false positives have been developed previously (Chambert, Grant, et al., 2018; Chambert, Waddle, et al., 2018; Miller et al., 2011; Royle & Link, 2006), the development has been limited by the lack of reliable scores to quantify the chance of potential false positives. Today, species classifiers provide continuous confidence scores for each detection (Aodha et al., 2022; Kahl et al., 2021), with higher scores indicating a greater chance of true positives.

To mitigate false positives in occupancy models, deterministic approaches have been applied, such as the use of threshold cutoffs along the continuous confidence score axis (Singer et al., 2024). This simple approach can reduce the bias caused by false positives and has been applied to occupancy models in eDNA sampling (Serrao et al., 2018), camera trap image recognition (Clare et al., 2019), and sound recognition models (Wood et al., 2024; Wood & Kahl, 2024). However, increasing cutoff thresholds not only gradually removes false positives but also increasingly excludes true positives. Consequently, threshold cutoffs at the optimal point do not guarantee the complete removal of misidentifications while keeping all true detections, due to the unknown true threshold and the non-probabilistic nature of arbitrary cutoffs.

As an alternative, a probabilistic sampling approach has been recently introduced in occupancy models (Cole et al., 2022; Kéry & Royle, 2020; Rhinehart et al., 2022). These models have combined the occupancy model with a classification model that probabilistically classifies true and false positives along the continuous confidence score axis (hereafter called classification-occupancy model). This probabilistic approach may provide more accurate estimates of species presence reflecting their inherent uncertainty, unlike the threshold-cutoff approach

which necessarily contains false positives and excludes true positives. However, the failure of parameter convergence is a problem for these complex models that rely on AI-detected data (Kéry & Royle, 2020; Rhinehart et al., 2022), with label switching during the classification process being a major issue (Stephens, 2000). Hence, a novel classification-occupancy model requires thorough assessments of model diagnosis, goodness-of-fit test, and external validation.

We therefore aimed to build a classification-occupancy model using only AI-detected species data, ensuring acceptable model convergence, fitting, and predictability. We first propose a Bayesian classification-occupancy model adapted to AI-detected species data with continuous confidence scores, considering imperfect detection and probabilistic classification (the “FN + FP” model in Figure 1a). For comparison, we constructed three alternative models as follows (Figure 1a):

1. The “Basic” model assumes perfect detection and imposes a threshold to continuous confidence scores for classification.

2. The “FN” (false negatives) model accounts for imperfect detection and imposes a threshold on continuous confidence scores for classification (i.e., standard occupancy models).

3. The “FP” (false positives) assumes perfect detection and uses probabilistic classification.

We assessed the inferential and predictive accuracies of our proposed model through a simulation study and compared it with the three alternatives (Figure 1b). We then fitted the classification-occupancy model to real acoustic survey data from a case study in an agricultural landscape (Figure 1c), assessing its goodness-of-fit and predictive performance validated by external data. We concluded by discussing its practical applications and limitations based on our findings.

# METHODS

## Modeling framework

### Standard occupancy model

In the standard occupancy model (the “FN” model in Figure 1a), input data follow the encounter history format based on a capture-recapture modeling framework (Otis et al., 1978): Let $y_{ij}$ be a species' observation at site $i$ and sampling occasion $j$ (i.e., the $j$th survey visit of site $i$; Appendix S1: Table S1). If the species is observed,
a. Proposed classification-occupancy model and three alternatives

<table><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="2">Detection model</th></tr><tr><th>Perfect detection</th><th>Imperfect detection</th></tr></thead><tbody><tr><td rowspan="2">Classification model<br>Threshold cutoff<br>True positive probability<br>Confidence score</td><td>"Basic" model</td><td>"FN" model<br>(standard<br>occupancy model)</td></tr><tr><td>"FP" model</td><td>"FN + FP" model<br>(proposed model)</td></tr></tbody></table>

b. Simulation study

Scenario 1: Varying extents of overlap

<figure><img src="image_2.png" /></figure>

Scenario 2: Varying false positive error probabilities ($q$): $q = 0.1$ versus $0.5$

c. Case study

<figure><img src="image_3.png" /></figure>

**FIGURE 1** Overview of study workflow to evaluate the classification-occupancy model. (a) General specification of the proposed classification-occupancy model (i.e., "FN + FP" model) and three alternative models based on automatically identified species data. "Basic": model assuming perfect detection and imposing a threshold to reduce false positives; "FN": model estimating false-negative detection parameters; and "FP": model estimating false-positive detection parameters. (b) Scenarios examined in the simulation study, testing the effect of two scale parameters in normal distributions (Scenario 1) and two false-positive error probabilities (Scenario 2) on the accuracy of model estimates across four models. (c) Study area of our case study in Saxony, Germany.

$y_{ij} = 1$, and if not observed, $y_{ij} = 0$. The model depicts the species' site occupancy probability ($\psi_i$) based on an unobservable occupancy status ($z_i$):

$$z_i \sim \text{Bernoulli}(\psi_i) \qquad (1)$$

Next, the model links the observations with the occupancy status, considering imperfect detection ($p_{ij}$) at each site and occasion, and assuming no false-positive detections:

$$y_{ij} \sim \text{Bernoulli}(z_i \times p_{ij}) \qquad (2)$$
Lastly, the model relates the occupancy probabilities and detection probabilities to environmental predictors:

$$ \operatorname{logit}(\psi_i) = \mathbf{X}_{\text{occ}i} \times \beta_{\text{occ}} \quad (3) $$

$$ \operatorname{logit}(p_{ij}) = \mathbf{X}_{\text{det}ij} \times \beta_{\text{det}} \quad (4) $$

where $\mathbf{X}_{\text{occ}i}$ and $\mathbf{X}_{\text{det}ij}$ indicate predictor vectors for occupancy probabilities at site i and for detection probabilities at site i and sampling occasion j, respectively. $\beta_{\text{occ}}$ and $\beta_{\text{det}}$ are the coefficient vectors for occupancy and detection, respectively.

## False-positive occupancy model

In our false-positive occupancy model, the species detection can result from three mutually exclusive events:

1. The species is present at the site and detected: $z_i \times p_{ij}$;

2. The species is present at the site, but the signal from the species was not detected. Instead, another signal, not originating from the target species, was detected—corresponding to a false positive: $z_i \times (1 - p_{ij}) \times w_{ij}$, or

3. The species is absent from the site, and there is a false positive: $(1 - z_i) \times w_{ij}$.

where $w_{ij}$ denotes a binary process that a detection status is true ($w_{ij} = 0$) or false ($w_{ij} = 1$) at site i and occasion j, when $y_{ij} = 1$. The summation of these events, the probability of a species detection, is described as follows:

$$ \begin{aligned} & z_i \times p_{ij} + z_i \times (1 - p_{ij}) \times w_{ij} + (1 - z_i) \times w_{ij} \\ & = z_i \times p_{ij} + (1 - z_i \times p_{ij}) \times w_{ij} \end{aligned} \quad (5) $$

The second event is absent in the original false-positive occupancy model proposed by Royle and Link (2006), but we consider that this event can occur in our case study using a species classifier. In summary, the observation model for species detection events is modified as follows:

$$ y_{ij} \sim \text{Bernoulli}(z_i \times p_{ij} + (1 - z_i \times p_{ij}) \times w_{ij}) \quad (6) $$

$$ w_{ij} \sim \text{Bernoulli}(q_{ij}) \quad (\text{when } y_{ij} = 1) \quad (7) $$

where $q_{ij} \sim \text{Uniform}(0, 1)$, representing a false-positive error probability at site i and occasion j. Note that we fix $w_{ij} = 0$ when $y_{ij} = 0$ so that the non-detection process is explicitly specified as true detection status.

## Classification model

In the classification model, we link the observed confidence scores to parameters in the false-positive occupancy model, specifically $\psi_i$, $p_{ij}$, and $q_{ij}$. Unlike the Gaussian mixture model developed by Kéry and Royle (2020) and Rhinehart et al. (2022), which handle multiple scores per site and occasion, our model uses only one continuous confidence score with each species detection at site i and occasion j (i.e., $x_{ij}$). The process of this classification model is conditional only when a species is detected (i.e., $y_{ij} = 1$). Our modified approach is a pragmatic reason in the case study to facilitate parameter convergence and reduce computational time and memory usage due to input data inflation caused by species detections every 3 s generated by the species classifier.

We assume the confidence scores for each of the true- or false-positive detections conform to a normal distribution (Assumption 4; Appendix S1: Table S2, but see alternative in Cole et al., 2022)—a widely used statistical distribution that can be flexibly adapted from a 0–1 scale by a logit transformation (e.g., the confidence score in BirdNET; Wood & Kahl, 2024) or from a >0 scale by a log transformation (e.g., the copy number in eDNA sampling; Serrao et al., 2018);

$$ x_{ij} | g_{ij} = 1 \sim \text{Normal}(\mu_1, \sigma) \quad (8) $$

$$ x_{ij} | g_{ij} = 2 \sim \text{Normal}(\mu_2, \sigma) \quad (9) $$

where $g_{ij}$ represents the type of positive detections, either true ($g_{ij} = 1$) or false ($g_{ij} = 2$); $\mu_1$ and $\mu_2$ indicate the mean values of true- and false-positive detection confidence scores, respectively, with the constraint that $\mu_1 > \mu_2$ (Assumption 5; Appendix S1: Table S2) with a shared scale parameter $\sigma$ (Assumption 4: Appendix S1; Table S2). We then construct a categorical distribution where every event of true- or false-positive detection at site i and occasion j is sampled from two probabilities: detection is either true (Pr($g_{ij} = 1$)) or false (Pr($g_{ij} = 2$));

$$ g_{ij} \sim \text{Categorical}(\Pr(g_{ij} = 1), \Pr(g_{ij} = 2)) \quad (10) $$

$$ \Pr(g_{ij} = 2) = 1 - \Pr(g_{ij} = 1) \quad (11) $$

## Model integration

We integrate the false-positive occupancy model into the classification model by equating the conditional probability of true positives at site i and occasion j. This is
achieved by dividing the true detection probability by the total detection probability.

$$ 
\begin{aligned}
\Pr(g_{ij} = 1) &= \Pr(\text{detected signal is true}) \\
&= \frac{\psi_i \times p_{ij}}{\psi_i \times p_{ij} + (1 - \psi_i \times p_{ij}) \times q_{ij}}
\end{aligned} 
\quad (12) $$

This model integration structure is similar to the one by Cole et al. (2022) and Kéry and Royle (2020), but it is adjusted for the parameters used in our false-positive occupancy and classification models. Although we use our model integration approach to address practical issues with parameter convergence and input data inflation in our case study, we advise researchers to choose their model integration based on their own study objectives and species classifier used, as long as all conditions of model fitting and computational efficiency are met.

We constructed this proposed model with the Bayesian framework using the software JAGS version 4.3.0 (Plummer, 2003). In both simulation and case study analyses, we set priors of mean confidence scores of true positives: $\mu_1$ ~ Normal(0, 10); mean confidence scores of false positives $\mu_2$ ~ Normal(0, 10) with $\mu_1 > \mu_2$; the scale parameter $\sigma$ ~ Uniform(0, 10); and both occupancy and detection coefficient parameters: $\beta_{occ}$ ~ Normal (0, 10) and $\beta_{det}$ ~ Normal(0, 10).

## Model assumptions

One main objective of this study was to propose a new occupancy modeling framework (i.e., “FN + FP” model). Therefore, we clarified assumptions related to our proposed model (Appendix S1: Table S2). Besides the assumptions in the standard occupancy model (Assumptions 1–3: Appendix S1: Table S2) and aforementioned Assumptions 4 and 5, we included Assumption 6 for the classification model: Confidence scores exclusively indicate the reliability of true-positive detections. This assumption might not hold if the scores are affected by factors like the distance between monitoring devices and species locations (Knight & Bayne, 2019; Somervuo et al., 2023). Additionally, sound recognition models can non-randomly misidentify species due to the presence of a non-target species with similar vocal characteristics in acoustic surveys (Funosas et al., 2024). This can make it difficult to accurately identify the target species when other non-target species are vocalizing. Considering these potential assumption violations in our case study, we evaluated whether our proposed classification-occupancy model showed practical usefulness.

## Comparison of the proposed model with alternative models

In both simulation and case studies, we compared our proposed model (i.e., the “FN + FP” model) with three alternative models (i.e., the “Basic,” “FN,” and “FP” models; Figure 1a). Classification-occupancy models (or non-threshold models; i.e., “FP” and “FN + FP” models) were constructed based on Equations (1) and (3)–(12), whereas threshold-cutoff models (i.e., “Basic” and “FN” models) were constructed based on Equations (1)–(4).

For the “Basic” and “FN” models, encounter history input data were generated using a threshold based on continuous scores, which minimized both false-negative and false-positive error rates. Scores larger than the threshold were considered as observed, whereas those smaller were considered as not observed. In the simulation study, the threshold was selected based on prior knowledge of $\mu_1$ and $\mu_2$ (see detail in Simulation study). In the case study, it was determined based on the dataset of expert verification, with corresponding continuous scores generated by the sound recognition model BirdNET (Kahl et al., 2021; see detail in Case study). Note that in real data, multiple true and false detections can happen per site and sampling occasion, but our model structure treats any detection, whether single or multiple, as one detection per site and occasion as observed ($y_{ij} = 1$).

Both “Basic” and “FP” models assumed perfect detection (i.e., $p = 1$). Although it was rare to assume perfect detection in occupancy models, this comparison allowed us to disentangle the effect of the respective model structures when presenting novel occupancy models: for example, Doser et al. (2023) fitted six different models, two of which assumed perfect detection, when presenting their multi-species occupancy model. Additional details of these “Basic” and “FP” models are given in Appendix S1: Section S1. The JAGS language scripts of all models and subsequent simulation and case study codes are available in Ogawa (2024).

## Simulation study

In our simulation study, we aimed to assess the inherent behavior of our proposed model and to contrast it with alternative models by using known true parameters (Saas & Gosselin, 2014). When generating simulation data, we fixed the mean confidence scores for true positives, $\mu_1 = 1$, and false positives, $\mu_2 = -1$. This allowed us to determine the optimal threshold point for the “Basic” and “FN” models in advance: Detection $y_{ij} = 1$ when $x_{ij} > (\mu_1 + \mu_2)/2 = 0$, and non-detection $y_{ij} = 0$ when $x_{ij} \le 0$.
We assigned two occupancy coefficients (i.e., intercept and one predictor) and one detection coefficient (i.e., intercept only), both randomly selected from a uniform distribution ranging between -1 and 1. The numbers of sites and occasions were set to 50 and 20, respectively, to align with site and occasion numbers in our case study.

We tested two scenarios to understand how accurately the four models predicted the estimates of the true parameter values (Figure 1b). First, we assigned the scale parameter ($\sigma$) of the normal distribution to be a value of either 0.5 or 1.0. We expected that the “FN + FP” model would perform better than the “FN” model (i.e., standard occupancy model) when there was greater mixing of true- and false-positive distributions along the confidence score axis. This was because the threshold-cutoff approach could not efficiently separate true- and false-positive detections when the overlap between the two distributions was large. Second, we set the false-positive error probability (i.e., $q$) at either 0.1 or 0.5. We expected that the “FN + FP” model would more accurately predict the true parameter values when the false-positive probability was higher (i.e., $q = 0.5$) than when the false-positive probability was lower (i.e., $q = 0.1$). This was because a lower occurrence of false positives could lead to data scarcity in false-positive detections when distinguishing between true and false positives in the classification model. Based on these parameter settings, we generated 100 simulated data points for each combination of $\sigma$ and $q$.

In our simulation study, we evaluated the success rate of parameter convergence across four models (Figure 1a) under equal conditions. This allowed us to fairly assess how likely each model reached convergence with different model complexities. Therefore, we ran three chains, each with 5000 burn-in iterations, 50,000 posterior samples after burn-in, and a thinning rate of one, resulting in a total of 150,000 Markov chain Monte Carlo (MCMC) posterior samples for each simulation. We examined parameter convergence using Gelman–Rubin statistics R-hat (Gelman et al., 2014). If we found R-hat >1.1 for at least one parameter in a simulation, we considered the model a failure and calculated the success rate of model convergence (in percent). Low numbers of effective sample sizes also produce unreliable parameter estimates and 95% credible interval (CRI). Therefore, given the models that showed R-hat <1.1 for all parameters (i.e., successfully converged models), we extracted the lowest effective sample size among all monitored parameters per simulation and calculated the mean of the values over the simulations of successfully converged models. To assess the accuracy of parameter estimation, we evaluated the 95% CRI coverage rates of occupancy probabilities ($\psi$: a vector of occupancy

probabilities $\psi_i$) and coefficients ($\beta_{occ}$) (i.e., how often 95% CRIs of estimates overlapped with true parameters). We lastly calculated the mean of the root mean square errors (RMSEs) between the true and estimated $\psi$ and $\beta_{occ}$ as another quantitative accuracy measure.

## Case study

### Model species and region

We evaluated the practical usefulness of the classification-occupancy model in real data analysis by predicting the occupancy of five farmland bird species, namely Eurasian skylark (*Alauda arvensis*), common whitethroat (*Curruca communis*), yellowhammer (*Emberiza citrinella*), Eurasian tree sparrow (*Passer montanus*), and western yellow wagtail (*Motacilla flava*) in Saxony, Germany (Figure 1c). These species were the most frequently observed in line-transect surveys conducted by humans, providing a sufficient sample size of external test data for our case study (Appendix S1: Table S3). The study area is a predominantly agricultural region with a total area of 285.3 km², comprising 49.2% cultivated crops, 15.7% agricultural grassland, 26.2% forest, 4.3% urban areas, 1.5% water bodies, and 2.0% wetland (Malinowski et al., 2020).

### Data collection and preprocessing

Line-transect bird surveys were conducted by ornithologists at 18 locations (Figure 1c) during two periods: the first from May 16 to 19, 2022, and the second from June 5 to 9, 2022. During the transect surveys, 49 automated sound recorders (AudioMoth 1.1 and 1.2, Hill et al., 2019) were deployed along the transects (Figure 1c). We analyzed bird vocalization data in recordings collected between 05:30 AM and 11:30 AM in the period from May 23 to 25. We split the survey period into 1-h time bins as the unit of sampling occasion *j*. Thus, the total numbers of sampling sites and hours of recordings were 49 and 18 (3 days × 6 h-time-bins), respectively. The minimum distance between automated recorders was 383 m, which is greater than the estimated sound extinction distance among varying frequencies and microphone directions, determined by our field calibration (i.e., 160 m; see details in Appendix S1: Section S3 and Table S4; Darras et al., 2016). Therefore, we assumed the distance between the sampling sites to be large enough to avoid spatial autocorrelation of site occupancy probabilities.

We used the deep-learning convolutional neural network (CNN) BirdNET Analyzer version 2.2 to automatically
identify bird vocalizations in sound recordings (Kahl et al., 2021). All BirdNET settings were kept at their default values for species detection (e.g., minimum confidence score = 0.1 and overlap of prediction segments = 0.0). We used the centroid coordinates of the region (i.e., 51.5° N, 12.6° E) as input. BirdNET annotated each 3-s snippet with species names and their confidence scores.

For “Basic” and “FN” models, annotations were aurally and visually verified on spectrograms by the same ornithologists who conducted transect surveys using the review mode of an online software tool (ecoSound-web, Darras et al., 2024). The resulting classified annotations were further grouped into true-positive or false-positive detections (see details in Appendix S1: Section S2). Using these verified annotations with their associated confidence scores, we determined the optimal threshold per species, which is defined as the point that maximizes the sum of the true-positive rate (sensitivity) and the true-negative rate (specificity).

For “FP” and “FN + FP” models, we selected the detection with the highest confidence score within each 1-h sampling interval, in cases where multiple detections were recorded. Note that we did not use any verified annotations to build the non-threshold “FP” and “FN + FP” models, where the optimal threshold is priorly unknown. We transformed the BirdNET confidence score using a logit function (Wood & Kahl, 2024). We then fitted the transformed score into the classification model.

We examined the effect of seven land cover types (Malinowski et al., 2020, based on Sentinel-2 remote sensing data, year 2017) as well as small woody features (European Environment Agency, 2023, years 2018–2021; see details in Appendix S1: Section S4) on the occupancy probability. Land cover information was extracted within a 160-m radius around the locations of the sound recorders, a radius determined through our field calibration method (Appendix S1: Section S3; Darras et al., 2016). To avoid multicollinearity for these variables, we conducted a principal components analysis on these variables (Zhang et al., 2023) and selected the first three components as predictors of the occupancy probabilities. Lastly, to account for wind sound noises that could reduce the chance of species detection by BirdNET, we modeled the detection probability as a function of hourly wind speed data, downloaded from the fifth generation European Centre for Medium-Range Weather Forecast (ECMWF) reanalysis for global climate and weather (Hersbach et al., 2023).

## Model diagnosis

In Bayesian occupancy models, the goodness-of-fit is tested by comparing the observed detection data with the

data that the model replicated. This comparison is conducted using a method called posterior predictive checks (Gelman et al., 1996; MacKenzie et al., 2018). We summed species detections over hours of recording for each site and calculated the chi-square discrepancy measure for occupancy models (Kéry & Royle, 2016). For the “FP” and “FN + FP” models, we also calculated the sum of square residuals for classification models. We considered the posterior predictive *p*-value (PPP) < 0.05 or PPP > 0.95 as a lack of fit.

## Model evaluation

Using a species classifier (e.g., BirdNET) for both training and testing datasets could inflate the model’s predictive performance if the species classifier incorrectly identifies non-target species as the target during both model building and evaluation processes in a similar manner (i.e., the violation of Assumption 6 in Appendix S1: Table S2; Grimm et al., 2015). To avoid such issues, we used species data collected through line-transect surveys conducted by humans to evaluate the models, where spatial and temporal coverages are matched to the locations of deployed sound recorders (Figure 1c). Line-transect data may not be the best dataset for testing model performance due to the potential inflation of pseudo-absences in the testing data. However, no dataset in bird surveys can provide true absence information, making it difficult to fully validate occupancy probability in the case study. Therefore, we considered this external data just to identify poorly predictive models, rather than to indicate how well the model predicted.

Since line-transect surveys are different sampling schemes from PAM, they offer complete independence in sampling methods and data characteristics, which can lead to a bias toward more conservative predictive performance rather than optimistic results. Acceptable predictive performance under such conditions can reflect its ability to generalize to new data. We believe this conservative approach is crucial when evaluating the usefulness of novel models: whether a model trained with data from one sampling scheme can predict the pattern of testing data from another sampling scheme.

In this process, we compared predicted species' presence and pseudo-absence sampled from line-transect survey data (see details in Appendix S1: Section S5) and calculated three discrimination metrics: the area under the receiver operating characteristic curve (AUC), true skill statistic (TSS), and Sørensen's similarity index (F-measure). AUC, a threshold-independent value, ranges from 0 to 1 (Shabani et al., 2018) and indicates a rank-based measure of binary classification accuracy. TSS ranges from -1 to 1 and is calculated by summing the true-positive rate (sensitivity)
and the true-negative rate (specificity) and then subtracting one (Allouche et al., 2006). We used the maximum value of the sum of sensitivity and specificity. F-measure balances the accuracy of positive predictions (precision) and sensitivity (i.e., recall), with scores ranging from 0 to 1 (Sørensen, 1948). We selected the maximum of the balanced mean of precision and sensitivity. For all these metrics, higher scores indicate better model performance.

To gain an adequate effective sample size (>1000) and parameter convergence (R-hat < 1.1; Gelman et al., 2014) with optimal numbers of burn-in (set to a minimum of 5000), posterior samples (set to a minimum to 10,000), and thinning rates (set to a minimum of 1 and a maximum of 1000), we used the R package “runMCMCBtadjust” that automatically determines these optimal MCMC settings based on the desired effective sample size (Gosselin, 2024). We also visually assessed traceplots for $\beta_{occ}$, $\beta_{det}$, $q_{ij}$, $\mu_1$, $\mu_2$, and $\sigma$ to ensure the MCMC parameter convergence.

All analyses were conducted using R version 4.4.0 and the following packages: “dismo,” “dplyr,” “ggplot2,” “runjags,” “lubridate,” “mcmcplots,” “sf,” “stringr,” and “terra” (Curtis, 2018; Denwood, 2016; Grolemund & Wickham, 2011; Hadley, 2016; Hijmans, 2022; Hijmans et al., 2021; Pebesma, 2018; R Development Core Team, 2024; Wickham, 2022; Wickham et al., 2023). We provide a step-by-step guide to reproduce the method and results of our case study and to be adapted to other case studies and species classifiers (Appendix S2).

# RESULTS

## Simulation study

Our proposed model (i.e., “FN + FP”) had a success rate of parameter convergence ranging from 70% to 90% (Table 1). The effective sample sizes ranged from 492 to 769, except for the case at $\sigma = 0.5$ and $q = 0.1$ (effective sample size = 33,383; Table 1). Although these values were lower than those of three alternative models, the “FN + FP” model achieved 95% CRI coverage rates of over 80% for both $\psi$ and $\beta_{occ}$, except in the scenario at $\sigma = 0.5$ and $q = 0.1$ where they attained 44.83%. In contrast, none of the alternative models exceeded coverage rates greater than 50% for either occupancy probabilities ($\psi$) or coefficients ($\beta_{occ}$) for any combinations of $\sigma$ and $q$ (Table 1). The “FN + FP” model had the lowest RMSEs for $\psi$ among all four models, although this was not the case for $\beta_{occ}$ when $\sigma = 1$ (Table 1): the “Basic” model showed the lowest RMSE for $\beta_{occ}$ among all four models when $\sigma = 1$, whereas the “FN + FP” models had the lowest RMSE for $\beta_{occ}$ when $\sigma = 0.5$ (Table 1).

# Case study

To achieve adequate effective sample size (>1000) in the “FN + FP” model, we required thinning rates ranging from 666 to 1000 and a total number of posterior samples from 794,731 to 7,663,302 (Appendix S1: Table S5). The “FN + FP” models showed acceptable goodness-of-fit for both chi-square and sum of square discrepancy measures across all species (i.e., $0.05 < \text{PPP} < 0.95$; Figure 2a and Appendix S1: Figures S1 and S2). Chi-square discrepancy measures were less than 0.05 for both “Basic” and “FN” models across all species (Figure 2a). The “FP” model showed acceptable goodness-of-fit for the Eurasian tree sparrow (i.e., $0.05 < \text{PPP} < 0.95$), but not for other species (Figure 2a).

The “FN + FP” model did not exhibit any noticeable trends of improved predictive performance compared with the alternative models (Figure 2b-d). The “FN + FP” model for the common whitethroat and Eurasian tree sparrow showed lower predictive performance compared with the Eurasian skylark, yellowhammer, and western yellow wagtail (Figure 2b-d). For all species, the predicted species occupancy and their associated uncertainty maps varied largely across the proposed and alternative models (Figure 3 and Appendix S1: Figures S3-S6), where the uncertainty was measured as the SD of species' occupancy probabilities over MCMC posterior samples at each grid cell. For example, the Eurasian skylark distribution maps differed between the “FN + FP” model and the alternative “Basic” and “FN” models (Figure 3). For all models, coefficients of intercept and wind speed in detection models and true-positive probability along BirdNET confidence score for five species are shown in Appendix S1: Figure S7.

# DISCUSSION

The simulation study highlighted the advantage of the proposed model (i.e., “FN + FP” model) over alternative models, either when there was a large extent of overlap between true- and false-positive detections along the confidence score axis or when the occurrence of false-positive error probability was high. The case study results demonstrated that (1) alternative models mostly failed to meet the criteria of goodness-of-fit and that (2) there were species-specific cases where classification-occupancy models showed poor predictive performance even when the goodness-of-fit was acceptable. These results highlight the negative consequence of ignoring false negatives and positives in simulation analyses (Table 1). Furthermore, the model fitting
<table><caption>TABLE 1 Simulation analysis results for the proposed (“FN + FP”) and alternative (“Basic,” “FN,” and “FP”) models.</caption><thead><tr><th>Metric</th><th>σ</th><th>q</th><th>Basic</th><th>FN</th><th>FP</th><th>FN + FP</th></tr></thead><tbody><tr><td rowspan="4">Convergence success rate<br>(all R-hat &lt; 1.1; unit: %)</td><td>0.5</td><td>0.1</td><td>100</td><td>99</td><td>99</td><td>82</td></tr><tr><td>0.5</td><td>0.5</td><td>100</td><td>98</td><td>100</td><td>70</td></tr><tr><td>1</td><td>0.1</td><td>100</td><td>100</td><td>99</td><td>90</td></tr><tr><td>1</td><td>0.5</td><td>100</td><td>99</td><td>98</td><td>82</td></tr><tr><td rowspan="4">Lowest effective sample size</td><td>0.5</td><td>0.1</td><td>47,146</td><td>18,626</td><td>13,051</td><td>33,383</td></tr><tr><td>0.5</td><td>0.5</td><td>47,669</td><td>16,387</td><td>28,089</td><td>769</td></tr><tr><td>1</td><td>0.1</td><td>46,490</td><td>20,864</td><td>9998</td><td>492</td></tr><tr><td>1</td><td>0.5</td><td>48,967</td><td>20,880</td><td>5053</td><td>804</td></tr><tr><td rowspan="4">95% CRI coverage rate (ψ) (unit: %)</td><td>0.5</td><td>0.1</td><td>2.26</td><td>3.64</td><td>0.18</td><td><strong>44.83</strong></td></tr><tr><td>0.5</td><td>0.5</td><td>1.58</td><td>1.06</td><td>1.24</td><td><strong>93.20</strong></td></tr><tr><td>1</td><td>0.1</td><td>1.76</td><td>4.86</td><td>0.06</td><td><strong>94.22</strong></td></tr><tr><td>1</td><td>0.5</td><td>4.82</td><td>0.83</td><td>4.04</td><td><strong>84.46</strong></td></tr><tr><td rowspan="4">RMSE (ψ)</td><td>0.5</td><td>0.1</td><td>0.27</td><td>0.41</td><td>0.50</td><td><strong>0.12</strong></td></tr><tr><td>0.5</td><td>0.5</td><td>0.24</td><td>0.52</td><td>0.25</td><td><strong>0.07</strong></td></tr><tr><td>1</td><td>0.1</td><td>0.28</td><td>0.40</td><td>0.50</td><td><strong>0.18</strong></td></tr><tr><td>1</td><td>0.5</td><td>0.23</td><td>0.51</td><td>0.25</td><td><strong>0.11</strong></td></tr><tr><td rowspan="4">95% CRI coverage rate (β<sub>occ</sub>) (unit: %)</td><td>0.5</td><td>0.1</td><td>21.00</td><td><strong>49.49</strong></td><td>48.48</td><td>47.56</td></tr><tr><td>0.5</td><td>0.5</td><td>19.00</td><td>47.96</td><td>20.00</td><td><strong>95.71</strong></td></tr><tr><td>1</td><td>0.1</td><td>20.50</td><td>48.00</td><td>48.48</td><td><strong>96.67</strong></td></tr><tr><td>1</td><td>0.5</td><td>13.50</td><td>50.00</td><td>34.69</td><td><strong>96.34</strong></td></tr><tr><td rowspan="4">RMSE (β<sub>occ</sub>)</td><td>0.5</td><td>0.1</td><td>0.93</td><td>2.81</td><td>10.45</td><td><strong>0.40</strong></td></tr><tr><td>0.5</td><td>0.5</td><td>0.82</td><td>8.18</td><td>0.85</td><td><strong>0.41</strong></td></tr><tr><td>1</td><td>0.1</td><td><strong>0.98</strong></td><td>2.54</td><td>10.25</td><td>2.09</td></tr><tr><td>1</td><td>0.5</td><td><strong>0.74</strong></td><td>8.80</td><td>0.93</td><td>1.17</td></tr></tbody></table>

Note: Data were generated from four scenarios, each combining two scale parameters ($\sigma$ = 0.5 or 1.0) and two false-positive error probabilities ($q$ = 0.1 or 0.5) in the classification model. The lowest effective sample size and two accuracy metrics were computed only when models converged at R-hat <1.1. The 95% CRI coverage rate indicate how often 95% CRI of estimates overlapped with true parameters out of converged iterations, while RMSEs from the true value for both occupancy probabilities ($\psi$) and coefficients ($\beta_{occ}$) were averaged over converged iterations. The numbers highlighted in bold in coverage rates and RMSE represent the highest accuracy model at each scenario across four models.

Abbreviations: CRI, credible interval; FN, false negative; FP, false positive; RMSE, root mean square error.

result indicated that only the “FN + FP” model was acceptable to interpret estimates in our case study (Figure 2a).

## Simulation study

Our proposed model maintained over 70% of convergence success rates and achieved high 95% CRI coverage rates and low RMSEs for $\psi$ and $\beta_{occ}$, despite its complexity. In a post hoc analysis, we extended posterior sampling up to 2 days, but not all simulations reached parameter convergence (R-hat > 1.1). This suggests that certain randomly generated parameter values caused non-convergence. Conversely, alternative models achieved nearly 100% of convergence success rates but provided inaccurate

estimates. This indicates that while our proposed model may struggle with parameter convergence in some cases, it avoids providing inaccurate parameter estimates by checking parameter convergence.

Unexpectedly, the “Basic” model had the lowest RMSE for $\beta_{occ}$ when the overlap between true- and false-positive detections along the confidence score axis was large at $\sigma$ = 1.0. However, the 95% CRI coverage rates of the “Basic” model at $\sigma$ = 1.0 were at most 20.50%, indicating that the 95% CRI coverage rate for $\beta_{occ}$ was overly precise. RMSE can indicate the degree of deviation between true and estimated parameters, but cannot reflect the direction of bias (Lotterhos et al., 2022). Therefore, RMSE alone may not be the most effective metric for evaluating model accuracy in
<figure><img src="image_4.png" /><figcaption>**FIGURE 2** Model diagnosis and evaluation for "Basic," "FN," "FP," and "FN + FP" models on the example of the Eurasian skylark (*Alauda arvensis*), common whitethroat (*Curruca communis*), yellowhammer (*Emberiza citrinella*), Eurasian tree sparrow (*Passer montanus*), and western yellow wagtail (*Motacilla flava*). (a) Posterior predictive *p*-values (PPP) based on chi-square discrepancy measure. (b) Area under the receiver operating characteristic curve (AUC), (c) true skill statistic (TSS), and (d) Sørensen's similarity index (F-measure). Low values of PPP indicate an inconsistency between the model and data based on chi-square discrepancy, while greater values of AUC, TSS, and F-measure indicate higher predictive performance of the models.</figcaption></figure>

the comparative analyses among the four models. Instead, considering both 95% CRI coverage rates and RMSE can ensure a more balanced and accurate interpretation of simulation analysis results, both of which were also used in the assessment to introduce a novel occupancy model in the past (e.g., Doser et al., 2023).

## Case study

The case study demonstrated successful parameter convergence (i.e., $\beta_{occ}$, $\beta_{det}$, $q_{ij}$, $\mu_1$, $\mu_2$, $\sigma$) in the "FN + FP"

model of all species. This convergence success can be attributed to the long duration of MCMC posterior sampling, which is a strength of MCMC-based estimation in hierarchical models when the sample size is low (Rajaratnam & Sparks, 2015). The residual plot of chi-squared discrepancy measures detected multiple local optima in observed residuals (Appendix S1: Figure S1). The binary latent variables ($z_{ij}$ and $w_{ij}$) might have been trapped in local optima during the long posterior sampling without disrupting the MCMC parameter convergence of the monitored priors. Additionally, our method for estimating posterior predictive Bayesian *p*-value tends
<figure><img src="image_5.png" /><figcaption>**FIGURE 3** Maps of occupancy probability and its SD for the Eurasian skylark (*Alauda arvensis*) to show model prediction and its uncertainty with “Basic,” “FN,” “FP,” and “FN + FP” models (see Figure 1a for difference). SD is calculated from Markov chain Monte Carlo (MCMC) posterior samples. FN, false negative; FP, false positive.</figcaption></figure>

to be conservative when accepting goodness-of-fit (Conn et al., 2018; Gosselin, 2011). Nevertheless, most alternative models did not meet the criteria of the posterior predictive check given its conservative approach (Figure 2a). This indicates that ignoring false-negative and false-positive error probabilities in AI-detected species data might have caused issues with goodness-of-fit in the alternative occupancy models.

Discrimination metrics (i.e., AUC, TSS, and F-measure) are derived from classification rates, which are based on predicted occupancy probabilities and presence/ pseudo-absence in the testing dataset. Even if the values of the discrimination metrics satisfy a general criterion (e.g., AUC > 0.7 used as a common threshold), these metrics do not necessarily evaluate the absolute values of occupancy probabilities. Instead, they measure the correlations between estimated and

actual occupancies (MacKenzie et al., 2018). The value of AUC can also be inflated when pseudo-absence points are used as absence points (Lotterhos et al., 2022). That may be the reason why predicted species distribution maps of the Eurasian skylark and their associated uncertainties differed between the proposed model and alternatives (Figure 3), while AUC scores were similar across the four models for the Eurasian skylark (Figure 2b). Therefore, we recommend examining goodness-of-fit tests before evaluating discrimination metrics.

Rather than emphasizing how well the model fits through discrimination metrics, we placed high importance on the species data collected through line-transect surveys conducted by humans as an external dataset to eliminate “FN + FP” models which acceptably fitted the data but yielded low predictive performance. For example, the common whitethroat and Eurasian tree sparrow
had low predictive performance in our discrimination metrics (Figure 2b–d). For the common whitethroat, this could be because of the violation of Assumption 4, assuming normal distributions in the classification model (see the high frequency of confidence score clustered around 0.9–1.0 in the histogram compared with other species in Appendix S1: Figure S8b). One solution to this problem could be to switch from normal to beta distributions (Cole et al., 2022). Alternatively, using different scale parameters for the Gaussian mixture model might also be considered (Rhinehart et al., 2022).

Detections of the Eurasian tree sparrow might have been mixed up with the similar vocal traits of the house sparrow (*Passer domesticus*), which would violate Assumption 6: The confidence score exclusively indicates the reliability of true species detection. Among our five target species, only the BirdNET scores of the Eurasian tree sparrow had positive relationships with those of the co-identified house sparrow (Appendix S1: Figure S9d). Indeed, BirdNET can accidentally co-identify two or more species that have similar vocal characteristics (Funosas et al., 2024). As one analytical solution, one could modify a model structure that addresses species misidentifications (Chambert, Grant, et al., 2018; Chambert, Waddle, et al., 2018). We kept our classification-occupancy model flexible enough to implement other types of occupancy models (see “Solutions” column in Appendix S1: Table S2) so that experienced model developers can further modify our proposed model structure with the practical guideline documented in Appendix S2.

The next step is to compare our proposed model with a standard occupancy model, with experts examining all snippets at each hour of recording, to evaluate how accurately our model handles false positives. While we only verified a limited number of annotated snippets and could not provide a comparative study, we stress that such an analysis would robustly prove whether our proposed model can infer species–habitat relationships and predict species distributions.

## CONCLUSIONS

We conducted two separate evaluations of the proposed model, using both simulation and case studies. These evaluations provided a more comprehensive assessment of the model’s robustness from both a theoretical and practical standpoint. The simulation study highlighted the potential adverse consequence of overlooking false negatives and false positives in occupancy models using AI-detected species data. The case study results demonstrated that the classification-occupancy model

outperformed alternative models in terms of goodness-of-fit, but not discrimination metrics based on external data.

Although further research is needed to address the limitations identified in our case study, our findings underscore the promise of integrating AI approaches with occupancy modeling. The proposed model may be particularly advantageous over the three alternative models tested in cases where accurate ecological inference and predictability are crucial. To ensure the reliability of occupancy maps when applying the proposed model to other case studies, we recommend (1) testing the correlations of confidence scores between co-identified species and (2) examining the histogram of confidence scores for the normality assumption in the classification model.

## AUTHOR CONTRIBUTIONS

**Ryo Ogawa:** Conceptualization; methodology; software; validation; formal analysis; investigation; data curation; writing—original draft; writing—review and editing; visualization. **Frédéric Gosselin:** Methodology; writing—review and editing. **Kevin F. A. Darras:** Methodology; investigation; writing—review and editing. **Stephanie Roilo:** Investigation; writing—review and editing. **Anna F. Cord:** Conceptualization; resources; project administration; funding acquisition; writing—review and editing.

## ACKNOWLEDGMENTS

We thank Lisanne Hölting, Jan Engler, Julian Wendler, Nina Hagemann, Stefan Hannabach, and Michael Bokämper for conducting the fieldwork. Stefan Hannabach and Michael Bokämper contributed to the transect survey and annotations of species’ vocalization data. We thank Elise Zipkin and two anonymous reviewers for their constructive comments, which greatly helped to improve the manuscript. We also thank the Centre for Information Services and High Performance Computing (ZIH) of TUD—Dresden University of Technology for generous allocation of computing resources. This research was made possible by funding from the German Federal Ministry of Education and Research (BMBF) within the Research Initiative for the Conservation of Biodiversity (FEdA) for the project ECO²SCAPE (Funding code 03LW0079K), by the European Union’s Horizon 2020 research and innovation program (BESTMAP, Grant agreement no. 817501), and by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy – EXC 2070 – 390732324. The views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them. The responsibility for the content of this publication lies with the authors. We used the AI-based
BirdNET sound recognition model to automatically identify species in the data analysis. Open Access funding enabled and organized by Projekt DEAL.

## CONFLICT OF INTEREST STATEMENT

The authors declare no conflicts of interest.

## DATA AVAILABILITY STATEMENT

Data and code (Ogawa, 2024) are available in Zenodo at https://doi.org/10.5281/zenodo.12802535.

## ORCID

Ryo Ogawa https://orcid.org/0000-0002-6991-5976

Frédéric Gosselin https://orcid.org/0000-0003-3737-106X

Kevin F. A. Darras https://orcid.org/0000-0002-9013-3784

Stephanie Roilo https://orcid.org/0000-0002-1826-9978

Anna F. Cord https://orcid.org/0000-0003-3183-8482

## REFERENCES

Allouche, O., A. Tsoar, and R. Kadmon. 2006. "Assessing the Accuracy of Species Distribution Models: Prevalence, Kappa and the True Skill Statistic (TSS)." *Journal of Applied Ecology* 43: 1223–32. https://doi.org/10.1111/j.1365-2664.2006.01214.x.

Aodha, O. M., S. M. Balvanera, E. Damstra, M. Cooke, P. Eichinski, E. Browning, M. Barataud, et al. 2022. "Towards a General Approach for Bat Echolocation Detection and Classification." *bioRxiv*. https://doi.org/10.1101/2022.12.14.520490.

Besson, M., J. Alison, K. Bjerge, T. Gorochowski, T. Høye, T. Jucker, H. Mann, and C. Clements. 2022. "Towards the Fully Automated Monitoring of Ecological Communities." *Ecology Letters* 25: 2753–75. https://doi.org/10.1111/ele.14123.

Chambert, T., J. H. Waddle, D. A. W. Miller, S. C. Walls, and J. D. Nichols. 2018. "A New Framework for Analysing Automated Acoustic Species Detection Data: Occupancy Estimation and Optimization of Recordings Post-Processing." *Methods in Ecology and Evolution* 9: 560–570. https://doi.org/10.1111/ 2041-210X.12910.

Chambert, T., E. H. C. Grant, D. A. Miller, J. D. Nichols, K. P. Mulder, and A. B. Brand. 2018. "Two-Species Occupancy Modelling Accounting for Species Misidentification and Non-Detection." *Methods in Ecology and Evolution* 9: 1468–77. https://doi.org/10.1111/2041-210X.12985.

Clare, J. D. J., P. A. Townsend, C. Anhalt-Depies, C. Locke, J. L. Stenglein, S. Frett, K. J. Martin, A. Singh, T. R. Van Deelen, and B. Zuckerberg. 2019. "Making Inference with Messy (Citizen Science) Data: When Are Data Accurate Enough and How Can They Be Improved?" *Ecological Applications* 29: e01849. https://doi.org/10.1002/eap.1849.

Cole, J. S., N. L. Michel, S. A. Emerson, and R. B. Siegel. 2022. "Automated Bird Sound Classifications of Long-Duration Recordings Produce Occupancy Model Outputs Similar to Manually Annotated Data." *Ornithological Applications* 124: 1–15. https://doi.org/10.1093/ornithapp/duac003.

Conn, P. B., D. S. Johnson, P. J. Williams, S. R. Melin, and M. B. Hooten. 2018. "A Guide to Bayesian Model Checking for

Ecologists." *Ecological Monographs* 88: 526–542. https://doi.org/10.1002/ecm.1314.

Curtis, S. M. 2018. "mcmcplots: Create Plots from MCMC Output." https://cran.r-project.org/web/packages/mcmcplots/index.html.

Darras, K. F. A., N. Pérez, L. Dilong, T. Hanf-Dressler, M. Markolf, T. C. Wanger, and A. F. Cord. 2024. "ecoSound-Web: An Open-Source, Online Platform for Ecoacoustics." *F1000Research* 9: 1224.

Darras, K. F. A., P. Pütz, Fahrrurrozi, K. Rembold, and T. Tscharntke. 2016. "Measuring Sound Detection Spaces for Acoustic Animal Sampling and Monitoring." *Biological Conservation* 201: 29–37. https://doi.org/10.1016/j.biocon.2016.06.021.

Denwood, M. J. 2016. "runjags: An R Package Providing Interface Utilities, Model Templates, Parallel Computing Methods and Additional Distributions for MCMC Models in JAGS." *Journal of Statistical Software* 71(1): 1–25. https://doi.org/10.18637/jss.v071.i09.

Doser, J. W., A. O. Finley, and S. Banerjee. 2023. "Joint Species Distribution Models with Imperfect Detection for High-Dimensional Spatial Data." *Ecology* 104: e4137. https://doi.org/10.1002/ecy.4137.

European Environment Agency. 2023. "Small Woody Features 2018 (Raster 5 m), Europe, 3-Yearly, May 2023." EEA Geospatial Data Catalogue. https://sdi.eea.europa.eu/catalogue/srv/api/records/a8e683b1-2f96-45c8-827f-580a79413018.

Funosas, D., L. Barbaro, L. Schillé, A. Elger, B. Castagneyrol, and M. Cauchoix. 2024. "Assessing the Potential of BirdNET to Infer European Bird Communities from Large-Scale Ecoacoustic Data." *Ecological Indicators* 164: 112146. https://doi.org/10.1016/j.ecolind.2024.112146.

Gelman, A., J. B. Carlin, H. S. Stern, and D. B. Rubin. 2014. *Bayesian Data Analysis*. Boca Raton, FL: Chapman & Hall/CRC.

Gelman, A., X. L. Meng, and H. Stern. 1996. "Posterior Predictive Assessment of Model Fitness Via Realized Discrepancies." *Statistica Sinica* 6: 733–760.

Gosselin, F. 2011. "A New Calibrated Bayesian Internal Goodness-of-Fit Method: Sampled Posterior p-Values as Simple and General p-Values that Allow Double Use of the Data." *PLoS One* 6: e14770. https://doi.org/10.1371/journal.pone.0014770.

Gosselin, F. 2024. "runMCMCBtadjust: Runs Monte Carlo Markov Chain – With Either ‘JAGS’, ‘nimble’ or ‘Greta’ – While Adjusting Burn-in and Thinning Parameters." R Package Version 1.1.0. https://cran.r-project.org/web/packages/runMCMCBtadjust/index.html.

Grimm, D. G., C. A. Azencott, F. Aicheler, U. Gieraths, D. G. MacArthur, K. E. Samocha, D. N. Cooper, et al. 2015. "The Evaluation of Tools Used to Predict the Impact of Missense Variants Is Hindered by Two Types of Circularity." *Human Mutation* 36: 513–523. https://doi.org/10.1002/humu.22768.

Grolemund, G., and H. Wickham. 2011. "Dates and Times Made Easy with Lubridate." *Journal of Statistical Software* 40: 1–25. https://doi.org/10.18637/jss.v040.i03.

Hadley, W. 2016. *ggplot2: Elegant Graphics for Data Analysis*. Houston, TX: Springer.

Hersbach, H., B. Bell, P. Berrisford, G. Biavati, A. Horányi, J. Muñoz Sabater, J. Nicolas, et al. 2023. "ERA5 Hourly Data on Pressure Levels from 1940 to Present. Copernicus Climate Change Service (C3S) Climate Data Store (CDS)." https://doi.org/10.24381/cds.bd0915c6.
Hijmans, R. J. 2022. “terra: Spatial Data Analysis.” R Package Version 1.5-34. https://cran.r-project.org/package=terra.

Hijmans, R. J., S. Phillips, J. Leathwick, and J. Elith. 2021. “dismo: Species Distribution Modeling.” R Package Version 1.3-5. https://CRAN.R-Project.Org/Package=dismo.

Hill, A. P., P. Prince, J. L. Snaddon, C. P. Doncaster, and A. Rogers. 2019. “AudioMoth: A Low-Cost Acoustic Device for Monitoring Biodiversity and the Environment.” *HardwareX* 6: e00073. https://doi.org/10.1016/j.ohx.2019.e00073.

Kahl, S., C. M. Wood, M. Eibl, and H. Klinck. 2021. “BirdNET: A Deep Learning Solution for Avian Diversity Monitoring.” *Ecological Informatics* 61: 101236. https://doi.org/10.1016/j.ecoinf.2021.101236.

Kéry, M., and J. A. Royle. 2016. *Applied Hierarchical Modeling in Ecology: Analysis of Distribution, Abundance and Species Richness in R and BUGS*, Vol. 1. Amsterdam: Elsevier.

Kéry, M., and J. A. Royle. 2020. *Applied Hierarchical Modeling in Ecology: Analysis of Distribution, Abundance and Species Richness in R and BUGS*, Vol. 2. Amsterdam: Elsevier.

Knight, E. C., and E. M. Bayne. 2019. “Classification Threshold and Training Data Affect the Quality and Utility of Focal Species Data Processed with Automated Audio-Recognition Software.” *Bioacoustics* 28: 539–554. https://doi.org/10.1080/09524622.2018.1503971.

Lonsinger, R. C., M. M. Dart, R. T. Larsen, and R. N. Knight. 2024. “Efficacy of Machine Learning Image Classification for Automated Occupancy-Based Monitoring.” *Remote Sensing in Ecology and Conservation* 10: 56–71. https://doi.org/10.1002/rse2.356.

Lotterhos, K. E., M. C. Fitzpatrick, and H. Blackmon. 2022. “Simulation Tests of Methods in Evolution, Ecology, and Systematics: Pitfalls, Progress, and Principles.” *Annual Review of Ecology, Evolution, and Systematics* 53: 1–24. https://doi.org/10.1146/annurev-ecolsys-102320-093722.

MacKenzie, D. I., J. D. Nichols, G. B. Lachman, S. Droege, J. A. Royle, and C. A. Langtimm. 2002. “Estimating Site Occupancy Rates When Detection Probabilities Are Less than One.” *Ecology* 83: 2248–55. https://doi.org/10.1111/j.1365-3040.1991.tb00950.x.

MacKenzie, D. I., J. D. Nichols, J. A. Royle, K. H. Pollock, L. L. Bailey, and J. E. Hines. 2018. *Occupancy Estimation and Modeling: Inferring Patterns and Dynamics of Species Occurrence*. Amsterdam: Elsevier.

Malinowski, R., S. Lewiński, M. Rybicki, E. Gromny, M. Jenerowicz, M. Krupiński, A. Nowakowski, et al. 2020. “Automated Production of a Land Cover/Use Map of Europe Based on Sentinel-2 Imagery.” *Remote Sensing* 12: 3523. https://doi.org/10.3390/rs12213523.

Miller, D. A., J. D. Nicholes, B. T. McClintock, E. H. C. Grant, L. L. Bailey, and L. A. Weir. 2011. “Improving Occupancy Estimation When Two Types of Observational Error Occur: Non-Detection and Species Misidentification.” *Ecology* 92: 1422–28. https://doi.org/10.1890/10-1396.1.

Ogawa, R. 2024. “Data from: A Classification-Occupancy Model Based on Automatically Identified Species Data.” Zenodo. https://doi.org/10.5281/zenodo.12802535.

Otis, D. L., K. P. Burnham, G. C. White, and D. R. Anderson. 1978. “Statistical Inference from Capture Data on Closed Animal Populations.” *Wildlife Monographs* 62: 3–135.

Pebesma, E. 2018. “Simple Features for R: Standardized Support for Spatial Vector Data.” *The R Journal* 10: 439–446. https://doi.org/10.32614/RJ-2018-009.

Plummer, M. 2003. “JAGS: A Program for Analysis of Bayesian Graphical Models Using Gibbs Sampling.” Proceedings of the 3rd International Workshop on Distributed Statistical Computing (DSC 2003), *Vienna* 20-22: 1–10.

R Development Core Team. 2024. *R: A Language and Environment for Statistical Computing*. Vienna: R Foundation for Statistical Computing. https://www.r-project.org/.

Rajaratnam, B., and D. Sparks. 2015. “MCMC-Based Inference in the Era of Big Data: A Fundamental Analysis of the Convergence Complexity of High-Dimensional Chains.” *arXiv*. https://doi.org/10.48550/arXiv.1508.00947.

Rhinehart, T. A., D. Turek, and J. Kitzes. 2022. “A Continuous-Score Occupancy Model that Incorporates Uncertain Machine Learning Output from Autonomous Biodiversity Surveys.” *Methods in Ecology and Evolution* 13: 1778–89. https://doi.org/10.1111/2041-210X.13905.

Royle, J. A., and W. A. Link. 2006. “Generalized Site Occupancy Models Allowing for False Positive and False Negative Errors.” *Ecology* 87: 835–841. https://doi.org/10.1890/0012-9658(2006)87[835:GSOMAF]2.0.CO;2.

Saas, Y., and F. Gosselin. 2014. “Comparison of Regression Methods for Spatially-Autocorrelated Count Data on Regularly- and Irregularly-Spaced Locations.” *Ecography* 37: 476–489. https://doi.org/10.1111/j.1600-0587.2013.00279.x.

Serrao, N. R., S. M. Reid, and C. C. Wilson. 2018. “Establishing Detection Thresholds for Environmental DNA Using Receiver Operator Characteristic (ROC) Curves.” *Conservation Genetics Resources* 10: 555–562. https://doi.org/10.1007/s12686-017-0817-y.

Shabani, F., L. Kumar, and M. Ahmadi. 2018. “Assessing Accuracy Methods of Species Distribution Models: AUC, Specificity, Sensitivity and the True Skill Statistic.” *Global Journal of Human-Social Science: B Geography, Geo-Sciences, Environmental Science & Disaster Management* 18: 1–13.

Singer, D., J. Hagge, J. Kamp, H. Hondong, and A. Schuldt. 2024. “Aggregated Time-Series Features Boost Species-Specific Differentiation of True and False Positives in Passive Acoustic Monitoring of Bird Assemblages.” *Remote Sensing in Ecology and Conservation* 10: 517–530. https://doi.org/10.1002/rse2.385.

Somervuo, P., P. Lauha, and T. Lokki. 2023. “Effects of Landscape and Distance in Automatic Audio Based Bird Species Identification.” *The Journal of the Acoustical Society of America* 154: 245–254. https://doi.org/10.1121/10.0020153.

Sørensen, T. 1948. “A Method of Establishing Groups of Equal Amplitude in Plant Sociology Based on Similarity of Species Content and Its Application to Analyses of the Vegetation on Danish Commons.” *Biologiske Skrifter* 5: 1–34.

Stephens, M. 2000. “Dealing with Label Switching in Mixture Models.” *Journal of the Royal Statistical Society, Series B: Statistical Methodology* 62: 795–809. https://doi.org/10.1111/1467-9868.00265.

Wickham, H. 2022. “stringr: Simple, Consistent Wrappers for Common String Operations.” R Package Version 1.5.0. https://CRAN.R-project.org/package=stringr.
Wickham, H., R. François, L. Henry, K. Müller, and D. Vaughan. 2023. “dplyr: A Grammar of Data Manipulation.” R Package Version 1.1.2. https://CRAN.R-project.org/package=dplyr.

Wood, C. M., and S. Kahl. 2024. “Guidelines for Appropriate Use of BirdNET Scores and Other Detector Outputs.” *Journal of Ornithology* 165: 777–782. https://doi.org/10.1007/s10336-024-02144-5.

Wood, C. M., J. Socolar, S. Kahl, M. Z. Peery, P. Chaon, K. Kelly, R. A. Koch, S. C. Sawyer, and H. Klinck. 2024. “A Scalable and Transferable Approach to Combining Emerging Conservation Technologies to Identify Biodiversity Change after Large Disturbances.” *Journal of Applied Ecology* 61: 797–808. https://doi.org/10.1111/1365-2664.14579.

Zhang, H. T., W. Y. Guo, and W. T. Wang. 2023. “The Dimensionality Reductions of Environmental Variables Have a Significant Effect on the Performance of Species Distribution

Models.” *Ecology and Evolution* 13: e10747. https://doi.org/10.1002/ece3.10747.

## SUPPORTING INFORMATION

Additional supporting information can be found online in the Supporting Information section at the end of this article.

**How to cite this article:** Ogawa, Ryo, Frédéric Gosselin, Kevin F. A. Darras, Stephanie Roilo, and Anna F. Cord. 2025. “A Classification-Occupancy Model Based on Automatically Identified Species Data.” *Ecology* 106(5): e70086. https://doi.org/10.1002/ecy.70086