J OINT I MPLICIT N EURAL
R EPRESENTATIONS
FOR
G LOBAL -S CALE S PECIES M APPING
Anonymous authors
Paper under double-blind review
A BSTRACT
Estimating the geographical range of a species from sparse observations is a challenging and important geospatial prediction problem.
Given a set of locations where a species has been observed, the goal is to learn a model that can predict whether the species is present or absent at any location.
This problem has a long history in ecology, but traditional methods struggle to take advantage of emerging large-scale crowdsourced datasets which can include tens of millions of records for hundreds of thousands of species.
We propose a new approach based on implicit neural representations that jointly estimates the geographical ranges of 47k species simultaneously.
We also introduce a series of benchmarks that measure different aspects of species range estimation and spatial representation learning.
We find that our approach scales gracefully, making better predictions as we scale up the number of species used for training and the amount of training data per species.
Despite being trained on noisy and biased crowdsourced data, our models can approximate expert-developed gold standard range maps for many species.
I NTRODUCTION
We are witnessing a dramatic decline in global biodiversity, which has severe ramifications for natural resource management, food security, and ecosystem services that are crucial to human health (Watson et al., 2019).
If we want to take effective conservation action we must understand where different species live.
In ecology, this problem is known as species distribution modeling (SDM) (Elith & Leathwick, 2009).
Ideally we would have up-to-date global SDMs for all species. Unfortunately, we only have SDMs for a relatively small number of species and locations.
A key obstacle is that most SDM methods are incompatible with the most common form of species data. An SDM must predict whether a species is present or absent at any location given spatially sparse observation records.
With a sufficient amount of presence-absence data – records of where a species has been observed to occur and where it has been confirmed to be absent – this problem can be approached using standard methods from machine learning and statistics (Beery et al., 2021). Most SDM methods require presence-absence data.
Unfortunately, presence-absence data is scarce due to the difficulty of verifying that a species is absent from an area.
Presence-only data – which consists only of locations where a species has been observed, with no confirmed absences – is much more abundant.
For instance, at the end of 2022 the iNaturalist community science platform (iNa) had collected over 70M observations across 300k species, all of which is presence-only data.
Though presence-only data is not without drawbacks (Hastie & Fithian, 2013), it is important to develop SDM methods that can take advantage of this vast supply of data.
Deep learning is currently one of our best tools for making use of large-scale datasets.
Deep neural networks also have a key advantage over many prior SDM methods because they can jointly learn the distribution of many species in the same model (Chen et al., 2017; Tang et al., 2018; Mac Aodha et al., 2019).
By learning representations that share information across species, the models can make improved predictions (Chen et al., 2017).
However, the majority of these deep learning approaches need presence-absence data for training, which prevents them from scaling beyond the small number of species and regions for which sufficient presence-absence data is available.
R ELATED W ORK
Species distribution modeling (SDM) refers to a set of methods that aim to predict where (and sometimes when and in what quantities) species of interest are likely to be found (Elith & Leathwick, 2009).
The literature on SDM is vast. Readers interested in an overview of SDM should see the classic review by Elith & Leathwick (2009) or the recent review of SDM for computer scientists by Beery et al. Note that we focus narrowly on the problem of estimating species range, i e. we do not consider more complex problems like abundance estimation (Potts & Elith, 2006).
Traditional approaches to SDM train conventional supervised learning models (e g. logistic regression (Pearce & Ferrier, 2000), random forests (Cutler et al., 2007), etc.) to learn a mapping between hand-designed sets of environmental features (e g. altitude, average rainfall, etc.) and species presence or absence (Phillips et al., 2004; Elith et al., 2006). Readers interested in traditional SDM approaches should consult Norberg et al.
; Valavi et al.
(2021; 2022) and the references therein. More recently, deep learning methods have been introduced that instead jointly represent multiple different species within the same model (Chen et al., 2017; Botella et al., 2018b; Tang et al., 2018;
Mac Aodha et al., 2019).
These models are typically trained on crowdsourced data, which can introduce additional challenges and biases that need to be accounted for during training (Fink et al., 2010; Chen & Gomes, 2019; Johnston et al., 2020; Botella et al., 2021).
We build on the work of
Mac Aodha et al.
(2019), who proposed a neural network approach that forgoes the need for environmental features (as required by e.g. Botella et al. (2018b)
; Tang et al.
by learning to predict species presence from geographical location alone.
The problem of joint SDM with presence-only data can be viewed as an instance of multi-label classification with limited supervision. In particular, it is an example of single positive multi-label learning (SPML) (Cole et al., 2021;
Verelst et al., 2022; Zhou et al., 2022).
The goal is to train a model that is capable of making multi-label predictions at test time, despite having only ever observed one positive label per training instance (i.e. no confirmed negative training labels).
Our work connects the SPML literature and species range mapping literature, and sets up large-scale joint species distribution modeling as a challenging real-world SPML task. This setting presents significant new challenges for SPML, which has previously been limited to relatively small label spaces ( < 100 categories). We investigate the role of the number of categories in our experiments.
Some SPML methods such as ROLE (Cole et al., 2021) are not computationally viable when the label space is large. One of our baselines will be the SPML method of Zhou et al.
(2022), which is scalable and attains state-of-the-art performance on standard SPML benchmarks.
Our task is related to the growing number of works that use coordinate neural networks for implicitly representing images (Tancik et al., 2020) and 3D scenes (Sitzmann et al., 2019), including those that perform novel-view image synthesis (Mildenhall et al., 2020).
There are many design choices in these methods that are being actively studied, including the impact of the activation functions in the network (Sitzmann et al., 2019; Ramasinghe & Lucey, 2022) and the effect of different input encodings (Tancik et al., 2020; Zheng et al., 2022).
In most research on implicit neural representa- tions, there is a natural choice of training objective e.g. mean squared error between the predictions and the data.
In the context of presence-only species range estimation, this choice is less clear.
We systematically investigate this question in our experiments.
Quantifying the performance of SDMs at scale is notoriously difficult due to the fact that we lack presence-absence data for most species and locations (Beery et al., 2021).
One approach is to evaluate performance on a small set of species (typically fewer than 300) from limited geographical regions where it is feasible to collect presence-absence data, as done in e.g. Potts & Elith (2006); Norberg et al. ; Valavi et al.
Two of our evaluation tasks are larger-scale versions of this idea, in which we compare the performance of our models against gold standard expert range maps for around 4,000 species.
An alternative evaluation approach is to measure the performance on a related “proxy” task.
For example, there have been a number of works that use models trained for species range estimation to assist deep image classifiers (Berg et al., 2014; Tang et al., 2015;
Mac Aodha et al., 2019; Chu et al., 2019; Mai et al., 2020; Terry et al., 2020; Skreta et al., 2020;
Yang et al., 2022).
By using images from platforms like iNaturalist, we can evaluate different range estimation methods on the task of aiding fine-grained image classification across tens of thousands of species.
Finally, we also evaluate the spatial representations learned by our models via transfer learning, using them as inputs for a suite of geospatial regression tasks. This suite of complementary benchmark tasks captures different aspects of SDM performance, and provides a starting point for large-scale SDM evaluation. See Appendix B.1 for a more detailed comparison of approaches to
SDM benchmarking.
M ETHODS 3. P RELIMINARIES
Problem statement.
Let x = [ lon, lat ] denote a geographical location (i.e. longitude and latitude).
Let y ∈ { 0 , 1 } S denote the true presence ( 1 ) or absence ( 0 ) of S different species at x .
We introduce z ∈ { 0 , 1 , ∅ } S to represent our observed data at x , where z j = 1 if species j is present, z j = 0 if species j is absent, and z j = ∅ if we do not know whether species j is present or absent.
Our goal is to develop a model that produces an estimate of y at any location x over some spatial domain
X , given observed data { ( x i , z i ) } Ni =1 .
We parameterize this model as ˆ y =
h ϕ ( f θ ( x )) , where f θ :
X → R k is a location encoder and h ϕ : R k
→ [0 , 1] S is a multi-label classifier.
The prediction ˆ y ∈ [0 , 1] S is our estimate of the presence or absence of each species at x .
Intuitively, the location encoder f
θ learns a representation of space that is used by the multi-label classifier h ϕ to predict species presence at a given location.
Input encoding.
Each species observation is associated with a point x = [ lon, lat ] .
We rescale these values so that lon, lat ∈ [ − 1 , 1] .
Following Mac Aodha et al.
we guard against boundary effects by using a sinusoidal encoding for the input.
The final result is an input vector
[Equation] [Equation]
Alternative input encodings for related coordinate networks have been explored in existing literature (Mai et al., 2020; Tancik et al., 2020; Mai et al., 2022; Zheng et al., 2022).
Their choice is orthogonal to the losses we explore, thus we leave their evaluation for future work. Implicit neural representations.
Traditionally, representation learning aims to transform complex objects (e.g. images, text) into simpler objects (e.g. low-dimensional vectors) that facilitate some downstream task (e.g. classification)
(Goodfellow et al., 2016).
Implicit neural representations offer a different perspective, in which a signal is represented by a neural network that maps the signal domain (e.g. R for audio, R 2 for images) to the signal values (Sitzmann et al., 2019; Tancik et al., 2020). In this work we learn an implicit neural representation for a large collection of crowdsourced species observation data, which can be viewed as a set of 2D point clouds (one for each species). This process yields an implicit neural representation for the geospatial distribution of each species, as well as a representation for any location on earth. Presence-absence vs. presence-only data.
Species observation datasets come in two varieties: (i)
Presence-absence data consists of locations where a species has been observed to be present and locations where it has been confirmed to be absent.
That is, we say we have presence-absence data for species j if |{ z i :
z ij = 0 }| > 0 and |{ z i :
z ij = 1 }| > 0 .
Unfortunately, presence-absence data
+ - - - - - - - - - ? ? ? ?
- targets at data location targets at random location AN-SSDL AN-SLDS AN-full random location data location - + + - - - - + unobserved ground truth + + - + + is costly to obtain at scale because confirming absence requires skilled observers to exhaustively search an area.
(ii) Presence-only data is easier to acquire and thus more abundant because absences are not collected, i e. z ij ∈ { 1 , ∅ } for i ∈ [ N ] and j ∈ [ S ] .
3. L EARNING FROM L ARGE -S CALE P RESENCE -O NLY D ATA
In the context of SPML image classification, a simple but effective approach is to assume that unobserved labels are negative (Cole et al., 2021).
This approach is based on a probabilistic argument: since natural images tend to have a small number of categories compared to the size of the label set, the vast majority of the labels will be negative.
This is also true for species distribution modeling.
Given an arbitrary location and a large set of candidate species, nearly all of them will be absent.
In this section we describe several simple and scalable loss functions based on this idea.
We illustrate three of our basic losses in Figure 2.
“Assume negative” loss (same species, different location).
This loss pairs each observation of a species with a pseudo-negative for that species at another location chosen uniformly at random:
[Equation]
where ˆ y ′
=
h ϕ ( f θ ( r )) with r ∼ Uniform( X ) and n pos =
(cid:80)
Sj =1
[ z j =1] .
This approach generates pseudo-negatives across the globe, but many of them are likely to be “easy” because they are in the ocean or far from the true species range.
“Assume negative” loss (same location, different species).
This loss pairs each observation of a species with a pseudo-negative at the same location for a different species:
[Equation]
where j ′ ∼ Uniform( { j : z j ̸ = 1 } ) .
Intuitively, this approach generates pseudo-negatives that are aligned with the spatial distribution of the observed data.
Full “assume negative” loss.
The previous two losses are inefficient in the sense that they do not use all of the entries in ˆ y .
We can combine the pseudo-negative sampling strategies of L AN − SSDL and L AN − SLDS and use all available predictions as follows:
[Equation]
where ˆ y ′
=
h ϕ ( f θ ( r )) with r ∼ Unif( X ) .
The hyperparameter λ > 0 can be used to prevent the negative labels from dominating the loss.
This is equivalent to the loss from Mac Aodha et al.
(2019) without their user modeling terms.
Maximum entropy loss.
The current state-of-the-art in SPML image classification is a technique proposed by Zhou et al.
The idea is to encourage predictions for unobserved labels to maximize entropy instead of forcing them to zero.
Zhou et al.
(2022) also includes an additional pseudo-labeling component, but we omit this because the resulting empirical improvement from it is small.
We can apply this idea to L AN − SSDL , L AN − SLDS , and L AN − full by replacing all terms of the form “ − log(1 − p ) ” with terms of the form “ H ( p ) ”
where H ( p ) =
− ( p log( p )+(1 − p ) log(1 − p )) .
We name these “maximum entropy” (ME) variants L ME − SSDL , L ME − SLDS , and L ME − full .
E XPERIMENTS
In this section we evaluate the performance of the different losses outlined previously across a range of species and environmental prediction tasks.
4. M ODELS
As described in Section 3.1, our model consists of a location encoder
f θ and a multi-label classifier h ϕ which produce a vector of predictions
ˆ y =
h ϕ ( f θ ( x )) for a location x .
The location encoder f
θ is implemented as the fully connected neural network shown in Figure A4.
We implement the multilabel classifier h ϕ as a single linear layer with sigmoid activations.
Full implementation details can be found in Appendix C.
There are two other model types represented in Table 1. The rows tagged with “LR” are logistic regression baselines (Pearce & Ferrier, 2000), in which the location encoder f
θ is replaced with the identity function and h ϕ is unchanged.
The “Best Discretized Grid” row does not use a location encoder, but instead makes predictions based on binning the training data (Berg et al., 2014).
Full details for these models can be found in Appendix C. These baselines allow us to quantify the importance of our deep location encoder.
4. T RAINING D ATA
We train our models on presence-only species observation data obtained from the community science platform iNaturalist (iNa).
The training set consists of 35.5 million observations covering 47, species observed prior to 2022.
For each species, we have the geographical coordinate where the species was observed.
To be included in the training set, each species needed at least 50 observations.
Some species are more common than others, and thus the dataset is heavily imbalanced – see
Figure A6. Later we use this data in its entirety (“All”), with different maximum observations per class (“X / Class”), or with different subsets of classes.
See Appendix D for more details on the dataset.
4. E VALUATION T ASKS AND M ETRICS
We propose four tasks for evaluating large-scale species distribution models.
We give brief descriptions here, and provide further details in Appendix E.
S&T: eBird Status and Trends.
This task quantifies the agreement between our presence-only maps and high-quality expert range maps from the eBird Status & Trends dataset (Fink et al., 2020), covering 536 bird species.
Performance is measured using mean average precision (MAP), i.e. computing the per-species average precision (AP) and averaging the results across species.
IUCN:
Range Maps.
This task compares our maps against expert range maps from the International Union for Conservation of Nature (IUCN) Red List (IUC). Unlike the bird-focused S&T task, this task covers 3,938 species from diverse taxonomic groups.
Performance is measured using MAP.
Geo Prior: Geographical Priors for Computer Vision.
This task measures the utility of our range maps as priors for fine-grained image classification (Berg et al., 2014; Mac Aodha et al., 2019).
For this task we collect 282,974 geo-located images from iNaturalist, covering 39,444 species from our training set.
The performance metric for this task (“ ∆ Top-1”) is the change in image classifier topaccuracy when using our range maps as a geographical prior. This occurs at test time – the image classifier is never trained with geographical information. A positive value indicates that the prior improves the performance of the classifier. Unlike S&T and IUCN , this is an indirect evaluation of range map quality since we are assessing only how useful the range maps are for a downstream task.
Geo Feature: Environmental Representation Learning.
Instead of evaluating the range estimates of our model, this transfer learning task evaluates the quality of the underlying geospatial representation that has been learned. The goal is to predict nine different geospatial characteristics of the environment e.g. above-ground carbon, elevation, etc.
First, we use the location encoder f θ to extract features for a grid of evenly spaced locations.
After splitting the locations into train and test sets, we fit a linear regressor to predict the geospatial characteristics from the extracted features.
Performance is evaluated using the coefficient of determination R 2 on the test set.
4. R ESULTS
Deep location encoders are crucial.
Our method outperforms logistic regression trained with the same inputs and loss by more than 50 MAP on the S&T task – compare “ L AN − full ” and “ L AN − full (LR, Coord. Inputs)” with 1000 examples per class in Table 1.
The only difference is the fact that our method uses a deep location encoder. See Appendix A.2 for more details.
Environmental
features are unnecessary for good performance.
Our method gains only 1.1 MAP on the S&T task when we switch from coordinate inputs (“ L AN − full ”) to environmental feature inputs (“ L AN − full (Env. Inputs)”).
The gain is much more significant for logistic regression, i.e.
+15.7 MAP for “ L AN − full (LR, Env. Inputs)”
vs. “ L AN − full (LR, Coord.
Inputs)”.
It seems that our approach can successfully use sparse presence-only data to learn about the environment, so that using environmental features as input provides a marginal benefit.
See Appendix A.1 for more.
Low-shot performance is surprisingly good.
In Table 1 we can see that L AN − full (using onlyexamples per category, i.e. 1.3% of the training data) beats the “Best Discretized Grid” baseline (which uses all of the training data) on every task.
Our models seem to be capable of capturing general spatial patterns using relatively little data.
While this is encouraging, we expect that more data is necessary to capture fine detail as suggested by Figure 5.
Pseudo-negatives that follow the data distribution are usually better.
L AN − SSDL and L AN − SLDS differ only in the fact that L AN − SSDL samples pseudo-negatives from random locations while
L AN − SLDS samples pseudo-negatives from data locations (see Figure 2).
We can see in Tablethat L AN − SLDS outperforms L AN − SSDL for all tasks except IUCN .
This could be due to the fact that some IUCN species have ranges far from areas that are well-sampled by iNaturalist.
As we can see in Figure A3 (Black Oystercatcher), L AN − SSDL can behave poorly in areas with little training data. This highlights the importance of using diverse tasks to study range estimation methods.
Joint learning across categories is beneficial, but more data is better.
In Figure 3 we study the effect of the amount of training data on performance for the S&T task.
We first note that increasing the number of training examples per species reliably and significantly improves performance.
One possible mechanism for this is suggested by Figure 5, which shows a more spatially detailed representation emerging with more training data.
Figure 3 also shows that adding additional species to the training set (which are not in the test set) also tends to increase performance, though by a smaller margin.
In the right panel of Figure 3 we can see that it is better to have more training data for fewer species than less training data spread across more species.
Table 1 shows a clear trend towards higher performance with more training data across tasks.
The exception is that both L AN − SLDS and L AN − full have worse performance on the Geo Prior task when using all data compared to using 1000 / class.
This may be due to the extreme imbalance in the full training set, as show in Figure A6.
Method rankings may not generalize across domains.
The presence-only SDM problem in this work and the SPML image classification problem in Cole et al. (2021) are both SPML problems.
Despite this formal equivalence, it does not seem to be the case that the best current method for SPML image classification is also the best methods for presence-only SDM.
Zhou et al.
(2022) show that their “maximum entropy” loss performs much better than the “assume negative” loss across a number of image classification datasets.
However, all of the “maximum entropy” losses in
Table 1 ( L
[Equation]
[Equation]
, L ME − full ) perform worse than their “assume negative” counterparts ( L AN − SSDL , L AN − SLDS , L AN − full ) in almost all cases.
L ME − full is a top performer on the Geo Feature task, but Figure 4 shows that no method seems to have a commanding advantage.
4. L IMITATIONS
There are some limitations associated with our analysis.
As noted, the training set is heavily imbalanced, both in terms of the species themselves and where the data was collected.
This is partially because some species are more common and thus more likely to be observed than others.
We do not explicitly deal with species imbalance in the training data, other than by showing that the ordering of methods does not significantly vary even when the training data is capped to the same upper limit per species, i.e. 10, 100, or 1,000 examples per class in Table 1. [Figure] locations
All models improve their range maps when given access to more data, as expected. L AN − SSDL over estimates the range with few examples, and refines the range with additional data, while L AN − full starts off conservative with few training examples and expands as more data is available. The range predicted by L AN − SLDS is somewhere in-between.
The magnitude of the score values is also interesting, with both L AN − SSDL and L AN − SLDS predicting more locations confidently than L AN − full . [Table] Reliably evaluating the performance of models for species range estimation is a long standing and challenging question.
To address this issue
[Table]
we present a suite of complementary benchmarks that attempt to evaluate different facets of this spatial prediction problem.
However, obtaining ground truth distribution data for tens of thousands of species remains a very difficult problem.
While we believe our benchmarks to be a significant step forward, they are likely to have blind spots.
Finally
[Table]
extreme care should be taken before making conservation decisions based on the outputs of models such as the ones presented here.
Our goal in this work is to demonstrate the promise of large-scale representation learning for species distribution modeling.
Our models have not been calibrated or validated beyond the experiments illustrated above [Table]
C ONCLUSION
We explored the problem of species range estimation through the lens of representation learning.
Across a range of spatial prediction tasks, we showed that adding more and diverse data improves model performance.
In doing so, we also made connections to recent literature on training implicit coordinate networks and learning multi-label classifiers from limited supervision.
We hope that our work will encourage more machine learning researchers to work on this important problem.
While the initial results are encouraging, there are still many avenues for future work.
For example, currently our models do not make use of any temporal input information (Mac Aodha et al., 2019), we do not account for the spatial biases present in the data (Chen & Gomes, 2019), and our neural network models could be adapted to have better inductive biases for encoding spatially varying signals (Ramasinghe & Lucey, 2022). R Appendix A A DDITIONAL R ESULTS A. H OW DOES OUR APPROACH COMPARE TO A TRADITIONAL SDM?
Our approach can handle large-scale SDM in a way that most traditional SDMs cannot.
However, for context we compare our results to a classic SDM.
One of the most scalable SDM approaches in the ecology literature is to use logistic regression with environmental features as input (Pearce &
Ferrier, 2000). Crucially, we can train this method using GPU-accelerated batch-based optimization. Full implementation details are provided in Appendix C.3.1. In Figure A1 we compare our approach against logistic regression.
We try both approaches with environmental feature inputs and with coordinate inputs.
First we note that our method has a decisive advantage over logistic regression in all cases.
Second, we observe that using environmental features as input (instead of coordinates) improves the performance of logistic regression significantly (+15.
MAP) and improves the performance of our method slightly (+1.1 MAP).
These facts are consistent with the claim that our method is able to learn a rich representation of the environment from species co-occurrence patterns alone, without the need for environmental covariates.
A. W HAT IS THE EFFECT OF THE ARCHITECTURE OF THE LOCATION ENCODER ?
Location encoder depth.
As discussed in Appendix C, the location encoder f θ used throughout the paper has one input layer followed by four residual layers.
We now consider the importance of the number of layers in the encoder.
Figure A2 shows a large performance increase when we go from the logistic regression model (0 hidden layers) to the next largest model (4 hidden layers).
Performance is fairly stable beyond that point, marginally increasing (8 hidden layers) and then dropping ( hidden layers).
We suggest that the performance drop for the deepest models may be due to the fact that, with enough capacity, there is no need to share information across species.
Residual modules vs. traditional layers.
The residual network used by us was chosen to allow us to directly compare to existing related work that uses the same architecture (Mac Aodha et al., 2019).
Figure A2 compares this architecture against a simpler multi-layer perceptron (MLP) architecture.
Here the traditional MLP is comparable, and in fact performs slightly better.
Though there is not much difference in performance, future work should consider using a traditional MLP for simplicity.
A. A DDITIONAL Q UALITATIVE R ESULTS
To build some intuition for the behavior of L AN − SSDL , L AN − SLDS , and L AN − full , we compare these losses on three species that are known to have interesting ranges in Figure A3.
B A DDITIONAL D ISCUSSION B. H OW DO THE BENCHMARK TASKS PROPOSED IN THIS PAPER COMPARE TO EXISTING SDM BENCHMARKS ?
Presence-only SDM is notoriously tricky to evaluate (Beery et al., 2021), and there are few public benchmark datasets available for the task.
Here we will discuss the two most relevant lines of prior work that have approached this evaluation problem (one from the ecology community and one from the machine learning community), and discuss where our benchmark is similar and different.
To the best of our knowledge, Elith et al.
(2006) was the first attempt to systematically compare presence-only SDM algorithms across many species and locations.
That work compared 16 SDM algorithms on a collection of taxonomically-specific datasets from 6 different regions, covering a total of 226 species.
Presence-only data was used for training, presence-absence data was used for evaluation.
Unfortunately the data was not made publicly available until Elith et al.
There are two main issues with this benchmark.
First, the benchmark is not suitable for studying largescale joint SDM.
It has a small number of species overall, and there are at most 54 covered in any region.
Second, the species in the dataset are anonymized.
This makes it impossible to use their dataset to study large-scale SDM, because we cannot increase the size of their training with external data nor can we evaluate our trained models on their test data.
Another line of work comes from the GeoLifeCLEF series of datasets and competitions (Botella et al., 2018a; 2019; Deneu et al., 2020; Cole et al., 2020; Lorieul et al., 2021; 2022).
These benchmarks represent an attempt to scale up presence-only SDM, with the 2022 dataset covering 17k species with 1.6M species observations the U.S. and France.
As in our benchmark, all of their training data is drawn from community science projects.
The primary limitation of the GeoLifeCLEF benchmarks is that they use biased presence-only data at test time, evaluating the problem as an information retrieval task instead of a spatial prediction task.
L
AN −
SS D L
L
AN −
S L D
S
L
AN − f u ll
Barn Swallow © C a r t
o © O pen
S t r ee t M ap c on t r i bu t o r s © C a r t
o © O pen
S t r ee t M ap c on t r i bu t o r s © C a r t
o © O pen
S t r ee t M ap c on t r i bu t
o r s
-Thrush © C a r t o ©
O pen
S t r ee t M ap c on t r i bu t o r s © C a r t
o © O pen
S t r ee t M ap c on t r i bu t o r s © C a r t
o © O pen
S t r ee t M ap c on t r i bu t
o r s
© C a r t
o ©
O pen S t r ee t M ap c on t r i bu t o r s © C a r t
o ©
O pen S t r ee t M ap c on t r i bu t o r s © C a r t
o ©
O pen S t r ee t M ap c on t r i bu t
o r s
P r e s en t
{+A b s en t 0 .0 .0 .
+}
M o d e l S c o r e
F i gu r e
A 3 :
L o ss f un c ti on c o m p a r i s on ( c o l u m n s )
f
o r t
h r ee d i ff e r e n t s p ec i e s o f b
i r d s (r o w s ) .
A ll m od e l s w e r e t r a i n e d w it
h
e x a m p l e
s p e r c l a ss .
S ee F
i gu r e
6 f
o r a n e xp l a n a ti on o f t
h e p l o t s .
T h
e s e s p ec i e s w e r e c ho s e n f o r v
i s u a li za ti on b eca u s e t h e
i r r a ng e s h a v e i n t e r e s ti ng c o m p l e m e n t a r y p r op e r ti e s .
( T op
R o w ) B a r n S w a ll o w
i s a s p ec i e s t h a t
o cc u r s ac r
o
ss t
h
e
g l ob e .
M i dd l e R o w )
B l u e
R o c k - T h r u s h
i s a s p
ec
i e s w ho s e r a ng e go e s fr o m t
h e d a t a r i c h
W e s t e r n P a l e a r c ti c
b i og e og r
aph
i c r e a l m , t h r ough a d a t a s p a r s e a r ea , a nd b ac k t
o a m o r e
d a t a r i c h a r ea o f
ea s t e r n
A
s i a .
( B o tt o m R o w )
B l ac k O y s t
e r ca t c
h e r
i s a s p
ec
i e s w ho s e r a ng e hug s t
h e w e s t
c
o a s t o f t h e
U n it e d
S t a t e s .
U n li k e i n t h e r e s t
o f t
h e p a p e r , t h e
s e v
i s u a li za ti on s f o r B l ac k
O y s t
e r ca t c h e r do no t u s e a m a s k t
o fi lt e r ou t p
r
e d i c ti on s fr o m non - l a nd r e g i on s .
H e r e , w e s p ec i fi ca ll y w a n t e d t
o s ee
i f t h
e m od e l s l ea r n e d t o f o ll o w t
h e c o a s tli n e .
W e ob s e r v e t h a t L
AN − S
L D S i n c o rr ec tl y e xp a nd s t
h e r a ng e i n t o t h e
P ac i fi c .
Our benchmark can be viewed as a significant expansion of the GeoLifeCLEF line of work.
Instead of being geographically limited to France and the U.S., we allow data from anywhere in the world.
Instead of evaluating with biased presence-only data, we use presence-absence data like Elith et al.
However, unlike Elith et al.
(2020), we work at a large scale that allows us to study data scaling in SDM.
Our indirect evaluation tasks ( Geo Prior and Geo Feature ) add complementary dimensions to presence-absence evaluation, and have no counterpart in Cole et al.
(2020) or Elith et al.
B. E NVIRONMENTAL C OVARIATES VS . C OORDINATES
One important characteristic of any SDM is whether or not it is spatially explicit .
Spatially explicit
SDMs include geospatial coordinates as part of the model input (Domisch et al., 2019).
Traditional covariate-based SDMs include only environmental features (e.g. altitude, distance to roads, average temperature, etc.)
in the input (Elith & Leathwick, 2009).
Covariate-based SDMs are often understood to reflect habitat suitability , because they learn a relationship between environmental characteristics and observed species occurrence patterns.
A covariate-based SDM will make the same predictions for all locations with same covariates, even if those locations are on different continents.
Furthermore, covariates sets must be selected by hand and are often limited in their spatial resolution and coverage.
By contrast, spatially explicit SDMs can model the fact that a species may be present in one location and absent in another, even if those two locations have similar characteristics.
However, spatially explicit models are unlikely to generalize to locations that are spatially distant from the training data – such locations are simply out of distribution.
Our proposed approach is spatially explicit, so our goal is not to learn from data in one location and extrapolate to distant locations.
Instead, our goal is to use abundant (but noisy and biased) species observation data to approximate high-quality expert range maps.
Our locations of interest are the same during training and testing.
The difference is the training data source and quality.
See Merow et al.
for a more nuanced discussion of extrapolation vs. interpolation and the role of model complexity in SDM.
B. T HE R OLE OF T IME
Some species are immobile (e.g. trees), while others (e.g. birds) may occupy different areas at different times of the year.
For this reason, there has long been interest in the temporal dynamics of species distributions (Collins & Glenn, 1991; Guisan & Rahbek, 2011).
However, traditional SDMs use environmental features as input, which seldom include temporal structure (Elith et al., 2020;
Norberg et al., 2019).
For instance, the famous WorldClim bioclimatic variables used in many SDM papers are non-temporal (Hijmans et al., 2005).
It is therefore not unusual for papers on SDM to make no explicit considerations for temporal information.
Similarly, in this work we do not use temporal information during training or evaluation.
However, we consider this to be an interesting area for future work.
C I MPLEMENTATION D ETAILS C. N ETWORK A RCHITECTURE
Unless otherwise specified, we use the network in Figure A4 for our location encoder f θ .
This is identical to the architecture in Mac Aodha et al.
(2019), and similar architectures have been used for other tasks (Martinez et al., 2017).
The right side of the figure shows the network structure, consisting of one standard linear layer and four residual layers.
The left side of the figure shows the structure of a single residual layer.
Note that all layers have the same number of nodes.
Every layer of the network has 256 nodes and we use p = 0 .
5 for the dropout probability.
C. T RAINING D ETAILS
Environment.
All models were trained on an Amazon AWS p3.2xlarge instance with a Tesla
V100 GPU and 60 GB RAM.
The model training code was written in PyTorch (v1.7.0).
Hyperparameters.
All models were trained for 10 epochs using a batch size of 2048.
We used the
Adam optimizer with an exponential learning rate decay schedule of learning rate = initial learning rate × epoch 0 .where epoch ∈ { 0 , 1 , . .
For each model we tuned the learning rate over { 1 e − 2 , 1 e − 3 , 1 e − 4 } and selected the model with the best validation performance on the “S&T” task.
For L AN − full and
L GP we set λ = 2048 . [Figure] [Figure]
[Figure]
B ASELINES C.3. L OGISTIC R EGRESSION WITH E NVIRONMENTAL C OVARIATES
This section discusses our implementation of logistic regression with environmental covariates, whose performance is discussed in Appendix A.1. The architecture for this approach is equivalent to using our method but replacing the location encoder f θ with the identity function. Then we can in principle use any of our loss functions for training.
All other training details follow Appendix C.2. This approach uses environmental covariates as input, not raw coordinates. This means that environmental covariates are required for both training and evaluation.
We use the elevation and bioclimatic rasters from WorldClim 2.1 (Fick & Hijmans, 2017) at the 5 arc-minute spatial resolution.
We normalize each covariate independently by subtracting the mean and dividing by standard deviation (ignoring NaN values). We the replace NaN values with zeros. Note that we use these same environmental covariates as the inputs to our model in Appendix A.1.
C.3. D ISCRETIZED G RID
Our baseline system for the S&T , IUCN , and Geo Prior tasks is a simple spatial binning method. Once we choose a resolution level, the H3 geospatial indexing library (H3W) defines a collection of W hexagons { H 1 , . . , H W } that partition the globe.
For instance, W = 2,016,830 at resolution level 5.
We show discretized grid results for a few different resolution choices in Table A1.
Below we describe the discretized grid baseline in more detail.
For the S&T and IUCN tasks, we can compute a score for any hexagon and species as follows:
[Table]
We compute the number of occurrences of species j in hex w as
[Equation] for 1 ≤ w ≤ W and 1 ≤ j ≤ S . 2. Let H t be a hexagon we wish to evaluate at test time. For any species 1 ≤ j ≤ S , we compute a prediction for H t as
[Equation]
That is, ˆ y j measures how often species j was observed in H t (relative to how often species j occurred in the location where it was observed most often).
These predictions always fall between 0 and 1, which ensures that they are compatible with the average precision metrics we use for S&T and IUCN evaluation.
For the Geo Prior task, the first step is the same but the second step is different:
We the number of occurrences of species j in hex w as
[Equation]
for 1 ≤ w ≤ W and 1 ≤ j ≤ S . 2.
Let H t be a hexagon we wish to evaluate at test time.
For any species 1 ≤ j ≤ S , we compute a prediction for H t as
[Equation]
That is, any species which were not observed in H t are “ruled out” for the downstream image classification problem.
D T RAINING D ATASET D. D ATASET C ONSTRUCTION
Our training data was collected by the users of the community science platform iNaturalist (iNa).
iNaturalist users take photographs of plants and animals, which they then upload to the platform. Other users review these images and attempt to identify the species. The final species labels are decided by the consensus of the community.
Each species observation consists of an image and associated metadata indicating when, where, and by whom the observation was made.
iNaturalist data only contains presence observations, i.e. we do not have access to any confirmed absences in our training data.
Specifically, our training data was sourced from the iNaturalist AWS Open Dataset 2 in May 2022.
We began by filtering {+the species observations according to the following rules: N u m . t r a i n i n g o b s e r v a t i o n s 
 Observations must have valid valid longitude and latitude data.+}
Observations must be identified to the species level by the iNaturalist community. Observations which can only be identified to coarser levels of specificity are discarded. Observations must have research grade status, which indicates that there is a consensus from the iNaturalist community regarding their taxonomic identity. After this filtering process, species with fewer than 50 observations were removed from the dataset.
We also remove any species which are marked as inactive 3 . Finally, we only included observations made prior to 2022. This will enable a temporal split from 2022 onward to be used as a test set in the future.
After filtering, we were left with 35,500,262 valid observations from 47,375 distinct species.
A visualization of the geographical distribution of the resulting data can be seen in Figure A5.
As Figure A6 shows, our training data is heavily imbalanced, reflecting the natural frequency with which they are reported to iNaturalist (Van Horn et al., 2018).
D. C HANGING THE N UMBER OF T RAINING E XAMPLES PER C ATEGORY
In the main paper we consider the impact of the number of observations per species in the training set by training on different sub-sampled datasets.
We construct these datasets by choosing k obser- vations per species, uniformly at random.
We set a seed to ensure that we are always using the same k observations per category.
We also make certain that sampled datasets are nested, so the dataset with k 1 examples per category is a superset of the dataset with k 2 < k 1 examples per category.
If a category has fewer than k observations, we use them all.
D. C HANGING THE N UMBER OF T RAINING C ATEGORIES
In the main paper we also consider the impact of the number of species in the training set.
In particular, we consider the following nested subsets:
• The set of 536 bird species in the eBird Status & Trends dataset (Fink et al., 2020).
• The eBird Status & Trends species plus an additional A randomly selected species, where A ∈ { 4000 , 8000 , 24000 } .
E E VALUATION T ASKS
Here we provide additional details on the benchmark tasks used in the main paper.
For each task, we outline the dataset properties, how it was collected, and the evaluation metrics used.
E. S&T: E B IRD S TATS AND T RENDS
Task:
The goal of this task is to evaluate how effective models trained on noisy crowdsourced data from iNaturalist are at predicting species range maps.
We use the eBird Status & Trends data from Fink et al.
(2020) to evaluate our range predictions.
This dataset consists of estimated range maps for 536 species of birds predominately found in North America, but also other regions.
The range maps are the output of an expert crafted model (Fink et al., 2020) that has been trained on tens of millions of presence and absence observations, makes use of additional expert knowledge to perform data filtering, and uses rich environmental covariates as input.
While not without its own limitations, we treat this data as ‘ground truth’ for evaluation purposes as it is developed using much higher quality data compared to what we use to train our models.
Dataset: We perform several pre-processing steps on the raw data.
We first spatially bin the data using hexagons at resolution five as defined by the H3 project 4 . 2 , 016 , 842 hexagons cover the world at this resolution, each with an average area of 252 .
9 km 2 .
Our goal is to predict the presence or absence of a given species in each hexagon using the eBird Status & Trends output as ‘psuedo’ ground truth.
The evaluation regions are restricted to those where the eBird Status & Trends models have determined that there is sufficient data to make a prediction for a given species.
Thus, the set of evaluation regions can vary from species to species.
For example, Melozone aberti has 23,282 locations with known presence or absence, of which 271 are deemed present.
On the other hand, Columba livia has 67,251 locations with known presence or absence, of which 39, are deemed present.
The eBird Status & Trends data provides species presence and absence information for each location over the course of the year.
For the purposes of our evaluation, we collapse all time points to one, and count a hexagon region as being a presence for a given bird if the output of their model is greater than zero for any week in the year for that species.
We create a validation split by choosing 50% of the hexes with known presence/absence status uniformly at random for each species.
This is in line with the notion of generalization we care about.
That is, we want to know how well we can reproduce expert range maps from raw, potentially noisy, data, not how well we can make predictions in previously unseen locations.
Evaluation: We use mean average precision (MAP) as our evaluation metric, only evaluating on valid regions for a given species.
[Figure]
E. ICUN: R ANGE M APS
Task:
The goal of this task is similar to the previous one, i.e. to predict the geographical range of a set of species.
However, instead of ‘ground truth’ target maps that are estimated by another model, here we use expert curated range maps from the International Union for Conservation of Nature (IUCN) 5 .
This set of data contains a more taxonomically diverse set of species compared to the
Stats and Trends task, as it contains mammal and fish species, among others, in addition to birds.
Dataset: We select a subset of 3,938 species that overlap with our training species.
For each species, we take the expert generated range polygon(s), which indicates the geographical range of a given species, and sample a set of points inside the polygons(s) and denote them as presence observations.
In addition, we sample another set of points, uniformly at random outside those polygons(s) and denote them as absence observations.
In total, we sample 2,000 observations per species, half presence and half absence.
A visualization of the resulting sampling process is shown in Figure A7.
Note that the absolute performance on this dataset in Table 1 is often well above 90%.
We believe this is due to the fact that the absence samples are “easy” because many of them are far away from the presence samples.
Alternative sampling processes could have been used, e.g. with more emphasis on the boundary regions between presence and absence areas as denoted by the expert maps.
However, as the expert maps cannot necessarily be assumed to be the objective ground truth (i.e. species ranges can shift over time), we instead opted for the strategy outlined above.
Evaluation:
Again we use mean average precision (MAP) as the evaluation metric, which results in a single score for a model averaged across all species.
E.
[Equation]
C OMPUTER V ISION
Task:
The goal of this task is to combine the outputs of the models trained for species range estimation on the iNaturalist dataset with computer vision image classifier predictions.
This evaluation protocol has also been explored in other work, e.g. Berg et al.
Mac Aodha et al.
We simply weight the probabilistic image classifier predictions for a given image with the species occupancy predictions from the location where that image was taken.
The intuition is that the occupancy prediction down-weights the probability of a given species being predicted by the vision model if the range estimation model predicts that the species is not likely to be present at that location.
Dataset: For the vision classifier, we use an image classification model developed by the iNaturalist community science platform 6 .
This model is an Xception network (Chollet, 2017) that has been trained on 55,000 different taxonomic entities (i.e. classes) from over 27 million images.
We take the predictions from the final classification layer of the classifier, and do not apply any of their sophisticated taxonomic post-processing.
There are a total of 49,333 species in the set of 55, classes, the rest are higher order groups, e.g. genera.
The images used to train the image classifier come from observations that were added to iNaturalist prior to December 2021.
We then constructed a test set consisting of all research grade observations (i.e. those observations for which there is a consensus from the iNaturalist community as to which species is present in the image).
The images in the test set only contain the set of species that were observed at training time, i.e. we do not consider the open-set prediction problem.
The observations were selected from between January and May 2022 to ensure that they did not overlap with the training set.
We take at most ten observations per species, which results in 282,974 total observations from 39,444 species.
In practice, many species do not have 10 observations.
In total there are 2,721 species, with 9, total images, that are not present in our range estimation training set.
For each of the 282, observations, we extract the predictions from the deep image classifier across all 39,444 remaining species.
Evaluation: Performance is evaluated in terms of top-1 accuracy, where the ground truth species label is provided by the iNaturalist community.
Without using any information about where an image was taken, the computer vision model alone obtains an accuracy of 75.4%, which increases to 90.4% for top-5 accuracy.
During evaluation, if a species is not present in a range model, we simply set the output for the range model for that species to 1 . 0 .
E. G EO F EATURE : E NVIRONMENTAL R EPRESENTATION L EARNING
Task:
This task aims to evaluate how well features extracted from deep models trained to perform species range estimation can generalize to other dense spatial prediction tasks.
Unlike the other benchmark tasks that use the species occupancy outputs directly, this is a transfer learning task. We remove the classification head h ϕ and evaluate the trained location encoder f θ in terms of downstream environmental prediction tasks.
The intuition is that a model that is effective at range estimation may have learned a good representation of the local environment.
If so, that representation should be transferable to other environmental tasks with minimal adaptation.
This task is inspired by the linear evaluation protocol that is commonly used in self-supervised learning, e.g. Chen et al.
In that setting, the features of the backbone model are frozen and a linear classifier is trained on them to evaluate how effective they are on various downstream classification tasks.
In our case, instead of classification, we aim to regress various continuous environmental properties from the learned feature representations of our range estimation models.
A related evaluation protocol was recently used in Rolf et al.
(2021) for the case of evaluating models trained on remote sensing data.
Dataset: The task contains nine different environmental data layers which have been collected using
Google Earth Engine 7 .
The nine data layers are described in Table A2.
For each of the layers, we have rasterized the data so that the entire globe is represented as a 2004 × 4008 pixel image.
Each pixel represents the measured value for a given layer for the geographical region encompassed by the pixel.
Example images can be seen in Figure A8.
Evaluation: For evaluation, we crop the region of interest to the contiguous United States and grid it into training and test cells.
The spatial resolution of the training and testing cells are illustrated in Figure {+A
+} (right).
Note, we simply ignore locations that are not in the training or test sets, e.g. the ocean.
This results in 51,422 training points and 50,140 test points.
Features are then extracted from the location encoder for the spatial coordinates specified in the training split, and then a linear ridge regressor is trained on the train pixels and evaluated on the held out test pixels.
The input features are normalized to the range [0 , 1] . We cross validate the regularization weighting term α of the regressor on the training set, exploring the set α ∈ { 0 . 0001 , 0 . 001 , 0 . 01 , 0 . 1 , 1 . 0 , 10 . 0 } .
Performance is reported as the coefficient of determination R 2 on the test pixels, averaged across all nine layers.
F R EPRODUCIBILITY S TATEMENT
The information needed to implement and train the models outlined in this paper are provided in
Section C.
In addition, the different training losses evaluated are described in Section 3.2.
Upon acceptance, we will provide code to enable researchers to build on our results.
[Figure]
ELE: Elevation
Train / Test Split IgnoreTrainTest
{+M g C / h a 04080120 m e t e r s 
+}
[Table]
G E THICS
This work makes use of species observation data provided by the iNaturalist community.
We explicitly only use data from the public data exports from iNaturalist to ensure that no data, that is not already available publicly, related to sensitive species (e [Table] g. those who are at risk of extinction) is used by our models.
As noted in the limitations section in the main paper, extreme care needs to be taken when attempting to interpret any species range predictions from the models presented in this paper.
The models have not been evaluated on those types of downstream tasks.