Active Learning-Based Species Range Estimation
Authors: Christian Lange
Elijah Cole
Grant Van Horn
Oisin Mac Aodha
Venue: NeurIPS 2023 poster (2023)
Keywords: species range estimation
active learning
implicit networks
TL;DR: We propose a new active learning approach for efficiently estimating the geographic range of a species from a limited number of on the ground observations.

Abstract: We propose a new active learning approach for efficiently estimating the geographic range of a species from a limited number of on the ground observations. We model the range of an unmapped species of interest as the weighted combination of estimated ranges obtained from a set of different species. We show that it is possible to generate this candidate set of ranges by using models that have been trained on large weakly supervised community collected observation data. From this, we develop a new active querying approach that sequentially selects geographic locations to visit that best reduce our uncertainty over an unmapped species’ range. We conduct a detailed evaluation of our approach and compare it to existing active learning methods using an evaluation dataset containing expert-derived ranges for one thousand species. Our results demonstrate that our method outperforms alternative active learning methods and approaches the performance of end-to-end trained models, even when only using a fraction of the data. This highlights the utility of active learning via transfer learned spatial representations for species range estimation. It also emphasizes the value of leveraging emerging large-scale crowdsourced datasets, not only for modeling a species' range, but also for actively discovering them.

Active Learning-Based Species Range Estimation
Christian Lange'
Elijah Cole
Grant Van Horn'
Oisin Mac Aodha'
University of Edinburgh
Altos Labs
Caltech
UMass Amherst
Abstract
We propose new active learning approach for efficiently estimating the geographic range of a species from a limited number of on the ground observations. We model the range of an unmapped species of interest the weighted combination of estimated ranges obtained from a set of different species. We show that it is possible to generate this candidate set of ranges by using models that have been trained on large weakly supervised community collected observation data. From this, we develop new active querying approach that sequentially selects geographic locations to visit that best reduce our uncertainty over an unmapped species' range. We conduct detailed evaluation of our approach and compare existing active learning methods using an evaluation dataset containing expert-derived ranges for one thousand species. Our results demonstrate that our method outperforms alternative active learning methods and approaches the performance of end-to-end trained models, even when only using fraction the data. This highlights the utility of active learning via transfer learned spatial representations for species range estimation. also emphasizes the value leveraging emerging large-scale crowdsourced datasets, not only for modeling species' range, but also for actively discovering them.
Introduction
Understanding the geographic range that biological species occupies is fundamental piece of information that drives models for ascertaining how vulnerable a species to extinction threats [2), for quantifying how they respond to climate induced habitat change [29], in addition to being important for assessing biodiversity loss. Estimated range maps are typically generated from statistical or
machine learning-brit based models that parameterized from sparsely collected in situ observations (7].
Such in situ observations can be obtained through various means, e.g., via experts conducting detailed field surveys to record the presence absence of a particular species in defined geographic area (27. from community scientists that record incidental observations in a less structured manner [44], or from autonomous monitoring solutions such as camera traps [46). One of the major limiting factors in scaling up the generation of reliable range maps to hundreds thousands of species is the underlying collection of data.
Recently, community science-based platforms such as iNaturalist 4. eBird 44 and PlantNet K have proven to be appealing and scalable way to collect species observation data by distributing the effort across potentially millions participants around the world. However, due to the incidental nature of the observations collected these platforms (i.e., users are not directed towards specific geographic location and asked to survey exhaustively for the presence of species of interest), there be strong biases with respect the places people go and the types of species they report (1T). Additionally, a conflict exists in that the ranges of rare species are the hardest to characterize owing the lack of data, but estimating their ranges would actually provide the most useful data for conservation purposes [30]. In this work, we explore an alternative collection paradigm for efficiently estimating geographic ranges of previously unmapped species inspired by ideas from
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Hypothesis Space
Spatial Active Learning
time
Bime
time
Observation
Estimated
Species present Species absent
Species Range
Figure Our goal is to estimate the geographic range of species (i.e., the locations where the species can be found during its lifetime) from small number of actively selected in situ observations. (Left) We assume we have access to a hypothesis set of candidate species range estimation models, where each member of the set encodes the range for the species it was trained on. Intuitively, models that are close this space encode similar spatial ranges (e.g., hú and h2). (Right) At each time step in our active learning- based species range estimation paradigm, we sample geographic location, human then goes there to determine if the species is present or not, and then we update our predicted
range based on the new data. The hypothesis space the left is used to guide the selection of locations via our active querying approach.
active learning [41] Our objective is to minimize the number of geographic locations that need to be sampled accurately estimate a species' geographic range. To achieve this, we leverage recent advances in deep learning methods for joint species range estimation [13]
Existing work in species range estimation typically focuses on the offline setting whereby the entire set of observations is assumed to be available at training time [18] [2 [3]. As result, the range estimation model is trained once offline and not updated. There have been attempts to use the trained models to predict which geographic location to investigate next to determine if a species might be present 38 32 However, these methods have typically not been developed,
quantified, in a fully online setting where new data is requested sequentially to update the model. Our approach instead makes the assumption that the geographic range of an unmapped species can be represented by weighted combination of existing ranges. Through iterative active querying guided by these existing ranges, we efficiently estimate these weights online using only a small number of observations. An overview of this process is illustrated in Fig
Our core contribution consists of novel active tearning-basse solution for efficient species range estimation. We quantitatively evaluate the performance of our approach and compare it to alternative active learning methods on two challenging datasets containing expert-derived evaluation range data from one thousand species. We obtain a 32% improvement in mean average precision after ten time steps compared to a conventional active learning baseline on our globally evaluated test set. This translates to a significant reduction in the number of locations that need to be sampled to obtain reliable estimate of species range. Code for reproducing the experiments in our paper can be found com/Chris. Chriss lange/ socttve__smmplaee
at https::///// // thub. com/
2 Related work
2.1 Species distribution modeling
Species distribution modeling refers to family of methods that avail of species observation and geospatial data to estimate the geographic range of a species [17]. These models typically take some encoding of spatial region of interest input and output numerical score indicating how likely it for a particular species to be found there. There a large body of work on this topic, but existing methods can be broadly categorized based on the type of data they train on (e.g., presence-absence or presence-only data) and the output quantity they attempt to predict (e.g.. species presence or abundance) 7
In the case of presence-absence data, at training time we have access to observations containing a set of locations where species of interest is confirmed to be present and also locations where has been confirmed to be absent. The advantage of this setting is that it is easy to fit standard supervised classification methods to the training data, e.g., decision trees [9], This type of data

---

can be obtained from skilled observers that fill out detailed checklists recording the species they observe in a given area and time 44 In contrast, in the presence-only setting. we do not have access to confirmed absences. Existing methods either factor this into the model [34] or synthetically
generate pseudo-absence data for training [55) Presence observations are easier to collect than
presence-absence data as they are gathered opportunistically by observers [4. Like all data-driven
methods, these approaches have difficulties in estimating accurate ranges in the low data regime [30]
While traditional machine learning solutions have been shown to be effective for range estimation [8] recently we have observed growing interest in the application of deep learning methods to the task [12 8 16 45 [3] [5]. In contrast to the majority of existing single species methods, these deep approaches jointly model multiple species in the same model, thus exploiting the fact that some species share similar range characteristics. In this work, we do not innovate on the underlying range estimation models used, but instead we show that existing methods that are trained on readily available weakly supervised presence-only data can provide transferable representations that can be used by active learning-ban methods.
2.2 Active learning and species range estimation
Motivated by the fact that obtaining sufficiently diverse and informative training data can be one of the main bottlenecks when applying machine learning solutions to many real-world problems, active learning attempts to frame the data collection process as an optimization problem [4.. Through interaction with a user, the goal is to minimize the number of datapoints that need to be labeled while still obtaining high performance on the task of interest, e.g., in our case, estimating species' range. A variety of active querying strategies have been proposed in the literature, including uncertainty sampling (88) committee-basee based methods [3], and methods that seek to label the datapoint that would have the largest impact on the model's parameters 42), among others.
The optimal design of spatial surveys for sampling geographic region of interest when estimating spatially varying quantities is central question in geostatistics [14] and has applications in tasks such as sensor placement [26]. Both non-adaptive and sequential methods have been explored for related tasks in the context of tractable parametric models [40]. In the case of deep learning [36]. active sampling methods have also been shown be effective for predicting spatially varying quantities in
the context of remote sensing data (37]. However, the application of active sampling based methods for large-scale species range estimation remains under explored. While we focus on estimating species ranges, our approach is applicable to any spatially varying quantity where it is possible to obtain a set of possible related predictions. Potential alternative applications include sensor placement, geographic priors for image classification, and disease modeling, in addition to related tasks such as active data collection for training image classifiers.
Xue et al. [48 explored different methods for guiding observers to under sampled locations to address the issue of spatial data bias in crowd collected datasets. However, their focus is not on estimating ranges, but instead to attempt to counteract data sampling biases. There have been attempts to use species distribution models to help guide the search for under observed species [2] 47 23
Rosner-Katz et al. (38] proposed spatial sampling approach that involves stacking the outputs from a small number of range estimation models in order to test if locations that are estimated to contain rare species are good locations to check for other rare species. Marsh et al. 32 outlined approach for defining a utility score for unsampled locations by measuring the difference in the output of a distribution model that is updated assuming the species of interest occurs there, does not. This score can be used for suggesting locations to sample. However, it is expensive to compute as requires updating the model for each possible location of interest. Different from these existing works, we propose a new efficient approach for the sequential acquisition of species presence or absence observations to efficiently parameterize models for range estimation.
Active species range estimation
3.1 Species range estimation
The goal of species range estimation is to predict the geographic range that species occupies given a sparse set of observations at training time. Specifically, in the presence-absence setting [7], we are given a dataset S {(x,, v4) N.N. where each æ; is feature vector encoding geographic location
3

---

of interest (i.e., a specific latitude and longitude). In this work we focus on spatial observations, but it possible to extend our models to opatio-temporal data. The corresponding yi {0, 1] is a binary label indicating if the species has been observed to be present (y:- = 1) or found to be absent (y= = 0) at the geographic location represented by a;;.
The input features a can simply represent transformed latitude and longitude values (i.e.. x [lat, lon]), set of environmental features that encode information about the climate and/or habitat present at the location (i.e., a = [e1, ,e,......... Ea) [1]. or could be a set of latent features extracted from geospatially aware deep neural network (i.e., a f(lat, lonl)) BT. In our binary classifi- cation setting, the goal is to learn the parameters of a model h such that it correctly classifies the
observed data. This can be achieved by minimizing the cross-entropy loss with respect to the training data:
LCE =
Σ y (lo(((())) + (1 -11-17 yll log(1 - h(x))).
(1)
(2
Here, we represent a prediction from the model for one species as h(x=)) E [0, 1]. In the simplest case, could be a logistic regressor with h((+)= = (0 æ²), where a is the sigmoid function and is the parameter (i.e., weight) vector that is learned for each species. Once trained, the model can be densely evaluated across all locations of interest ge. the entire globe) to generate a predicted range map for one species.
3.2 Active location sampling
In the previous section, we assumed we had access to a labeled training set to train the range estimation model h. As noted earlier, obtaining this data on a global scale for rare or difficult to observe species can be extremely challenging as well as time consuming, even with a spatially distributed set of observers. Given this, it would be highly desirable to target limited observation resources to geographic locations that would result in observations that would be most informative for estimating the parameters of the model.
By framing our task as an active learning problem geographic locations for observation in order to improve the performance of the range estimation model for a given species while minimizing the total number of required observations. The active learning process can be broken down into three main steps; querying/smmpling: determining the next location to observe, labeling: obtaining the observation label for the sampled location (i.e., a species present or absent over specified temporal interval), and updating: changing the model parameters based on the newly acquired data. Greedy active learning approaches simply query the next datapoint to be sampled one instance at time. In this greedy setting, after a datapoint is observed and labeled, the model is updated. This process continues until some termination criteria is met, e.g., a maximum number of iterations only a minor change in the model parameters is obtained.
our goal to efficiently select and prioritize
One of the most commonly used strategies for greedy active learning uncertainty sampling [28 This involves sampling the feature that represents the location æ that the current model is most uncertain about:
xx arg min 0.5 Ply 1|x, S.).
Here, Ply = 11xx = h' h((z) indicates the output of the model at the time step f, and St {(F1, M1), (I22, v2)..... "a....,(2,, (zr,)t} is the set of training observations acquired up to and including time f. Once sampled, x² is labeled (i.e., an individual goes to the location represented by x and determines if the species is present there, or not) and then added to the updated set of sampled observations S++1 St U (x",y"). Finally, the current model is updated by fitting it to the expanded set of sampled observations. Later in our experiments, we compare to uncertainty sampling, in addition to other baseline active learning methods, and show that they are not as effective for the task of species range estimation in early time steps.
3.3 Leveraging candidate range estimation models
Unlike conventional classification problems found in tasks such as binary image classification, the ranges of different species are not necessarily distinct, but can instead have potentially large amount of overlap (see Fig. (left). Inspired by this observation, we propose a new active sampling strategy that makes use of set of candidate range estimation models.

---

We assume that we have access to set of candidate pretrained models 7. h2, hy..... h Each individual model encodes the spatial range for a different species, with one model per species. In practice, these can be a set of linear models (e-g., logistic regressors) that operate on features from a geospatially aware deep neural network, where each model has an associated weight vector 0g- Having access to large and diverse H may initially seem like a strong assumption, however in practice, it is now feasible to generate plausible range maps for thousands of species thanks to the availability of large-scale crowd collected species observation datasets 45 13). Importantly, we assume the species that we are trying to model using active learning is not already present in H.
Assuming the candidate models are linear classifiers, we can represent our range estimation model h, for unseen species, as a weighted combination of the candidate models' parameters:
[2]
0
By averaging the parameters of our candidate models we generate a compact linear classifier that can used to predict the presence the new species, without requiring access to all models in H. during inference. When based on the agreement between a model h2 and the observed data in St, Assuming a uniform prior over the candidate models in H. we model the posterior as P(ha(S*) ((S*(h)) and the resulting likelihood is represented as:
observation is added to St, the posterior P(hy|S*) is updated
P(S = II yhx(2)) + +1 ([11-((111))) hx(x)).
(4)
(m.a)s
This is related to the seery--y-commiitee--ettiig in active learning 43 where our candidate models represent the individual 'committee' members. However, instead of being trained on subsets of the sampled data online during active learning, our candidate models are generated offline from completely different set of observation data from a disjoint set of species.
3.3.1 Active learning with candidate range estimation models
Given our estimate of the model parameters from Eqn. 3 can apply any number of active learning query selection methods from the literature to perform sample (i.c., location) selection. However, while candidate models in H provide with effective way to estimate the model parameters online, we can also use them to aid in the sample selection step. Specifically, given the likelihood of an observation from model, weighted by P(he|S*), we choose the location xx to query that is the most uncertain:
xx arg min |0.5
1
Σ hy(=))(h)((())) by N
[71]
The intuition here is that we would like to sample the location that is currently most uncertain across all the candidate models. Our approach is motivated by the desire make of existing well characterized ranges when modeling the range of a less well described species, where observations may be very limited, to reduce the sampling effort needed.
The success of our approach hinges on the hypothesis set containing a sufficiently diverse and representative set of candidate models. If the target test species' range is very different from any combination of the ones in H. our ability to represent its range will be hindered, Additionally, even the test species' range is broadly similar to some weighted combination of the models in H. some amer-grained nuances of the range might be different. To address this, we extend the hypothesis set by adding additional model honline- Aualime is also a logistic regressor, but it is trained on the progressively accumulated observations from the test species that are obtained during active learning using cross-entropy objective from Eqn. [ Adding heatine ensures that also have access to the maximum likelihood model according to the observed data.
We refer to our approach as HSS, which indicates that it uses a Weighted Average of the hypothesis set to represent the model weights, and performs Hypothesis Set Selection during active querying. Similarly, WA_ HSS+ is approach with the inclusion of healine-
5

---

4 Experiments
In this section we quantitatively evaluate our active learning approach on two existing benchmark datasets which we adapt to our online learning setting.
4.1 Implementation Details
Experimental paradigm. Our experiments are designed to compare different active learning strategies using synthetic presence-absence data drawn from expert-derived range maps. We begin by discretizing the world into a set of valid explorable locations which correspond to the centroids of all land overlapping resolution five H3 cells [T] Here, each cell encompasses on average. At each time step, and for each active learning strategy, we select one location x to query from any valid location that has not yet been sampled for the presence (or absence) of the species of interest. The corresponding species presence or absence label y* for x" is then generated according to the "ground truth" range map. In the real world, this step represents an observer visiting the queried location and searching for the species of interest. While our model could evaluate any continuous location, using H3 cell centroids ensures that queryable locations are equally spaced across the surface of the globe. We commence each experiment by randomly sampling one presence and one absence observation for each species. Experiments are performed independently for each species in the test set, and there is no interaction between species during the observation process. This realistic, as in practice would be unreasonable to assume that a single observer would be able to confirm the presence of more than a small number of species owing to their finite knowledge of different biological organisms. We repeat each experiment three times using different initial labelled examples and show standard deviations as error bars. We provide additional implementation details and a description of each of the baseline methods in the appendices.
area of 252 km2
Training. For our feature representation æ (discussed in Sec. 3.1 we use a learned spatial implicit neural representation obtained from pretrained neural network. Specifically, connected network f with presence-only data using the LAN loss proposed in [13.. We use the same original set of publicly available data from iNaturalist [5] but retrain the model from scratch after removing species that are in our test sets. This results in 44.181 total species. The linear classifiers corresponding these species form the set of candidate models H. used by our approach. Each model in H has a dimensionality of 256, i.e., each is a logistic regressor with 256 weights. During the active learning phase, every time a labeled observation is obtained, for models that require linear model to be estimated (e.g.. LR models or models with honline), we update the classifier parameters 0 for the species in question using the cross-entropy objective in Eqn. [ We do not update the parameters of the feature extractor f. which are kept frozen. In later experiments, we quantify the impact of different feature representations by changing the feature extractor, e.g., by comparing models trained with environmental features input randomly initialized networks.
train fully
Datasets. We make use of two expert curated sources of range maps for evaluation and to generate labels for observations during the active learning process: International Union for Conservation of Nature (IUCN) 2 and eBird Status and Trends (S&7) 21 We use the procedure outlined in [33 to preprocess the data and randomly select 500 species from each data source for evaluation. We apply an ocean mask to remove any cell that overlaps with the ocean to make the active learning procedure more "realistic" as we do not expect naturalists interested in improving land based species range to explore the ocean to ensure that the species is not present there. This data represents our expert- presence-absence evaluation range map data. Unlike the IUCN data which has a confirmed presence or absence state for each location on the earth, the S&T data does not cover the entire globe. In the case of S&T, the valid region is different Americas. As a result, during S&T evaluation, cells which are not included within the valid region for each species are excluded from sampling and evaluation. The distribution of each test set is illustrated in the appendices.
species by species case and is biased towards the
Evaluation metrics. Each experiment consists of comparisons between different active learning strategies where we start with the same initial set of two randomly selected observations (i.e., one absence and one presence). We report performance using mean average precision (MAP), where a model evaluated at the centroid of each valid H3 cell (i.e., limited to cells that only overlap with land) and is compared to "ground truth" presence- presence-absence obtained from the test set expert-derived

---

0.8
0.7
MAP
AN FULL E22
AN FULL E2E
0.6
WA
WA KS5
0.5
WA RSS
1188
WA uncertain
uncertatn
WA random
random
0.4
155
HSS
30
10 20
Time
Time
Figure 2: Comparison of different active learning querying methods for the task of species range estimation. For each of the Iwo datasets, display the performance of different active learning methods over time, where higher values are better. The results depict the average of three runs for each method. We observe that methods that use a weighted combination of the candidate models result in the best performance (e.g., WA versus LR) particularly in early timesteps. Our WA_HSS+ approach performs best overall in both datasets. (Left) IUCN results, where we observe a large difference between methods. (Right) S&7 results, where the performance difference is smaller but our approach is superior in the earlier timesteps.
range maps. During active learning, we measure average precision across all species in the test each time step and average them to obtain a single MAP score.
Baselines. In addition evaluating WA_ HSS+ approach, compare to several baseline active learning strategies. These include random and uncertainty sampling. For each strategy, we evaluate models resulting from weighted combination from the candidate set (WA) and conventional logistic regressors (LR). We also compare to AN_ DM_FULL_E22 E2E,, the end-to-end trained implicit network method from [13] using their fuN- loss trained with all available data for each species. While this model is trained only on presence-only data using pseudo-negatives, it has access to many more observations per species (i.e., minimum of 50 presence observations per species) and is not restricted to only sampling data from land. Additional comparisons are provided in the appendices.
4.2 Results
Comparison of active learning strategies. In Fig. 2 we present quantitative results on the JUCN and S&T datasets, and show qualitative results Fig. 4 We compare our WA_HSS: approach to multiple different baseline active learning methods and observe superior performance compared to each. The performance on individual species can vary a lot, but the average MAP across all test species is similar from run to run.
There clear advantage when using our active sample selection method (i.c., comparing WA_HSS+ to random sampling in WA_random) on the IUCN test set. The addition of the online model estimate also beneficial (i.e., comparing WA_HSS+ combination of the hypothesis set (i.e., WA methods) is significantly better than conventional logistic regression (i.e., LR methods). The performance improvement on the S&7 test set is less dramatic. Here, even the random selection baseline performs well. This can possibly be attributed to the limited valid locations to explore for the species, the relative larger sizes, and the American focus its ranges compared to the more globally distributed IUCN data. However, we still observe advantage when using WA methods in the early time steps. As an additional baseline, we also compare to an end-to-end trained presence-only model from [13] which has access to significantly more observation data at training time. We can see that our WS_H method approaches, and then exceeds, the performance of this baseline in fewer than 40 time steps for the IUCN species.
WA_HSS), Estimating model parameters as a weighted
Impact of the feature space. Our approach makes use of weakly supervised pretrained feature extractor f to generate the input features æ. Here, we explore different features spaces: nn loc feats are the standard deep network features from 13 that uses location only inputs, nn env feats are deep network features that use location and 20 additional environmental covariates (i.e., features that describe local properties such as amount of rainfall and elevation) from [20), and nn rand feats are from randomly initialized and untrained version of the standard network. We also
7

---

Impuct Foniaree Space
Impact Hypothasix Sut
0.0
0.8
0.6
0.6
0.5
0.6
1555 40101
WA. 188+. 10000
WA. 2555, 10000 up.
0.3
0.3
1000
Wi 1855+ rand Feass
WA 1855, 100
0.1
WA 1055+, lot dhate
0.1
IA 1855+, 10
WA Host one toals
WA,HSS, 10 sp
0.0
0.0
10
50
100
30
50
Time
Time
Impact Label Noisu
Impuct Hypotbasss Weighting
0.0
0.7
0.6
0.6
0.4
18S5 Saien
0.4
0.35
0.3
L.
0.2
WA. 188
suise 0.25
VIA H155
ionoectain notso 0.25
VIA. 1158
hard volee
0.1
1555 solas
0.1
1155-+ no weighhiing
wrig M
10 20 30
40
20
10
20 30
40
50
Figure 3: Ablations of our approach on the IUCN dataset. (Top Left) Impact of the feature space used to represent æ. (Top Right) Impact of the number of species in the candidate set H. (Bottom Left) Impact of observation noise by introducing false negatives, where higher label noise amounts indicate that the observer is more likely to incorrectly predict that species is not present at location. (Bottom Right) Impact of candidate model weighting during active query selection.
perform experiments using the same input features but without a deep feature extractor (i.c., loc feats and env feats). In this setting, the candidate models are just logistic regression classifiers operating directly on the input features. In Fig. (top left) we observe that the deep feature extractors significantly outperform non-deep ones. This highlights the power of transfer learning from disjoint species in the context of range estimation. This has not been explored in the exiting literature, where conventional approaches typically train models on environmental covariate input features (18). Interestingly, the fixed random higher dimensional projection of nn rand feats performs better than using only the low dimensional location or environmental features, loc feats and env feats. This is likely because the low dimensional features are not sufficiently linearly separable.
Impact of number of candidate ranges. One of the strengths of our approach is that it can leverage representations learned from disjoint species. In this experiment we quantify the impact of reducing the number of species available when training the backbone feature extractor which also results in reduction of the size of the candidate set H. Specifically, we train different feature extractors from scratch, where each model has access to different subset of the data during training, e.g., WA_HSS, 1000 sp is trained on data from only 1,000 species. In Fig. (top right) we unsurprisingly observe a drop in MAP when the size of H. is small (e.g., < 1,000 species). As the size increases, the model performance improves. This illustrates the benefit of larger and more diverse candidate sets. This reduction has larger impact on WA_HSS compared WA_HSS+ due to the inclusion of hontine in the case of the latter.
Impact of observation noise. Here we simulate the impact of observation noise by exploring the setting where the species is actually present at the location but the observer misses it. Thus we obtain false negative observations, but not false positives. We simulate different noise rates where the observer misses the species at each time step with a probability sampled from f0%, 10%, 25% 50%). In Fig. 3 (bottom left) observe that approach is surprisingly robust non-trivial amounts of observation noise, and only starts to fail when the noise level becomes very large (i.e., 50%). In contrast, the best logistic regressor baseline is much more impacted by such label noise.
Impact of hypothesis weighting. Unlike traditional query by committee methods where each committee member is trained on subset of the available data [33, our hypotheses represent models
8

---

MA __SHSS++ loc feats
MA. ra.random loc feats
LR. uncerraaii loc feats
Time step 0
Time step
Time step 8
Figure 4: Predicted range maps for the Yellox-footed Green Pigeon for three different active learning strategies, illustrated over three different time steps. We also display the expert range map centered on India and Southeast Asia, inset in the top left. The queried locations are marked with x, with green representing present locations and red representing absent ones. WA_HSS+ quickly identifies both the Indian and Southeast Asian part of the species' range.
trained different species. Thus, individual model will necessarily have the same range the test species. To compensate for this, we weight each E H by the likelihood that hypothesis, P(h S*), as indicated in Eqn. [ In Fig. (bottom right) observe that removing this weighting term (no weighting) significantly reduces the performance of WA HSS+, particularly at later time steps where St contains enough samples effectively identify and downweight hypotheses that are longer consistent with the data. On the other hand, averaging the "soft votes" of each hypothesis (i.e., multiplying by by (ar) as in Eqn. 5) (WA WWA_HS)),, averaging the "hard votes" (i.e., (a:) 0.5] used in WA HSS+ hard labels) as in traditional query by committee methods [24), results in similar performance. The uncertainty of any one model is not particularly important, potentially due the large number of hypotheses that can contribute to sample selection.
Limitations
In spite of the significant improvement in performance our approach provides, it does have some limitations which we leave for future work. Firstly, all active learning strategies we consider query single points rather than diverse batches 19]. This might limit the use of our strategies in situations where there are multiple observers available to gather data simultaneously. The geographic accessibility of specific locations is another limitation as we currently do not consider the practical difficulties associated with reaching certain geographic areas, e.g., remote regions. We also do not explore other forms of observation noise outside of failing to detect species of interest when present. However, errors associated with misidentifying species are mitigated on platforms such as iNaturalist who use community consensus mechanism to assign labels to observations.
The performance of our of approach is limited by the expressiveness of the provided learned feature space and the diversity of species present in the candidate hypothesis set. We are also impacted by any inaccuracies in the candidate ranges that may result from their inability to capture very high addition any class imbalance or geographic sampling
spatial resolution range characteristics, biases in the data used to train the underlying models. However, our method will further benefit from advances in handling noise and bias the context of training joint range estimation models [11] in addition to improvements in architectures and location encodings [39] Finally, the expert-derived evaluation data we use represents the best available source, but it is still not guaranteed to be free from errors as characterizing species' true range is challenging problem.

---

Broader impacts. The models trained as part of this work have not been validated beyond the use cases explored in the paper. Thus care should be taken when using them for making any decisions about the spatial distribution of species. We only performed experiments on publicly available species observation data, and caution should be exercised when using these models to generate range maps for threatened species.
6 Conclusion
Obtaining sufficient number of on the ground observations to train species range estimation models one of the main limiting factors generating detailed range maps for currently unmapped species. To address this problem, we presented one of the first investigations of active learning in the context of efficiently estimating range maps from point observation data. We also proposed a new approach that makes use of a pre-existing learned environmental feature representation and set of range maps from disjoint species to enable efficient querying of locations for observation. Even though this feature representation is derived from a model trained on weakly labeled presence-only data, we show that it is still very effective at transferring to new species. Our approach is computationally efficient operations are linear with respect to the input features which ensures that we can easily
as all of scale to tens of thousands of species. Through extensive quantitative evaluation, we demonstrated that it is possible to obtain significant improvement in the quality of the resulting range maps with much fewer observations when compared to existing active learning methods.
Acknowledgements. This project was funded by the Climate Change AI Innovation Grants program, hosted by Climate Change Al with the support of the Quadrature Climate Foundation, Schmidt Futures, and the Canada Hub of Future Earth. Funding was also provided by the Cornell-Edinburgh Global Strategic Collaboration Awards.
References
[1] HB. httts://h3geo.. org/ Accessed on 2023-05-16.
[2] The IUCN List of Threatened Species. Version 2022-2. https::/www..ucnreell.. Accessed on 2023-05-16.
[3] PlantNet. https://plantnet..om Accessed on 2023-05-16.
[4] iNaturalis,, fttps://ww.iinaturallie
Accessed on 2023-05-16.
[5] iNaturalist Accessed on 2023-01-13.
[6] N. Abe, Query learning strategies using boosting and bagging. ICML, 1998.
[7] S. Beery, E. Cole, J. Parker, P. Perona, and K. Winner. Species distribution modeling for machine learning practitioners: review. In SIGCAS Conference on Computing and Sustainable Societies, 2021.
[8] C. Botella, A. Joly, P. Bonnet, P. Monesticz. and F. Munoz. A deep learning approach to species distribution modelling. Multimedia Tools and Applications for Environmental and Biodiversity Informatics, 2018
[9] K. Brinker. Incorporating diversity in active learning with support vector machines. In ICML, 2003.
[10] W. Cai, Y. Zhang, and J. Zhou. Maximizing expected model change for active learning in regression. In ICDM, 2013.
[11] D. Chen and C. P. Gomes. Bias reduction via end-to-end shift learning: Application to citizen science. In AAAI, 2019.
[12) D. Chen, Y. Xue, Chen, Fink, and C. Gomes Deep multi-species embedding. arXiv:1609.09353. 2016.
10

---

[13] E. Cole, G. Van Horn, C. Lange, A. Shepard, P. Leary, P. Perona, S. Loarie, and O. Mac Aodha. Spatial implicit neural representations for global-scale species mapping. ICML, 2023.
[14] N. Cressie. Statistics for spatial data. John Wiley & Sons, 2015.
[15] C. L. Davis, Y. Bai, D. Chen, O. Robinson, V. Ruiz-Gutierrez, C. P. Gomes, and D. Fink. Deep learning with citizen science data enables estimation of species diversity and composition at continental extents. Ecology, 2023.
[16] R. De Lutio, Y. She, S. Aronco, S. Russo, P. Brun, J. D. Wegner, and K. Schindler. Digital taxonomist: Identifying plant species in community scientists' photographs. ISPRS journal of photogrammetry and remote sensing, 2021.
[17] J. Elith and J. R. Leathwick. Species distribution models: ecological explanation and prediction across space and time. Annual review of ecology, evolution, and systematics, 2009.
[18] J. Elith, C. H. Graham, R. P. Anderson, M. Dudík, S. Ferrier, A. Guisan, R. J. Hijmans, Huettmann, R. Leathwick, A. Lehmann, al. Novel methods improve prediction of species' distributions from occurence data. Ecography, 2006.
[19] J. Elith, J. R. Leathwick, and T. Hastie. A working guide boosted regression trees. Journal of animal ecology, 2008.
[20] S. E. Fick and R. J. Hijmans. Worldelim 2: new 1-km spatial resolution climate surfaces for global land areas. International Journal of Climatology, 2017.
[21] D. Fink, T. Auer, A. Johnston, M. Strimas-Mackey, O. Robinson, S. Ligocki, W. Hochachka,
L. Jaromczyk, C. Wood, Davies, M. Iliff, and L. Seitz. ebird status and trends, data version: 2020; released: 2021. Cornell Lab of Ornithology, Ithaca, New York, 10, 2020.
[22] M. Fois, Fenu, C. Lombrana, D. Cogoni, and G. Bacchetta. practical method to speed up the discovery of unknown populations using species distribution models. Journal for Nature Conservation, 24:42-48, 2015.
[23] M. Fois, A. Cuena-Lombrafia G. Fenu, and G. Bacchetta. Using species distribution models at local scale to guide the search of poorly known species: Review, methodological issues and future directions. Ecological Modelling. 2018.
[24] Y. Freund, H. S. Seung, E. Shamir, and N. Tishby. Selective sampling using the query by committee algorithm. Machine Learning, 1997.
[25] A. Guisan, O. Broennimann, R. Engler, M. Vust, N. G. Yoccoz, A. Lehmann, and N. E. Zimmermann. Using niche-based models to improve the sampling of rare species. Conservation biology, 2006.
[26] A. Krause, A. Singh, and C. Guestrin. Near-optimal sensor placements in gaussian processes: Theory, efficient algorithms and empirical studies. JMLR, 2008.
[27] Legendre, M. Dale, M.-J. Fortin, J. Gurevitch, M. Hohn, and D. Myers. The consequences of spatial structure for the design and analysis of ecological field surveys. Ecography, 2002.
[28] D. D. Lewis and W. Gale. A sequential algorithm for training text classifiers. In SIGIR Conference on Research and Development in Information Retrieval, 1994.
[29] S. R. Loarie, P. B. Duffy, H. Hamilton, G. P. Asner, C. B. Field, and D. D. Ackerly. The velocity of climate change. Nature, 2009.
[30] A. Lomba, L. Pellissier, C. Randin, J. Vicente, F. Moreira, J. Honrado, and A. Guisan. Overcom- ing the rare species modelling paradox: A novel hierarchical framework applied to an iberian endemic plant. Biological Conservation, 2010.
[31] O. Mac Aodha, Cole, and P. Perona. Presence-only geographical priors for fine-grained image classification. In ICCV, 2019.

---

[32] C. J. Marsh, Y. Gavish, M. Kuemmerlen, S. Stoll, P. Haase, and W. E. Kunin. Sdm profiling: tool for assessing the nnformation-conten of sampled and unsampled locations for species distribution models. Ecological Modelling, 2023.
[33] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 2011.
[34] S.J. Phillips, M. Dudík, and R. E. Schapire. A maximum entropy approach to species distribution modeling. In ICML, 2004.
[35] S. J. Phillips, M. Dudik, J. Elith, C. H. Graham, A. Lehmann, J. Leathwick, and S. Ferrier. Sample selection bias and presence-only distribution models: implications for background and pseudo-absence data. Ecological applications, 2009.
[36] Ren, Xiao, X. Chang, P.-Y. Huang. Z. Li, B. Gupta, X. Chen, and X. Wang. survey of deep active learning, ACM computing surveys, 2021.
[37] A. C. Rodríguez. S. D' Aronco, K. Schindler, and J. D. Wegner. Mapping oil palm density at country scale: An active learning approach. Remote Sensing of Environment, 2021.
[38] H. Rosner-Katz, J. L. McCune, and J. R. Bennett. Using stacked sdms with accuracy and rarity weighting optimize surveys for rare plant species. Biodiversity and Conservation, 2020.
[39] M. RuBwurm, K. Klemmer, E. Rolf, R. Zbinden, and D. Tuia. Geographic location encoding with spherical harmonics and sinusoidal representation networks. arXiv:2310.06743, 2023.
[40] E. G. Ryan, C. C. Drovandi, J. M. McGree, and A. N. Pettitt. A review of modern computational algorithms for bayesian optimal design. International Statistical Review, 2016.
[41] B. Settles. Active learning literature survey. University of Wisconsin, Madison, 2010.
[42] B. Settles, M. Craven, and S. Ray. Multiple-instance active learning. NeurlPS, 2007.
[43] H. S. Seung, M. Opper, and H. Sompolinsky. Query by committee. In COLT, 1992.
[44] B. L. Sullivan, C. L. Wood, M. J. Iliff, R. E. Bonney, D. Fink, and S. Kelling. ebird: A citizen- baarenn-baeeddde bird observation network in the biological sciences. Biological conservation, 2009.
[45] M. Teng, A. Elmustafa, B. Akera, H. Larochelle, and D. Rolnick. Bird distribution modelling using remote sensing and citizen science data. Tackling Climate Change with Machine Learning Workshop, 2023.
[46] R. C. Whytock, J. Swicžewski, J. A. Zwerts, T. Bara-Stupski, A. F. Koumba Pambo, M. Rogala, L. Bahaa-el din, K. Bockee, S. Brittain, A. W. Cardoso, et al. Robust ecological analysis of camera trap data labelled by a machine learning model. Methods in Ecology and Evolution. 2021.
[47] N. Williams, C. Seo, J. Thorne, J. K. Nelson, S. Erwin, J. M. O' O'Brien, and M. W. Schwartz. Using species distribution models to predict new occurrences for rare plants. Diversity and Distributions, 2009.
[48] Y. Xue, I. Davies, D. Fink, C. Wood, and C. Gomes. Avicaching: A two stage game for bias reduction in citizen science, In International Conference on Autonomous Agents and Multiagent Systems, 2016.
12

---

Appendix
A
Additional comparisons
In Table A1 we report the numerical performance of our approach compared to multiple different active learning strategies, including additional ones that are not present the main paper. Performance is reported using per-time step mean average precision (MAP). in addition to area under the MAP curve (MAP-AUC). We also illustrate a subset of these methods in Fig. AT We observe that our VA_SSS+ approach results in superior performance.
Table AI: Numerical results on the IUCN dataset reported at the end of four different time steps (5, 10, 20, or 50). Each row is a different active querying method. We report results as the average of three different runs.
Time step 5
Time step 10
Time step
Time step
MAP
MAP-AUC
MAP
MAP-AUC
MAP
MAP-AUC
MAP
MAP-AUC
WA_RSSH
0.65
0.61
0.69
0.64
0.73
0.68
0.77
0.72
WA_ uncertain+
0.61
0.58
0.64
0.60
0.68
0.63
0.73
0.68
WA_ random+
0.57
0.57
0.58
0.57
0.60
0.58
0.64
0.61
posiiiiiiie
0.58
0.57
0.58
0.58
0.58
0.58
0.58
WA_ _SS
0.64
0.61
0.68
0.63
0.71
0.67
0.73
0.70
WA_ uncertain
0.58
0.64
0.60
0.67
0.63
0.70
0.66
WA_ random
0.57
0.57
0.58
0.57
0.60
0.58
0.63
0.60
WA_ _A__osiiiiee
0.58
0.57
0.58
0.57
0.58
0.58
0.59
0.58
LR_ ARR_SSS
0.36
0.38
0.38
0.38
0.38
0.38
0.39
0.39
LR_ uncertain
0.48
0.47
0.52
0.49
0.65
0.54
0.74
0.64
LR_ random
0.48
0.46
0.50
0.48
0.53
0.50
0.58
0.53
LR_ positive
0.39
0.40
0.43
0.41
0.48
0.43
0.47
LR_ QBC
0..99
0.47
0.53
0.49
0.75
LR_ EMC
0.47
0.46
0.51
0.47
0.61
0.52
0.71
0.62
Performance of WA Baseline Strategies
Performance of LR Beseline Strategies
0.8
0.7
0.7
0.5
XA2
0.5
unoertain
HSS
HSS
uncertain
randan
0.4
0.4
100
20
30
40
50
10 20
30
40
50
Time
Time
Figure A1: Comparison of additional active learning methods for the task of species range estimation on the IUCN dataset. We display the performance of different active learning methods time, where higher values are better. This figure similar to Fig. 2 (left) the main paper, but includes additional comparisons.
B Visualizations
B.1 Learned representations
In Fig. A2 (left), similar to the visualizations in 13] we display visualization of the learned features from neural network backbone. We use ICA to project the 256D vector 3D and only display non-ocean features. Fig. A2 (right), we show 2D TSNE visualization of the candidate model hypothesis set H. where each point represents different species weight vector We show resulting range predictions (i.e., a00 x;)) for three different species (h = Ludwig's 's Bustard, hz Namib Rock Agama, and hg Black-headed Ibis), where we observe that the two species with similar ranges (h₃ and fiz) have similar now-dimensional embeddings.
13

---

In Fig A3 we illustrate the learned combination of species' weight vectors that are used to generate predictions for three unseen test species. We observe that highly weighted species from H. often form distinct clusters in the TSNE representation, where each cluster is comprized of species whose ranges form one part of the test set species range. This helps explain why it meaningful learn linear combination of pretrained ranges as in our method. With very few observations species with similar ranges can be identified and exploited to create good range map for the species of interest.
60
Noural Backbone
toats)
Candidatc Hypothesis Space
Predicted Ranges
Figure A2: (Left) 3D ICA projection of features for (Right) 2D TSNE visualization H for a nn coord feats neural network backbone with predicted range maps shown for species. Here, hı = Ludwig's 's Bustard, h2 Namib Rock Agama, and ly Black-headed lbis. Points close in the hypothesis space represent similar ranges.
feats neural network backbone.
House Finch
American White Pelican
Mouraful Spbinx
Scilt Sandpiper
Bicycle Dragon
Diamond Dove
Mallsofowl
Six-Bended Armadillo
Giant Ameiva
Chaco Chachalaca
Brown Widoo
TSNE
Range
Candidate Species
Candidate Species
Species 3
Figure A3: Visualization of the learned combination of species" weight vectors that are used to generate predictions for three unseen test species, each shown on a different row. On the left we show TNSE visualization of the hypothesis weight at the end of active learning for our WA_ WA_HSS+ approach. For each of the species visualized, we indicate the highest weighted candidate species from a set of highly weighted locations that are indicated using red, blue, or green marker on the left.
where brighter colours indicate species that have greater
B.2 Additional qualitative results
In Figs. A4 to A6 we display additional qualitative results. In each figure, we compare the outputs of different active sampling strategies across different time steps for the same species. In general, we observe that our WA_HSS+ approach results in predictions that are closer to the expert range map (shown the top left of each figure). However, Fig. A4 illustrates an interesting failure case where our approach fails to predict the species presence outside the region of the initial presence observation. However, from Fig. ATT we can see that the predictions for most species results higher MAP scores for HSS+.
14

---

WA _HS++ an loc feats
WA HSS nn loe feats
WA, Frrrandoo nn loc feats
LR un cecertain loc feats
Time step
Time step
Time step
Figure A4: Predicted range maps for the Yellowhammen for four different active learning strategies, illustrated over three different time steps. We also display the expert range map, inset in the top left. The queried locations are marked with Xs. with green representing present locations and red representing absent ones. this example HSS+ and WA_HSS fail to identify the most of the range while other sampling strategies are more successful.
C Implementation details
C.1 Dataset details
In Sec. 4.1 in the main paper we noted that the S&T dataset does not span the entire globe and is more spatially biased compared the /UCN dataset. Fig. A7 we visualize the number of coarse locations that have species presence for both datasets. We can see that the IUCN dataset on the left more globally distributed, compared to the S&T dataset on the right. In addition, in the case of the S&T dataset, it is only possible to evaluate IUCN dataset where we can evaluate on all possible locations on the earth's 's surface. In Table A2 we summarize the statistics of both datasets.
a subset of possible locations as opposed to the
Table A2: Comparison of dataset statistics. We evaluate our models at H3 [1] resolution of five, which results in 2,016,842 possible cells distributed across the globe before we exclude ocean locations. The S&T dataset covers smaller percentage of the globe compared to the IUCN dataset.
IUCN
S&T
Avg. # valid H3 cells
569,094
162,008
Avg. # positive HB cells
19,614
42,429
Avg. # valid percent of globe
28.22
8.03
Avg. # positive percent of globe
0.97
2.01
Avg. # positive / valid percent
3.45
26.19
15

---

WA JHS++ an loc feats
WA HSS nn loe feats
WA, Frrrandoo nn loc feats
LR un certatii nn loc feats
Time step
Time step
Time step
Figure A5: Predicted range maps for the Gold Tegu for four different active learning strategies, illustrated over three different time steps. We also display the expert range map centered on South America, inset in the top left. WA_HSS+ quickly identifies the majority of the range.
C.2 Input features
In our experiments we compare different input feature encodings æ to represent locations on earth's surface. Unless otherwise specified, we use four dimensional trigonometric encoding of the scaled latitude and longitude (either directly or via a neural network-bast bbsood feature extractor). This encoding is used to prevent discontinuities at the connected edges of the world map [3]
lalm 90
lata 90
lono 180
Lon
x
sin
cos
.sin
.coss
180
In the results in Fig. 3 (top left) in the main paper we compare to models that use features derived from neural-network backbone (i.e., those with in the name). neural network backbone is not specified, the input features are instead passed directly to the classifier. These neural network feature extractors are either randomly initialized using location only features and not trained (i.e.. rand), are trained using the procedure outlined in [33 with only location features (i.e., loc) as input or location and environmental covariates (i.e., env).
Environmental covariates are commonly used for species distribution models as it allows them to attempt to learn the "fundamental ecological niche" of the species of interest [7]. By also including the encoded coordinates we make our models spatially explicit and allow them to distinguish between locations that have similar environmental characteristics but are spatially distant. We use 20 environmental covariates (including elevation) generated from elevation and bioclimatic rasters at five are minute spatial resolution, taken from the widely used Worldelim 2.1 dataset [20]. Each covariate is normalized by subtracting the mean and dividing by the standard deviation, and NaN values are replaced with O. Bilinear interpolation is used to generate corresponding environmental covariates for any given location. Models that take environmental features as input therefore accept 24 dimensional inputs, with four dimensions encoding location information and the remaining 20 representing the bilincarily interpolated environmental covariates at the location.
16

---

WA JHS++ an loc feats
WA HSS nn loe feats
WA, Frrrandoo nn loc feats
LR un cerrrtain loc feats
Time step
step
Time step
Figure A6: Predicted range maps for the White Wagtail for four different active learning strategies, illustrated over three different time steps. We also display the expert range map centered on Asia, Africa and Europe, inset in the top left. WA_HSS+ quickly identifies the majority of the range.
TUCN
58T
222
To sxcetber of provatco erac
120
Figure A7: Count of the number species presences for each coarse geographic location for the IUCN (left) and S&T (right) datasets. We can observe that IUCN dataset much more globally distributed and less North American biased than the S&T dataset.
C.3 Backbone feature extractor
In addition to the input encoding, our method involves extracting features from a deep species range estimation model trained on community collected presence-only data. This network uses the same architecture, training procedure, and hyperparameter settings the LAN-full model from [13] We also use the same training data from iNaturalist [5]. but we remove classes (i.c., species) that are present in the full set of species (i.e., not just the 1,000 in subset of the test species) in either of our test datasets. This reduces the number of species to 44,181.
The deep species range estimation model is a fully connected neural network consisting of an input layer, then four residual layers, each comprised of two fully connected layers with skip connections between them, and finally classification head consisting of línear classifier per-species, each of which predicts the presence or absence of single species using a sigmoid output. can be considered location encoder f that feeds into set of single layer classifiers H which all share the
17

---

same input features. ReLU nonlinearities are used between layers, and each hidden layer consists of 256 neurons. Dropout layers are also utilized. This model is trained for 10 epochs using Adam optimizer with a dropout probability of 0.5 and batch size of 2048. Training the feature extractor takes 2.5 hours on an NVIDIA RTX A6000 GPU with 48GB of RAM.
After training, the frozen network f can then be used as feature extractor, generating 256 dimensional embeddings used for hypothesis set selection and for training new classifiers. Each per-species classifier & can be used as the set of hypotheses H. in our WA approaches. As f is frozen during active sampling and the locations that can be sampled are limited to the centroid of resolution five H3 cells, features can be generated ahead of time by applying the feature extractor to each centroid and storing the results. Thus f not required during the active sampling process.
C.4 Active sampling
In each experiment, we apply the chosen active sampling strategy 500 held-out test species randomly selected from each of the International Union for Conservation of Nature (IUCN) 2 and eBird Status and Trends (S&T) [21] datasets. We use the same 500 species from each dataset for each experiment, and use the same initial samples consisting of a single presence and absence observation for each experiment. The species set in each dataset is not the same, e.g., the S&T dataset contains only birds. Each experiment is repeated three times with different seeds and initial samples. The same frozen feature extractor is used across all experiments except 'Impact of Feature Space' experiments in Fig. 3 in the main paper and the additional ablations in this appendix.
We evaluate our performance using per-time step mean average precision (MAP), and using area under the MAP curve (MAP-AUC). As we synthetically generate data from the underlying expert range maps provided, do not have specified train test split and merely evaluate at each valid H3 cell centroid and measure performance as the difference between a model's 's predictions and the expert-derived range map. Unlike [3] we evaluate our models only on land and not also over the ocean.
The logistic regression range estimation model h we optimize during active sampling is implemented using Siikitleearn i linear_meer maner_modeel... If the features from f have been stored ahead of time, then each experiment can be performed without a GPU as it consists only of training and evaluating a logistic regression model. single experiment using WA_HSS+ method involving 500 species across 50 time steps takes 4 hours using unoptimized code on an AMD EPYC 7513 32-Core Processor.
class with default settings [33].
C.5 Baseline strategies
Here we provide additional information about the baseline active sampling strategies we evaluated. In all cases, the set of possible locations from which to select sample x² consists of all valid resolution five H3 cell centroids (i.c., those that are located over land and have not yet been sampled, and for the S&T dataset only those where the ground truth is provided). The LR strategies below update the range estimation model at each time step by minimizing the cross entropy loss in Eqn. from the main paper.
LR random. x" is selected randomly from all valid H3 cell centroids.
LR_ A___ncertain. x² is determined by Eqn. 2, and corresponds to the location where the model's probability of presence is closest to 0.5. This is of the most well established and commonly used methods of active sampling for classification.
LR EMC. xx is selected to maximize the absolute gradient of the model parameters O with respect to the loss function, taking an expectation over the current distribution of class labels. We are looking for the sample that will result in the largest expected model change. The intuition is that we expect this location will cause our model to "learn" the most [0]
LR HSS. x' is selected according to Eqn. 5 in the main paper, where the weights of the hypothesis set H are determined by agreement between the hypotheses and the sampled data, and the location that the weighted hypotheses are most uncertain about is then selected. However unlike WA_SSS, h not updated by averaging these hypotheses, but by updating the weights for a logistic regression classifier to minimize the cross entropy loss.
18

---

LR QBC. This "query by committee' method 43 24] is similar to LR_HSS above but H. is now set of classifiers trained on subsets of the currently available data S in [6] Due to the small size of S¹, especially in early time steps, subsets are created by removing single data point from S", and then discarding subsets that only contain points from a single class.
LR_ _o_sssiteee. x" is selected as the location that has the highest probability of presence according to h. This querying method has been used within the ecology literature as a method for evaluating generated species distribution models or as a 'guided search' method [22 23
We also compare WA_HSS+ strategy others that use weighted averaging of H. update h. but do not use hypothesis set selection to determine x=. The following WA strategies can also be modified by the inclusion of healine, leading to WA random+, WA uncertain+, and WA positive+ baselines.
WA_random. xx is selected randomly from all valid H3 cell centroids.
WA_uncertain, x" is determined by Eqn. 2 in the main paper, and corresponds to the location where the model's probability of presence is closest 0.5.
vosiiiveeee. x' is the location that has the highest probability presence according to h.
D Additional ablations
Here we provide the results of additional ablation experiments.
D.1 Impact of label noise
In Fig. A8 we demonstrate the impact of adding label noise for WA_HSS+ compared to WA_HSS This expands on the results in Fig. 3 (bottom left) in the main paper. While WA_HSS- outperforms WA HSS in the presence of modest label noise, as more noise is added, the WA_HSS strategy performs slightly better than WA_ HSS+.
Impact Label Noise
0.8
MAP
WA, HSS nouse 0.0
WA, HSS. toine 0.0
WAA HS5++ noiso 0.1
WA 1155, boine - 0.1
WA 1155+ matee 0.25
WIA, HSS. cuise - 0.25
WIA H55+ nosso 0.5
WAA H55, moise - 0.5
Time
Figure A8: Impact of adding increasing amounts of label noise the IUCN dataset. Our WA. +SS+ and WA_HSS approaches are robust to label noise compared to conventional logistic regression-based uncertainty sampling used in LR_ uncertain and seen in Fig. 3 (bottom left) in the main paper.
HSS
D.2 Impact of hypothesis set size
In Fig. A9 (left) we illustrate the impact of removing species from the training set when training the backbone feature extractor. We compare the performance of HSS using different backbone models trained with a limited number of classes to WA_HSS where we remove classifiers from H. but keep the same feature extractor trained on all classes. Fig. 3 (top right) in the main paper shows large impact from using backbones trained with fewer classes. Here we observe that in almost all cases the model using backbone trained on all classes performs best, and the performance difference increases when more classes are removed.
In Fig. A9 (right) we further show how using backbones trained on fewer classes impacts the baseline active sampling strategy. The change performance for the LR_ uuuerraii strategy underlines that jointly modeling many species in the backbone is vital for good performance due to the features
19

---

extracted becoming more useful for range estimation, even when the predicted species ranges are not directly used, as they are in WS_
Impact of Feature Extractor Training
Impact of Hlpothesss Set Size for LR Strategies
0.8
0.
0.7
0.7
0.6
0.6
0.5
0.5
MAP
0.4
0.4
WA HSS. 44:81
0.3
WA, 1158,
10000
0.3
LR LLNGNG 44161
WA 1158, 10000 hypothents sp
LR LNC, 22000
0.2
WA 1155, 1000 up
02
LR UNC. 10000
WA 1155, 1000 hypotthett sp.
LR UNC. 1000
0.1
WA 1155, 100 up
0.1
LRt UNC, 100
WA HS8, 100 typotthesis sp.
LRR IRRUNC, 50.5p.
0.0
Time
Time
Figure A9: (Left) Impact of removing species from the backbone on the IUCN dataset. Here compare models trained without species in the neural network backbone and hypothesis set (solid line) to models that use all the species train the backbone but simply remove some from the hypothesis set (dashed lines). The solid lines are the same as the solid entries in Fig. (top right) in the main paper. (Right) Impact of changing the hypothesis size the IUCN dataset for the LR_ uncertain strategy. When many fewer species are used to train the backbone the performance worse.
D.3 Impact of feature space
In Fig. A10 we explore the impact of using learned input features for our WA_HSS+ method and the logistic regression baseline LR_und LR_uncertain. The feature spaces are subset of those shown in Fig. (top left) from the main paper. We observe that using the features generated by the deep feature encoder (nn loc feats) is vital for good performance for both the LR_ uncertain and HSS+ strategies. In contrast, only using location (loc feats) or location and environmental features (env feats) directly with no neural network results in poor performance.
Impact of Peature Space
0.5
Lnnooeetunn feuts
TISS-
H88 lco foots
LR mnneectinnn lon feats
WA, H18- any fwate
LR unoeciuin, cne deats
0.2
0.0
Figure A10: Impact of using different feature spaces evaluated on the IUCN dataset. Both methods require the use of the deep feature encoder features, nn loc feats, for good performance.
D.4 Per-species performance
In Figs. ATI to A14 we display histograms of the per-species performance across six different time steps (2, 5, I0, 20, 30, and 50) on the IUCN dataset. Each individual histogram displays the distribution of MAP scores for each the species in the IUCN dataset. In general, we observe trend that performance improves over time, i.e., the mass of the histograms move the right. Our WA_ WA_HSS+ approach Fig. ATT is noticeably better than the naive random selection using logistic regression
baseline in Fig. A14
20

---

34
Figure A1l: Histograms per species MAP scores our WA. HSS+ approach across different time steps the IUCN dataset.
Imangx Trotees
10
I
1x
Figure A12: Histograms of per species MAP scores for our WA_HSS HSS approach across six different time steps on the IUCN dataset.
21

---

feunge Toekiis
L
Figure A13: Histograms per species MAP scores LR_ uuunnerrtain (i.e., logistic regression with uncertainty-baac based sample selection) across six different time steps on the IUCN dataset.
Awngge Tredkin
Aneigs Treence
10.5
15.0
Figure A14: Histograms of per species MAP scores for the LR_ _R_random baseline (i.e., logistic regression with random sample selection) across six different time steps the IUCN dataset.
22


Reviews:
[{"review_id": "lx0A731YEh", "reviewer": "Reviewer_WCfR", "summary": "This paper proposes to:\n1) Use crowdsourced data to obtain species range maps for a large set of data-rich species, and then\n2) Learn a linear combination of these maps to represent the range of other, data-poor, species, while minimizing the number of required presence/absence queries via active learning.", "questions": "As reflected in the weaknesses and limitations sections:\n1) What other fields of application and methods in the ML community would have connections to the presented methodology?\n2) Would this method eventually be feasible in reality?", "limitations": "Most limitations are properly addressed. The only thing I missed would be the feasibility of obtaining presence/absence queries. Is this something that has been done? Obtaining a presence given that the species is present is one thig, but obtaining an absence could be very challenging. Would this method eventually be feasible in reality?", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The paper is well motivated, well structured, reads well and provides a good overview of the existing literature.\nThe proposed method is simple and intuitive, and the performance improvements quite substantial.", "weaknesses": "Although I'm quite positive about the paper, I do have some small doubts about whether a more general context placement would be required for a better fir in NeurIPS. I'm no expert, but it seems to me that the authors could connect the work to methods beyond species range estimation in order to connect better with the ML community. After all, the proposed method seems to perform reconstruction with a predefined basis. Otherwise, it seems like a paper more oriented exclusively towards the SDM community. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[WCfR-1] What other application areas could your method be applied to and is this a good fit for NeurIPS?**   \nThis is an interesting question. Our approach is applicable to any spatially varying quantity where it is possible to obtain a hypothesis set over possible related predictions. Potential applications include sensor placement, geographical priors for image classification, and disease modeling, in addition to related tasks such as active data collection for training image classifiers. We will add a discussion of other application areas to the text. However, it is worth noting that the percentage of species, of the total known to science, with documented ranges is very small. As a result, there are a large number of unmapped species that methods such as ours, and future advancements, could be applied to. While the task of species range estimation has achieved attention from the machine learning community (e.g. *Phillips et al. ICML 2004*, *Chen et al. ICML 2018*, and *Cole et al. 2023*), our work lays the groundwork for future progress on the active learning version of the task which has not been explored to date.  \n\n**[WCfR-2] Confirming if a species is present is easy. How difficult is it to obtain absence information, i.e. to confirm that it is absent for a particular location?**  \nPlatforms such as iNaturalist only record presence observations, however other platforms such as eBird ask users to upload both presence and absence observations. To date, eBird users have uploaded approximately 100 million checklists where each checklist contains information about the presence and absences of all bird species for the location they sampled. Thus this type of presence and absence data is readily available and often collected. \n\nIt is true that confirming absences (true negatives) may be more time consuming than presences (true positives) for some species. However, the benefit of our approach is that we can significantly reduce the number of locations that need to be sampled to obtain an estimate of a species' range (see *Fig 2*). Furthermore, our approach is robust to noise in the form of false negatives (i.e. incorrectly stating that something is absent) that might arise in the observation process due to the potential difficulty of obtaining reliable absences (see *Fig 3 (bottom left)* and *Fig A1* in the appendix). Importantly, our hypothesis candidate set (line 164) is generated from a model that is only trained on presence data which can easily be obtained in abundance. However, we agree that it is important to point out the distinction between presence and absence observations and so will add a discussion of this in the updated text. \n"}, {"review_id": "vaICvlZuRk", "reviewer": "Reviewer_yFB1", "summary": "This paper proposes an active learning framework for estimating the geographical range of species from sparse observations. The core idea is to exploit a large set of pretrained range estimation models from many different species. This paper also provides extensive experiments including detailed ablation studies and comparisons with baselines, demonstrating the effectiveness of the proposed active learning approach.", "questions": "1. Could you elaborate on the details of LR_HSS? For HSS (hypothesis set selection)? If logistic regression instead of weighted averaging is applied during training, how to apply HSS (Eqn. 5) at the test time? It looks like the weights per hypothesis H is required during active querying.  \n2. Typos: #199-#200, WA_HHS should be WA_HSS.", "limitations": "1. Limited technical novelty. The core idea of this paper is to exploit a set of pretrained range estimation models to reduce the number of observations required for training a new model for target species. This problem studied in this work is very specific to the range estimation problem. \n2. A stronger baseline that trains a general classifier should be considered as described above.  \n3. The role of weighted averaging needs more clarification.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. It is technically sound to exploit a large set of pretrained range estimation models from diverse species to reduce the required observations for training a new model.  \n2. This paper provides detailed ablation studies and comparisons with existing methods and baselines and convincingly demonstrates superior performance.\n3. This paper is well-written and easy to follow.", "weaknesses": "1. The technical novelty is limited. It is reasonable yet straightforward to exploit a set of pretrained range estimation models for training a model for a target species.\n2. The exact reason why it is meaningful to learn a linear combination of pretrained range estimation models needs further clarification. Is this because the range of different species is highly correlated and the learned weights indicate some correlation among different species, or such a linear combination indicates some general habitat candidates? A baseline probably stronger than the LR_random (Fig. 2) is to train a general classifier on joint presence data from all available species. Such a general model may indicate whether a given location is a good habitat for birds or other animals.   \n3. Is it visualize the weights in the linear combination for some target species? Do the weight visualizations indicate any correlations among species? If so, is such a correlation related to species taxonomy? It might also be useful to incorporate the species taxonomy as input to the model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[yFB1-1]  The technical novelty is limited. It is reasonable, yet straightforward, to exploit a set of pretrained range estimation models for training a model for a target species.**  \nRespectfully, we disagree. To the best of our knowledge, we are the first work to show that it is possible to make use of a large set of pretrained models to perform efficient active learning in the context of spatially varying quantities. We quantitatively show that our approach is superior to over ten different active learning baselines (see section C4 for a description of the baselines) on the task of species range estimation across two different datasets. The problem of efficiently mapping species ranges is an important one for the ecological and conservation applications which we believe needs more attention from the machine learning community. Our problem setting is distinct from seemingly related tasks (e.g. active learning of image classifiers) where leveraging such a prior is not meaningful as the potential similarity between categories is not as useful of a signal. \n\n**[yFB1-2] Why is it meaningful to learn a linear combination of pretrained range estimation models?**   \nOur solution is motivated by the observation that “the ranges of different species are not necessarily distinct, but can instead have a potentially large amount of overlap” (line 161). The intuition behind this assumption is visualized in *Fig 1* where we illustrate that species that are similar in weight space have similar ranges (see also rebuttal Fig B2). Given a sufficiently diverse set of possible pretrained range estimation models, it is thus reasonable to assume that some combination of those ranges will be a good proxy for representing the range of a previously unseen test species. As noted on line 122, we can use a set of learned features extracted from a jointly trained geospatial deep neural network as our feature vector for each geographical location (which can be viewed as a learned encoding of the local “habitat”). The range for a species of interest is then predicted as the dot product between these neural network extracted features and the learned weight vector for that species (line 127). To further emphasize the impact of the set of pretrained range estimation models, in *Fig 3 (top right)* we observe that the performance of our approach drops when we reduce the number of species in the candidate set of pretrained range estimation models. To provide additional motivation as to how the linear combination works, in *Fig B3* in the rebuttal PDF we illustrate an example learned combination. We will add additional visualizations with an associated discussion to the final text. \n\n**[yFB1-3] A baseline stronger than the LR_random would be to train a general classifier on joint presence data from all available species.**     \nApologies, if we have misunderstood your request, but we believe that your suggestion is already included in the paper. To clarify, *LR_random* uses the same neural network backbone 256 dimensional feature space (line 228-232) as our model. However, it differs in how active sampling is performed (here random selection) and how the classifier is obtained for a test species (it uses the conventional binary cross entropy training objective). In *Fig 2* we can see that this model does not perform very well. A stronger baseline is the *LR_uncertain* model. In this instance, the location selection is performed using uncertainty sampling. Thus both of these baselines make use of the rich and informative feature space learned from the neural network that has been trained jointly on the large set of species observation data. We will update the descriptions of the baselines in section C4 in the appendix to clarify this. If we have misunderstood your question, please let us know in the discussion phase and we will happily run any necessary additional experiments. \n\n**[yFB1-4] Does the linear combination of weights indicate any correlations among species? If so, is such a correlation related to species taxonomy?**  \nSpecies that have similar learned weight vectors ($\\mathbf{\\theta}$ have similar predicted ranges. To explain a range for a new species, we make the assumption that this can be achieved by estimating a weighted combination of existing species weight vectors (Equation 3). In *Fig B3* in the rebuttal PDF, we illustrate an example of this weighted combination. We will expand on this discussion in the final text. Please see our discussion of taxonomy in the next question. \n\n**[yFB1-5] Would it be useful to use the species taxonomy as an input to the model?**  \nThis is an interesting question. Currently our approach does not encode or require domain specific information such as taxonomic relatedness. However, it is not clear if taxonomic information would be helpful. As a result of divergent evolution, species with common ancestors can evolve and accumulate differences, e.g. a preference for different types of habitats which can manifest as different ranges (e.g. African versus Indian Elephants). As a result, even if two species are taxonomically related, this information may not provide any useful signal for our range estimation task (even though taxonomic information would likely be helpful for image classification). It is possible that other forms of domain knowledge could be encoded into our model, but we leave this for future work. \n\n**[yFB1-6] Could you elaborate on the details of LR_HSS?**   \nAs noted on line 133 in the appendix, for this model, the next location to sample is selected based on the weighted hypothesis set and then the estimated range model is updated using binary cross entropy loss. This results in a single weight vector that can be applied at test time using the logistic model.  We will clarify this description in the final text. \n\nThanks for pointing out the typo, we have fixed it.\n"}, {"review_id": "0HZVbfVNUY", "reviewer": "Reviewer_B3F8", "summary": "The paper explores a method for geographical range estimation of a species.  One of the primary challenges in range estimation is the scarcity and costliness of data collection, particularly for rare species. To address this issue, the paper introduces an active learning framework as a solution. The proposed method offers two key advantages. Firstly, it utilizes a joint feature representation that combines information from multiple species, enhancing the accuracy of the range estimation. Secondly, it employs an active learning scheme to efficiently collect and learn from data to estimate a geographical range of new species. The paper also proposes a linear combination of learned classifiers to develop a new classifier for each new species. The effectiveness of the proposed method is evaluated on two datasets: the IUCN dataset and the S&T dataset.", "questions": "What is the unique machine learning challenge of the range estimation problem that distinguishes it from other machine learning problems or applications?\n\nEquation (3) averages the classifier parameters. How does the model perform if averaging is applied to the output rather than to the parameters? Why did the authors decide to apply the average on parameters rather than output?\n\nEquation (5) uses average over outputs while equation (3) averages over parameters. Why should uncertainty computation (Eq. 5) be different from inference computation (Eq. 3)? Is there an intuition behind this?\n\nWhat happens if the pretrained classifiers are not used, but instead train a new species classifier from scratch on S_t? This is an important baseline but is missing in the paper.\n\nIt is interesting that nn_rand performs way better than the other two features; loc_feats and env_feats. What would be the reason for this? Is nn_rand trained on S_t? Or is nn_rand fixed as the random initial weights?\n", "limitations": "The paper addresses the limitations of the proposed method and its broader impacts. ", "rating": 4, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 2, "strengths": "The paper is written clearly. The description of the problem setup and the technical approach is intuitive, allowing readers to grasp the concepts easily.\n\nThe paper makes a significant contribution to the field of species range estimation. It achieves a notable performance increment and thoroughly explores the problem with a comprehensive analysis.\n\nFurthermore, the paper provides a comprehensive analysis of the proposed method, particularly focusing on the active learning scheme. This includes multiple stages such as initial representation learning, acquisition function, and model training. The authors thoroughly examine and analyze the proposed active learning framework, providing a comprehensive understanding for each components.\n", "weaknesses": "While the paper thoroughly analyzes its proposed method and provides empirical results for the specific problem of range estimation, it does not explicitly highlight the unique machine learning challenges of this problem or how they connect to the technical contribution of the paper. Range estimation can be seen as sharing commonalities with other common machine learning problem settings. For instance, the presence-absence data setting can be considered a supervised classification problem, while the presence-only data setting resembles one-class learning or semi-supervised learning, where the labeled set only contains positive samples.\n\nThe paper primarily focuses on the application aspect by developing an efficient active learning framework for the range estimation problem. However, it does not provide a detailed explanation of why the chosen solution, which employs widely used machine learning techniques, was devised among numerous alternative approaches. The paper would benefit from explicitly stating the unique machine learning challenges posed by the range estimation problem and how the proposed method effectively addresses those challenges. Additionally, a more comprehensive discussion of the motivation behind the chosen approach would enhance the clarity and understanding of the paper.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[B3F8-1] The paper does not explicitly highlight the unique machine learning challenges of this problem or how they connect to the technical contribution of the paper.**   \nAs noted on line 118, when presence and absence data is available, the range estimation problem for a single species can be viewed as a binary classification task. The core machine learning task explored in our work is how to estimate a species’ range while sampling the fewest number of locations (see Sec 3.2). While not the focus of this work, the training of the backbone neural network feature extractor from presence-only data can be viewed as a single positive multi-label problem (see [11]). As noted on line 161, our setting differs from conventional binary classification (e.g. image classification) as the ranges of different species are not necessarily distinct, but can instead have a potentially large amount of overlap. This observation is what makes our proposed solution possible. We will update the text to explicitly state the core machine learning task explored and precisely indicate how our proposed solution addresses it.\n\n**[B3F8-2] A more comprehensive discussion of the motivation behind the chosen approach would enhance the clarity and understanding of the paper.**  \nThe motivation behind our approach is illustrated in *Fig 1*. The main assumption is that given a sufficiently diverse set of candidate species range estimation models it should be possible to describe the range of a previously unseen test species. The intuition is that by leveraging this prior set of possible predictions we are able more rapidly estimate a species range compared to learning from scratch or any of the numerous other baseline we compare to. In *Fig 3 (top right)* we show that as we reduce the number of candidate models available, performance on held-out species decreases. We will update the text to better explain why the chosen solution was devised.  \n\n**[B3F8-3] Eqn. 3 averages the classifier parameters. How does the model perform if averaging is applied to the output rather than to the parameters? Why did the authors decide to apply the average on parameters rather than output?**  \nBy averaging the classifier parameters we obtain a compact 256 dimensional encoding of a species’ global range. Our neural network feature extractor is trained on 44,181 species (line 227). If we instead averaged the output of this model, it would significantly increase the amount of computation required as this scales with the number of species (which could be up to the order of millions of species as opposed to the fixed dimensionality of the network’s features). In addition, we would not have a compact representation of the classifier for the new species once active learning is complete and would instead have to store and evaluate all 44,181 classifiers to estimate the new species’ range. \n\n**[B3F8-4] Eqn. 5 uses average over outputs while eqn. 3 averages over parameters. Why should uncertainty computation (eqn. 5) be different from inference computation (eqn. 3)?**  \nEqn. 3 updates the model parameters based on the agreement with the data currently observed and eqn. 5 selects the next location to be sampled that is most uncertain according to the set of candidate models. The intuition here is that we would like to select geographical locations that allow us to downweight the largest number of irrelevant candidate models at each step. In doing so, we are able to focus in on the candidate models that have the most agreement with the observed data and then efficiently represent the learned classifier as a single weight vector.\n\n**[B3F8-5] What happens if the pretrained classifiers are not used, but instead train a new species classifier from scratch on $S^t$?**   \nTo clarify, the logistic regression baselines in *Fig 2* and *Table A1* are already trained from scratch on $S^t$ for each species. They make use of the pretrained feature space (line 123) but do not use the pretrained candidate models. Unsurprisingly, without the backbone neural network features, these models perform very badly as the raw location or environment features are not very informative (see *Fig B1* in the rebuttal PDF where we train the requested classifiers on S_t directly using different features). We observe similar poor performance for the *WA_HSS+* model in *Fig 3 (top left)* when not using the network’s features. \n\n**[B3F8-6] Why does nn_rand perform much better than loc_feats and env_feats (Fig 3 - top left)?**   \nIn *Fig 3 (top left)*, as in all our experiments, the feature space remains fixed during active learning (i.e. we do not update the neural network) and only the species classifier weights are trained (line 231).  The *nn rand feats* come from a randomly initialized but untrained version of the standard network (line 285). In Fig 3 (top left) we see that these features perform much worse than the trained network features (*nn env feats* or *nn loc feats*). However, this fixed random higher dimensional projection of the input coordinates performs much better than only using the low dimensional location or environmental features (*env feats* or *loc feats*). This is because the low dimensional features are not sufficiently linearly separable, but the higher dimensional fixed random projections are. We will add this discussion to the revised text."}, {"review_id": "pwlY5MK2Fx", "reviewer": "Reviewer_yX8t", "summary": "**Paper Summary**\n\nThis paper tackles the data-driven estimation of geographical habitat area for a given species; that is, given a certain geographic location (i.e., a \"point on the map\"), the goal is to predict how likely a certain species is present in this location. Prior efforts train models in an \"offline\" setting, assuming a finite training data set that indicates (for a fixed set of different species) the presence (or absence) of the species. Datasets that signal the presence (and possibly absence) of a large number of species are readily available. \n\nThis paper points out that dataset expansion for species not covered by existing datasets would be cumbersome- we would need to collect presence indication signals for new (or not yet recorded) species worldwide by possibly uniformly sampling presence indication evidence, which can then, in turn, be used to update/re-train existing models.\n\nThis paper investigates an interesting angle of this general problem: can we leverage *existing* (already collected) datasets to obtain a \"weak\" classifier for a new species, which can then guide the data collection efforts to regions, such that with as few samples we can decrease the overall presence entropy *as much as possible*? \n\nThis is akin to active learning, where the idea is to label as-informative-as-possible recorded data points. Different to that, in this case, the model suggests geographic regions in which we should sample the data for the new species (i.e., obtain presence/absence signals), such that the range estimates for new species will be as accurate as possible with as few data points (i.e., manual collections) as possible. \n\nThe overall idea behind the proposed work is to leverage the similarity between the (new) query species and existing species: this is based on the intuition that \"similar species\" (in terms of similarity in the feature space) will likely habitat similar territories. Such an approach can then be applied iteratively by incorporating newly-obtained presence/absence signals, suggesting increasingly more informative locations for the data collection. \n\nThe idea behind the approach is simple and intuitive: assuming an ensemble of (trained) linear models for a set of species, a model for a new species can be estimated as a weighted combination of existing linear models (where the model weight is proportional to the similarity in the feature space between the new (query) species and species database). This model then provides a \"heatmap\" over geographic locations, and guides the data collection -- new data points can be collected in the areas where the model is most uncertain about the new species' presence. \n\n**Core Contributions/Insights**\n\nThe core contribution of this work is the insight that techniques similar to active learning can be used to streamline the data collection process. This is based on the key insight that existing datasets contain a sufficiently large number of species samples, such that a linear combination of already trained linear classifiers provide a reasonably good predictor for the presence of new species. \n\n**Key Results**\n\nThe paper convincingly demonstrates that the proposed hypothesis ensembling outperforms logistic regression, and adding an online-trained model to the ensemble further improves the performance. I cannot comment on whether any relevant baselines are missing, as I am not actively working in species range estimation or active learning, and I am, therefore not familiar with related work. I hope my colleagues can comment on this. \n\nExperimental validation also discusses the impact of the number of existing species (models) for model averaging on the final performance and establishes that the proposed approach is useful when a large number of specie presence models is readily available. \n\n**Potential for Impact**\n\nSpecie habitat estimation has the potential to assist scientists in monitoring how the habitats of species are changing over time, which is especially important in the era in which global warming presents an existential threat to several species. Therefore, data-driven methods that can provide reasonable estimates based on (I assume difficult to obtain) evidence of species have important practical implications. As monitoring (and consequentially, data collection) is a continual process, developing models that can guide data collection efforts using active learning techniques makes sense. \n\n**Rating Justification**\n\nI think this paper presents an interesting, well-motivated, and well-executed work with direct practical relevance to monitoring species habitats. At this point, I see no obvious objections to why this paper should not be accepted.\n\nThis is a low-confidence review written by a reviewer entirely unfamiliar with bird-species range estimation and only weakly familiar with the field of active learning. \n\n**Post Rebuttal Comments**\n\nI want to thank the authors for their thorough responses. All my questions were addressed, and I see other reviewers are also positive about this paper. I am upgrading my rating to accept. ", "questions": "Did the authors perform any kind of analysis on whether the intuition that most related species are a good indicator for the presence of new species holds? I.e., for a query species, are highly-weighted classifiers indeed representing most related species, and their presence is a good indicator? Did the authors observe any failure cases? For example, the presence of domestic cat, which is related to wild cats, is probably not a good presence indicator for, e.g., lion. ", "limitations": "The paper thoroughly discusses the limitations in a designated section. I find this section insightful. ", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* Application relevance: this work has the potential to guide future data collection efforts for species range estimation. Monitoring species range is important for understanding how the earth's biosphere evolves under rapidly changing environmental conditions. \n* Paper presentation: this paper is very well written and summarizes well both the application domain (species range estimation), as well as existing methods that are commonly being used, together with simple and intuitive taxonomy of prior efforts. Core contributions are presented clearly, well-motivated, and experimentally validated. As someone reading a paper on species range estimation for the first time, I really appreciate Sec. 3, which summarizes how this problem is typically tackled in the literature. \n* Motivation: the paper makes a convincing case for investigating which areas should be used to collect data in the next training cycles using active learning techniques. \n* Results: I find the paper's claims convincingly validated. \n", "weaknesses": "In addition to the weaknesses discussed in the limitation section, it would be fair to point out that the proposed approach is most likely *only* applicable to new species that are sufficiently similar to species for which data is readily available. I assume this approach would fail in the case of potentially newly-discovered species that would hypothetically be quite unlike anything that was observed before. As I am not familiar with the biological aspect of this task, I am not sure if this is a practical concern. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[yX8t-1] Is the proposed approach only applicable to new species that are sufficiently similar to species for which data is readily available?**  \nIf the candidate hypothesis set does not sufficiently cover the space of possible ranges for a held out test species of interest this will indeed impact performance. In *Fig 3 (top right)*, we explore a variant of this setting whereby we reduce the number of species present in the hypothesis set from 44,181 down to 100. As more species are removed, the amount of overlap with the test species is also reduced. However, we observe that our approach is relatively robust to large reductions in the number of species. Given that our full candidate set contains tens of thousands of species, we have good coverage of the types of ranges that might be encountered for a potential held-out species. As indicated in *Fig 3 (top right)*, it is reasonable to assume that as more species are added to the candidate set (e.g. there are several hundred thousand species on the iNaturalist website), performance may improve further. The requirement for a diverse set of species in the candidate set is briefly mentioned on line 328, but we agreed that this is an important point and we will emphasize this constraint in the final text.\n\n**[yX8t-2] Do highly-weighted classifiers indeed represent most related species, and is their presence a good indicator?**  \nThis is an interesting question. In *Fig B2* in the rebuttal PDF we illustrate the similarity between different species from the candidate set. In *Fig B3* in the rebuttal PDF, we illustrate an example of the weighted combination of the classifiers and show how they represent the test range of interest. We will include more visualizations of the weighted combinations in the final text. \n\n**[yX8t-3] Did the authors observe any failure cases? For example, the presence of domestic cat, which is related to wild cats, is probably not a good presence indicator for a lion.**  \nIn *Figs A5* and *A6* in the appendix we observe a histogram of the per-species performance for our approaches at different time steps. We observe that after 50 timesteps, there are only a relatively small number of species where the estimated range is poor. An example failure case is illustrated in *Fig A9* where our WA_HSS approach fails to query observations for the species in Africa. This failure could be explained by the relatively unusual required combination of European and Sub Saharan ranges for this particular species. However, it is worth noting that in the example in *Fig A9*, we only observe the model predictions up until the 8th time step, and it is very plausible that in later timesteps the performance will improve as a result of better samples, and thus a better model weighting, being obtained. \n\nIn our setting, \"relatedness\" is computed based on the similarity between two species in the candidate hypothesis set which are in themselves indicative of range similarity, i.e. two species that have similar ranges will have similar weight vectors $\\mathbf{\\theta}_k$. As a result, taxonomic similarity between two species (as in the cat example provided in the question) is not actually taken into account or modeled explicitly and thus we would not be susceptible to the limitation outlined in the question. However, this is an interesting question and we will clarify the point about taxonomic relatedness in the updated text. \n\nWe also discuss taxonomic relatedness in our response to **yFB1**. As a result of divergent evolution, species with common ancestors can evolve and accumulate differences, e.g. a preference for different types of habitats which can manifest as different ranges (e.g. African versus Indian Elephants). As a result, even if two species are taxonomically related (e.g. a domestic and wild cat) this information may not provide any useful signal for our range estimation task as visual appearance is not taken into account. "}, {"review_id": "hJV2c4FL3r", "reviewer": "Reviewer_RtuW", "summary": "This paper tackles the problem of species distribution modeling from limited observation data. An active learning approach is proposed that makes the assumption that an unmapped species can be represented by a weighted combination of candidate species models (previously trained offline on a disjoint dataset). A sampling strategy is described that uses the aggregate uncertainty of the candidate models to select the next location to query. For representing location, a pre-existing feature representation is derived from a model trained on weakly labeled presence-only data.  Results on two benchmark datasets outperform baselines and an ablation study is presented.", "questions": "My initital rating is a weak accept. This paper is exceptionally well written and was fun to read. The use of active learning for species range modeling has real-world utility. The proposed approach, while simple, has novelty in its construction and the results improve over baselines. I think this paper will be of interest to the community.\n\nSuggestions:\n\n- An interesting byproduct of the proposed approach would be to show which species contributed (and their range models) the most in the weighted average.\n\n- The annotations on Fig. 4 are hard to see without zooming in.", "limitations": "Limitations sufficiently addressed in the manuscript.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- Addresses an important and interesting problem. Species distribution modeling has received a great deal of attention in the literature lately, especially related to fine-grained visual categorization. This work is of the first to explore how active learning learning can be applied to this problem to estimate range models from limited amounts of observation data.\n\n- The proposed approach has technical novelty. The key idea is to take advantage of an existing set of species range models as a hypothesis set. During training, a range model for an unseen species is estimated by a weighted average of the candidate models. To capture fine-details, this candidate set is augmented with a model trained on the accumulated observations of the new species. In addition, a modified form of uncertainty sampling is also used which leverages the candidate models.\n\n- Extensive evaluation, including an ablation study. The results indicate that design decisions directly lead to performance improvements, for example the hypothesis set sampling approach and the inclusion of the online model in the candidate set. \n\n- Well written, easy to follow.\n\n- Sufficient detail provided that the proposed method is likely reproducible.", "weaknesses": "- The proposed approach is a relatively straightforward application of active learning beyond the introduction of the candidate models.\n\n- The approach is highly dependent on the hypothesis set (candidate models) being diverse such that a new species can be represented by some combination of them (L190). To partially address this, the authors add a model trained on the accumulated observations to the hypothesis set. \n\n- The evaluation is largely limited to simple baselines in the form of\n  a self-ablation (incrementally adding to an active learning baseline).  Only one recent paper is compared against (not active learning-based), with diverging results across the two benchmark datasets. That said, it appears there are no obvious other baselines to compare against.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[RtuW-1] The proposed approach is a relatively straightforward application of active learning beyond the introduction of the candidate models.**  \nOur approach consists of a novel exploration of active learning to the problem of efficiently estimating spatially varying quantities when there are informative priors available. It is made possible by the recent availability of large-scale citizen science collected species observation data and efficient neural network-based models for estimating their ranges. Our approach significantly outperforms conventional active learning baselines. We believe that the problem setting is unique, the application area is important, and the task will benefit from future investigation from the machine learning community. \n\n**[RtuW-2] The approach is highly dependent on the candidate hypothesis set being diverse such that a new species can be represented by some combination of them. To partially address this, the authors add a model trained on the accumulated observations to the hypothesis set.**  \nIndeed, the diversity of the hypothesis set is an important part of our approach. In *Fig 3 (top right)* in the main paper we quantify the impact on downstream performance of significantly reducing the size of the hypothesis set. We observe that our approach is robust to large reductions and still outperforms the logistic regression random and uncertainty baselines even with very small numbers of candidate models. As noted in the question, the benefit of the added online trained model is most apparent when the candidate set only contains 1000 models or fewer. Adding this model to the candidate set is trivial and it has no meaningful impact on the amount of additional computation required. \n\n**[RtuW-3] The evaluation is largely limited to simple baselines in the form of a self-ablation (incrementally adding to an active learning baseline). That said, it appears there are no obvious other baselines to compare against.**  \nIn section C4 in the appendix we outline the twelve different baseline methods that we compare against. This does not include different ablated variants of our proposed model where we explore the impact of different feature spaces, i.e. *Fig 3 (top left)*. Given that we are one of the first works to explore the active learning setting of this problem, there are no other obvious baselines to compare against. We hope that our work, and the comprehensive experiments contained within, will serve as a valuable benchmark for this problem going forward. \n\n**[RtuW-4] An interesting byproduct of the proposed approach would be to show which species contributed (and their range models) the most in the weighted average.**  \nThis is a very interesting suggestion - thanks! We show an example of such as visualization in *Fig B3* of the rebuttal PDF. We will add additional improved images of this form to the final text to showcase these combinations across a diverse set of species. \n\n**[RtuW-5] The annotations on Fig. 4 are hard to see without zooming in.**  \nWe will make these larger. Thanks for pointing this out. \n"}]

Author rebuttal:
We thank the reviewers for their constructive comments and insightful questions. We appreciate the detailed reviews and words of support regarding the overall quality and clarity of the paper provided by each of the five reviewers. As there are no significant common concerns from the reviewers, we respond to their comments individually. We also provide three additional experiments/visualizations in the rebuttal PDF in response to specific reviewer comments. Finally, if there are any questions that are still not resolved via our replies, we are very happy to provide additional responses during the author/reviewer discussion period. 

Only relatively minor changes in the text are requested to better motivate the problem setting, provide additional baselines that we have already added to the rebuttal PDF, justify some of the choices made, and draw connections to other related machine learning tasks. These changes are straightforward to implement and will further enhance the overall clarity of the paper. 


Meta-review: This paper considers the problem of learning species distributions from observations, and how best to direct limited ecological monitoring efforts. The authors propose an active learning method to identify high-priority locations for gaining additional data, by using the distributions inferred for some species to feed into a mixture of experts informing the potential distributions of others. While there was some minor disagreement on this paper during review, there was general consensus that the method proposed was a valuable ML contribution in addressing an important problem. The biggest concern raised was in "novelty", but I concur with several of the reviewers in seeing this paper as an effective example of application-guided innovation, where the methodological advances were appropriately motivated by application-specific challenges and criteria for success. Accordingly, I recommend the paper strongly for acceptance.
Decision: Accept (poster)