SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In Figs. A4 to A6 we display additional qualitative results. In each figure, we compare the outputs of different active sampling strategies across different time steps for the same species. In general, we observe that our WA_HSS+ approach results in predictions that are closer to the expert range map (shown in the top left of each figure). However, Fig. A4, illustrates an interesting failure case where our approach fails to predict the species presence outside the region of the initial presence observation. However, from Fig. A11 we can see that the predictions for most species results in higher MAP scores for WA_HSS+.
Figure A4: Predicted range maps for the Yellowhammer for four different active learning strategies, illustrated over three different time steps. We also display the expert range map, inset in the top left. The queried locations are marked with Xs, with green representing present locations and red representing absent ones. In this example WA_HSS+ and WA_HSS fail to identify the most of the range while other sampling strategies are more successful.
C Implementation details
C.1 Dataset details
In Sec. 4.1 in the main paper we noted that the  $S \& T$  dataset does not span the entire globe and is more spatially biased compared to the IUCN dataset. In Fig. A7 we visualize the number of coarse locations that have a species presence for both datasets. We can see that the IUCN dataset on the left is more globally distributed, compared to the  $S \& T$  dataset on the right. In addition, in the case of the  $S \& T$  dataset, it is only possible to evaluate on a subset of possible locations as opposed to the IUCN dataset where we can evaluate on all possible locations on the earth's surface. In Table A2 we summarize the statistics of both datasets.
Table A2: Comparison of dataset statistics. We evaluate our models at a H3 [1] resolution of five, which results in 2,016,842 possible cells distributed across the globe before we exclude ocean locations. The S&T dataset covers a smaller percentage of the globe compared to the IUCN dataset.
IUCN	S&T
Avg. # valid H3 cells	569,094	162,008
Avg. # positive H3 cells	19,614	42,429
Avg. # valid percent of globe	28.22	8.03
Avg. # positive percent of globe	0.97	2.01
Avg. # positive / valid percent	3.45	26.19
Figure A5: Predicted range maps for the Gold Tegu for four different active learning strategies, illustrated over three different time steps. We also display the expert range map centered on South America, inset in the top left. WA_HSS+ quickly identifies the majority of the range.
C.2 Input features
In our experiments we compare different input feature encodings  $\pmb{x}$  to represent locations on the earth's surface. Unless otherwise specified, we use a four dimensional trigonometric encoding of the scaled latitude and longitude (either directly or via a neural network-based feature extractor). This encoding is used to prevent discontinuities at the connected edges of the world map [31]:
$$
\boldsymbol {x} = \left[ \sin \left(\frac {l a t \pi}{9 0}\right), \cos \left(\frac {l a t \pi}{9 0}\right), \sin \left(\frac {l o n \pi}{1 8 0}\right), \cos \left(\frac {l o n \pi}{1 8 0}\right) \right]. \tag {6}
$$
In the results in Fig. 3 (top left) in the main paper we compare to models that use features derived from a neural-network backbone (i.e., those with nn in the name). If a neural network backbone is not specified, the input features are instead passed directly to the classifier. These neural network feature extractors are either randomly initialized using location only features and not trained (i.e., rand), or are trained using the procedure outlined in [13] with only location features (i.e., loc) as input or location and environmental covariates (i.e., env).
Environmental covariates are commonly used for species distribution models as it allows them to attempt to learn the "fundamental ecological niche" of the species of interest [7]. By also including the encoded coordinates we make our models spatially explicit and allow them to distinguish between locations that have similar environmental characteristics but are spatially distant. We use 20 environmental covariates (including elevation) generated from elevation and bioclimatic rasters at a five arc minute spatial resolution, taken from the widely used Worldclim 2.1 dataset [20]. Each covariate is normalized by subtracting the mean and dividing by the standard deviation, and NaN values are replaced with 0. Bilinear interpolation is used to generate corresponding environmental covariates for any given location. Models that take environmental features as input therefore accept 24 dimensional inputs, with four dimensions encoding location information and the remaining 20 representing the bilinearly interpolated environmental covariates at the location.
Figure A6: Predicted range maps for the White Wagtail for four different active learning strategies, illustrated over three different time steps. We also display the expert range map centered on Asia, Africa and Europe, inset in the top left. WA_HSS+ quickly identifies the majority of the range.
Figure A7: Count of the number species presences for each coarse geographic location for the IUCN (left) and S&T (right) datasets. We can observe that the IUCN dataset is much more globally distributed and less North American biased than the S&T dataset.
C.3 Backbone feature extractor
In addition to the input encoding, our method involves extracting features from a deep species range estimation model trained on community collected presence-only data. This network uses the same architecture, training procedure, and hyperparameter settings as the  $\mathcal{L}_{\mathrm{AN - full}}$  model from [13]. We also use the same training data from iNaturalist [5], but we remove classes (i.e., species) that are present in the full set of species (i.e., not just the 1,000 in our subset of the test species) in either of our test datasets. This reduces the number of species to 44,181.
The deep species range estimation model is a fully connected neural network consisting of an input layer, then four residual layers, each comprised of two fully connected layers with skip connections between them, and finally a classification head consisting of a linear classifier per-species, each of which predicts the presence or absence of a single species using a sigmoid output. It can be considered a location encoder  $f$  that feeds into set of single layer classifiers  $\mathcal{H}$  which all share the
same input features. ReLU nonlinearities are used between layers, and each hidden layer consists of 256 neurons. Dropout layers are also utilized. This model is trained for 10 epochs using Adam optimizer with a dropout probability of 0.5 and a batch size of 2048. Training the feature extractor takes 2.5 hours on an NVIDIA RTX A6000 GPU with 48GB of RAM.
After training, the frozen network  $f$  can then be used as a feature extractor, generating 256 dimensional embeddings used for hypothesis set selection and for training new classifiers. Each per-species classifier  $h$  can be used as the set of hypotheses  $\mathcal{H}$  in our WA approaches. As  $f$  is frozen during active sampling and the locations that can be sampled are limited to the centroid of resolution five H3 cells, features can be generated ahead of time by applying the feature extractor to each centroid and storing the results. Thus  $f$  is not required during the active sampling process.
C.4 Active sampling
In each experiment, we apply the chosen active sampling strategy on 500 held-out test species randomly selected from each of the International Union for Conservation of Nature (IUCN) [2] and eBird Status and Trends  $(S\& T)$  [21] datasets. We use the same 500 species from each dataset for each experiment, and use the same initial samples consisting of a single presence and absence observation for each experiment. The species set in each dataset is not the same, e.g., the  $S\& T$  dataset contains only birds. Each experiment is repeated three times with different seeds and initial samples. The same frozen feature extractor  $f$  is used across all experiments except 'Impact of Feature Space' experiments in Fig. 3 in the main paper and the additional ablations in this appendix.
We evaluate our performance using per-time step mean average precision (MAP), and using area under the MAP curve (MAP-AUC). As we synthetically generate data from the underlying expert range maps provided, we do not have a specified train test split and merely evaluate at each valid H3 cell centroid and measure performance as the difference between a model's predictions and the expert-derived range map. Unlike [13], we evaluate our models only on land and not also over the ocean.
The logistic regression range estimation model  $h$  we optimize during active sampling is implemented using scikit-learn's linear_model. LogisticRegression class with default settings [33]. If the features from  $f$  have been stored ahead of time, then each experiment can be performed without a GPU as it consists only of training and evaluating a logistic regression model. A single experiment using our WA_HSS+ method involving 500 species across 50 time steps takes 4 hours using unoptimized code on an AMD EPYC 7513 32-Core Processor.
C.5 Baseline strategies
Here we provide additional information about the baseline active sampling strategies we evaluated. In all cases, the set of possible locations from which to select a sample  $\boldsymbol{x}^*$  consists of all valid resolution five H3 cell centroids (i.e., those that are located over land and have not yet been sampled, and for the S&T dataset only those where the ground truth is provided). The LR strategies below update the range estimation model at each time step by minimizing the cross entropy loss in Eqn. 1 from the main paper.
LR_random.  $x^{*}$  is selected randomly from all valid H3 cell centroids.
LR Uncertain.  $x^{*}$  is determined by Eqn. 2, and corresponds to the location where the model's probability of presence is closest to 0.5. This is one of the most well established and commonly used methods of active sampling for classification.
LR_EMC.  $x^{*}$  is selected to maximize the absolute gradient of the model parameters  $\theta$  with respect to the loss function, taking an expectation over the current distribution of class labels. We are looking for the sample that will result in the largest expected model change. The intuition is that we expect this location will cause our model to "learn" the most [10].
LR_HSS.  $x^{*}$  is selected according to Eqn. 5 in the main paper, where the weights of the hypothesis set  $\mathcal{H}$  are determined by agreement between the hypotheses and the sampled data, and the location that the weighted hypotheses are most uncertain about is then selected. However unlike WA_HSS,  $h$  is not updated by averaging these hypotheses, but by updating the weights for a logistic regression classifier to minimize the cross entropy loss.
LR_QBC. This 'query by committee' method [43; 24] is similar to LR_HSS above but  $\mathcal{H}$  is now a set of classifiers trained on subsets of the currently available data  $S^t$  as in [6]. Due to the small size of  $S^t$ , especially in early time steps, subsets are created by removing a single data point from  $S^t$ , and then discarding subsets that only contain points from a single class.
LR_positive.  $x^{*}$  is selected as the location that has the highest probability of presence according to  $h$ . This querying method has been used within the ecology literature as a method for evaluating generated species distribution models or as a 'guided search' method [22; 23].
We also compare our WA_HSS+ strategy to others that use weighted averaging of  $\mathcal{H}$  to update  $h$ , but do not use hypothesis set selection to determine  $x^{*}$ . The following WA strategies can also be modified by the inclusion of  $h_{\mathrm{online}}$ , leading to WA_random+, WA Uncertain+, and WA Positive+ baselines.
WA_random.  $x^{*}$  is selected randomly from all valid H3 cell centroids.
WA Uncertain.  $x^{*}$  is determined by Eqn. 2 in the main paper, and corresponds to the location where the model's probability of presence is closest to 0.5.
WA_positive.  $x^{*}$  is the location that has the highest probability of presence according to  $h$ .
D Additional ablations
Here we provide the results of additional ablation experiments.
D.1 Impact of label noise
In Fig. A8 we demonstrate the impact of adding label noise for WA_HSS+ compared to WA_HSS. This expands on the results in Fig. 3 (bottom left) in the main paper. While WA_HSS+ outperforms WA_HSS in the presence of modest label noise, as more noise is added, the WA_HSS strategy performs slightly better than WA_HSS+.
Figure A8: Impact of adding increasing amounts of label noise on the IUCN dataset. Our WA_HSS+ and WA_HSS approaches are robust to label noise compared to conventional logistic regression-based uncertainty sampling used in LR Uncertain and seen in Fig. 3 (bottom left) in the main paper.
D.2 Impact of hypothesis set size
In Fig. A9 (left) we illustrate the impact of removing species from the training set when training the backbone feature extractor. We compare the performance of WA_HSS using different backbone models trained with a limited number of classes to WA_HSS where we remove classifiers from  $\mathcal{H}$  but keep the same feature extractor trained on all classes. Fig. 3 (top right) in the main paper shows a large impact from using backbones trained with fewer classes. Here we observe that in almost all cases the model using a backbone trained on all classes performs best, and the performance difference increases when more classes are removed.
In Fig. A9 (right) we further show how using backbones trained on fewer classes impacts the baseline active sampling strategy. The change in performance for the LRuncertain strategy underlines that jointly modeling many species in the backbone is vital for good performance due to the features
extracted becoming more useful for range estimation, even when the predicted species ranges are not directly used, as they are in WA_HSS.
Figure A9: (Left) Impact of removing species from the backbone on the IUCN dataset. Here we compare models trained without species in the neural network backbone and hypothesis set (solid line) to models that use all the species to train the backbone but simply remove some from the hypothesis set (dashed lines). The solid lines are the same as the solid entries in Fig. 3 (top right) in the main paper. (Right) Impact of changing the hypothesis set size on the IUCN dataset for the LR Uncertain strategy. When many fewer species are used to train the backbone the performance is worse.
D.3 Impact of feature space
In Fig. A10 we explore the impact of using learned input features for our WA_HSS+ method and the logistic regression baseline LR Uncertain. The feature spaces are a subset of those shown in Fig. 3 (top left) from the main paper. We observe that using the features generated by the deep feature encoder (nn loc feats) is vital for good performance for both the LRuncertain and WA_HSS+ strategies. In contrast, only using location (loc feats) or location and environmental features (env feats) directly with no neural network results in poor performance.
Figure A10: Impact of using different feature spaces evaluated on the IUCN dataset. Both methods require the use of the deep feature encoder features, nn loc feats, for good performance.
D.4 Per-species performance
In Figs. A11 to A14 we display histograms of the per-species performance across six different time steps (2, 5, 10, 20, 30, and 50) on the IUCN dataset. Each individual histogram displays the distribution of MAP scores for each of the species in the IUCN dataset. In general, we observe a trend that performance improves over time, i.e., the mass of the histograms move the right. Our WA_HSS+ approach in Fig. A11 is noticeably better than the naive random selection using logistic regression baseline in Fig. A14.
Figure A11: Histograms of per species MAP scores for our WA_HSS+ approach across six different time steps on the IUCN dataset.
Figure A12: Histograms of per species MAP scores for our WA_HSS approach across six different time steps on the IUCN dataset.
Figure A13: Histograms of per species MAP scores for LR Uncertain (i.e., logistic regression with uncertainty-based sample selection) across six different time steps on the IUCN dataset.
Figure A14: Histograms of per species MAP scores for the LR_random baseline (i.e., logistic regression with random sample selection) across six different time steps on the IUCN dataset.