BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning
Authors: Jianyang Gu
Samuel Stevens
Elizabeth G Campolongo
Matthew J Thompson
Net Zhang
Jiaman Wu
Andrei Kopanev
Zheda Mai
Alexander E. White
James Balhoff
Wasila Dahdul
Daniel Rubenstein
Hilmar Lapp
Tanya Berger-Wolf
Wei-Lun Chao
Yu Su
Venue: NeurIPS 2025 spotlight ()
Keywords: Biology Foundation Models
Multi-modal Representation Learning
Primary area: machine_learning_for_sciences

Abstract: Foundation models trained at scale exhibit remarkable emergent behaviors, learning new capabilities beyond their initial training objectives. We find such emergent behaviors in biological vision models via large-scale contrastive vision-language training. To achieve this, we first curate TreeOfLife-200M, comprising 214 million images of living organisms, the largest and most diverse biological organism image dataset to date. We then train BioCLIP 2 on TreeOfLife-200M to distinguish different species. Despite the narrow training objective, BioCLIP 2 yields extraordinary accuracy when applied to various biological visual tasks such as habitat classification and trait prediction. We identify emergent properties in the learned embedding space of BioCLIP 2. At the inter-species level, the embedding distribution of different species aligns closely with functional and ecological meanings (e.g., beak sizes and habitats). At the intra-species level, instead of being diminished, the intra-species variations (e.g., life stages and sexes) are preserved and better separated in subspaces orthogonal to inter-species distinctions. We provide formal proof and analyses to explain why hierarchical supervision and contrastive objectives encourage these emergent properties. Crucially, our results reveal that these properties become increasingly significant with larger-scale training data, leading to a biologically meaningful embedding space.

BIOCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning
Jianyang Gu Samuel Stevens', Elizabeth Campolongo / Matthew J Thompson', Net Zhang', Jiaman Wu', Andrei Kopanev', Zheda Mai', Alexander E. White",
James Balhoff', Wasila Dahdul', Daniel Rubenstein Hilmar Lapp",
Tanya Berger-Wolf' Wei-Lun Chao', Yu Su The Ohio State University, Smithsoniin Institution, UNC Chapel Hill, University of California, Irvine, Princetto University, Duke University
Beak size
Juvenile
Juvenile
Male
Female
Species
span
Adult
Adult
Female
Male
Figure 1: While BIoCLIP 2 is trained to distinguish species, it demonstrates emergent properties beyond the initial training objective. Left: At the inter-species level, the embedding distribution of different species aligns with ecological relationships; the embeddings Darwin's finches arrange themselves by beak size from left right. Right: Instead of collapsing, the sera-species variations are preserved in subspaces orthogonal to the inter-species variation (the black lines point from the mean embedding of one variant to that of the other variant). Orthogonality increases with scale (see Figure 3c).
Abstract
Foundation models trained scale exhibit remarkable emergent behaviors, learning new capabilities beyond their initial training objectives. We find such emergent behaviors biological vision models via large-scale contrastive languae training. To achieve this, we first curate TREEOFLIFE- 200M, comprising 214 million images living organisms, the largest and most diverse biological organ- ism image dataset to date. We then train BIOCLIP on TREFOFLIFFE-20M to distinguish different species. Despite the narrow training objective, BIOCLIP 2 yields extraordinary accuracy when applied to various biological visual tasks such as habitat classification and trait prediction. We identify emergent properties in the learned embedding space of BIOCLIP 2. At the inter-species level, the embedding distribution of different species aligns closely with functional and eco- logical meanings (e.g., beak sizes and habitats). At the intra-species level, instead being diminished, the intra-species variations (e.g., life stages and sexes) are preserved and better separated in subspaces orthogonal inter-species distinctions. We provide formal proof and analyses to explain why hierarchical supervision and contrastive objectives encourage these emergent properties. Crucially, results reveal that these properties become increasingly significant with larger-scale training data, leading biologically meaningful embedding space.
Models, data, and code available at mmngcomics.iituubiibbiiilo (gu. su. su.809 (09. @osu. edu
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Recent advances in artificial intelligence (AI) are transforming core scientific workflows become more efficient and automated [1. Tasks that once demanded overwhelming time and labor, like inferring atomic protein folds, designing functional materials, or producing global weather forecasts, can now be efficiently accomplished by large-scale predictive models [3, 4, 5, 6, 7, 8J. Particularly, growing class of domain-specific foundation models demonstrate capabilities that arise without explicit definition during training [9, 10J. For example, language models trained purely on large-scale amino-acid strings unexpectedly develop an understanding of 3D chemistry predict atomic folds with experimentall accuracy [4, 1 11]. These scale- driieeddinnnn emergent abilities are reshaping scientific inference and opening new avenues for data-centric discovery.
In ecology and evolutionary biology, previous efforts leveraged hierarchical taxonomic labels and CLIP-style contrastive training [12] to achieve pronounced species classification accuracy across the tree of life [13, 14, 15]. This work asks a simple but intriguing question: what properties emerge if we scale up hierarchical contrastive training? To answer this question, curate TREEOFLIFEE200M, comprising 214M organism images spanning 952K taxonomic classes, making it the largest and most diverse visual catalog of life to date. Through training scale, our model BIOCLIP improves species classification accuracy by 18.0% over BIOCLIP [13]. More importantly. we explore whether representations learned solely through species-level supervision can generalize diverse biological questions beyond species classification.
To probe these capabilities, evaluate BIOCLIP 2 variety of existing biological visual tasks, including habitat classification [16], trait prediction [17, 18), new-species identification [19], and agricultural disease detection [20]. These applications push beyond simple species recognition and apply directly to biodiversity conservation, trait organization, and agricultural health. Despite being trained primarily with species-level supervision. BIOCLIP 2 outperforms both vision-language (e.g., SigLIP [21J) and vision-only baselines (e.g.. DINOv3 [22]) by an average margin of 10.3% on these tasks. We then look deeper into the embedding space of BIOCLIP 2 and identify two emergent properties as the training scales up.
At the inter-species level, the embedding distribution of different species aligns with their ecological relationships. As shown on the left side of Figure 1, BIOCLIP 2 embeddings of Darwin's 's finches demonstrate an increasing beak size from left to right, which is not observed in the original CLIP embedding space. We attribute the property to the adopted hierarchical taxonomic labels, which inherently encode functional and ecological information [23]. The hierarchical supervision at scale pushes related species to co-locate in functionally coherent "macro-clusters" " In such a way, BIOCLIP 2 acquires functional trait knowledge without using explicitly labeled traits.
At the intra-species level, contrary to the intuition that fine-grained differences collapse after extensive training [24, 25], BIoCLIP 2 keeps the intra-species variations (e.g., life stages and sexes) distinct. On the right side of Figure 1, three species form tight clusters when projected onto the "species plane," while their intra-species variations fan out along axes orthogonal to the plane. Such variation cues are not encoded in taxonomic labels. We theoretically prove that when species prototypes nearly orthogonal (i..., species are well separated), the contrastive objective prioritizes orthogonality between intra-species variations and inter-species differences over raw magnitude (Theorem 5.1). Furthermore, these variations are observed to be increasingly separable as training data scales up. This microstructure preserves the intra-species representational diversity without interfering with inter-species distinctions, enabling various attribute recognition applications ($5.
We show in $5.2 through quantitative and qualitative analyses that larger-scale training improves both species ecological alignment and separation of intra-species variants. These scale-amplife patterns make the embedding space more interpretable and biologically meaningful. B1OCLIP evidences that combining domain-specifer scaling with structured supervision can unlock qualitatively new emergent behaviors in scientific vision models.
2 Related Work
Emergent properties in foundation models. Emergent properties refer to the capabilitiess implicitly acquired from the training process and generalized beyond the initial training objective. Large language models (LLMs) illustrate variety of in-context learning skills after next-token mexd-tooen-prediction
2

---

Number of Images
Number of Taxa
Cephalopoda
214M
952K
162M
454K
366K
10.4M
ToI--1MM
BioTrove
TaL-200M
ToL-10M
BioTrove
Tol-200M
Tol-200M
Tol-10M
(b)
Figure 2: (a) Number of images across organismal biology datasets. (b) Biodiversity comparison across datasets (measured unique 7-tples for TREEOFLIFE (ToL) datasets, species count provided by BroTrove). (c) The taxa distributional difference the Cephalopoda class (octopuses, squids, etc.) between ToL-200M TOL-10M.
pre-training [9, 26]. Such emergence also arises in computer vision [27]. DINO learns semantic segmentation through purely visual self-supervision [28], whereas Group learns semantic segmen- tation solely through text supervision [29]. The studies closest this work are [30, 31]. Alper and Averbuch-Elor explore the visual-semantic hierarchies in CLIP models [30]. Although hierarchical semantic structures are never explicitly presented the training supervision, CLIP models acquire the capability of matching images with varying levels of descriptions. Abbasi el al. discover that CLIP models possess disentangled representational sub-spaces for different factors of variations [31]. It allows CLIP models to generalize across compositional out-of of-distribution concepts. This work leverages CLIP to distinguish different species with hierarchical labels, which is different scenario from the above work. We investigate the emergent properties under this setting.
Computer vision for ecology & evolutionary biology. Ecology and evolutionary biology naturally challenging for computer vision systems due to long-tail distributions, extremely fine- grained classifications, and wide variety of image distributions. Existing work formalizes these challenges into specific visual tasks such as attribute prediction [NeWT, 18], trait prediction [FishNet, 16], and plant disease detection [PlantDoc, 20].
Recent advancements computer vision have led to the development of foundation models for biological applications. BIOCLIP incorporates taxonomic labels in the vision-language contrastive training, yielding promising species classification accuracy [13]. Follow-up work scaled data to 162M images [BioTrove, 14], specialized the data camera traps [CATALOG and WildCLIP, 32, 33], and added additional model modalities [TaxaBiind 34). We investigate both data and model scaling, with a focus on both broad biological applications and any emergent properties after extensive training.
3 TREEOFLIFE-2000
Large-scale, clean, diverse data drives progress in machine learning. There have been efforts such as TREEOFLIFE-II 10M [35] and BioTrove [14] to create large-scale biological organismal datasets for machine learning (ML). As shown in Figure 2, TREEOFLIFE- -IFF--IMM [35] improves on prior work such as iNat21 [36] and BIOSCAN- M [37] by increasing taxa diversity by a factor of 45. BioTrove [14] increases data scale to 162M but fails to match the biodiversity of TREEOFLIFE- 10M. In this work, we combine the vast breadth of Global Biodiversity Information Facility (GBIF) [38] images
with those of the Encyclopedia of Life project (EOL) [39], BIOSCAN-5M [40], and FathomNet Database [41]. With nearly 214 million images representing 952,257 taxa, TREEOFLIFE- the largest and most diverse public MI-ready dataset for computer vision models in biology.
Unlike BioTrove [14], which relies solely on iNaturalist and contains 162M images but only 366K unique species, our use of museum, camera-trap, and citizen-science contributions expands the taxonomic coverage to 2.6x more taxa. Our curation efforts ensure this breadth does not come the cost of data quality. We quantify image type diversity from GBIF: 51 8M museum specimen, 617.8K camera trap, and 151M citizen science images. Beyond the taxa- taxa-wiss wise diversity, these images also provide more observing perspectives for the focal species. In such a way, the robustness of models trained on TRREEFFLIFE-2001 we demonstrate that BIOCLIP 2 yields a 22.8% performance gap compared with BIOCLIP on camera trap images (See Table 1). We provide detailed statistics of the image number and taxa diversity from each data provider in Figure 11.
significantly enhanced against a variety use cases. Specifically,
3

---

3.1 Images
TREEOFLIFE-20OM consists of images curated from four core data providers: GBIF, EOL, BIOSCAN-5M, and FathomNet. GBIF and EOL aggregate biological data from various sources, such as INaturalist [42], the Smithsonian Institution [43], and Flickr [44]. BIOSCAN-5M and FathomNet curated collections of expert-annotated images designed to improve cataloging and identification of species from highly diverse and under-represented branches the tree of life. BIOSCAN-SM is part of the ongoing DNA Barcoding project improve insect identification (one of the most diverse classes, Insecta). FathomNet focuses on a habitat rather than a clade: all animals that live in the ocean. Together, these comprise TREEOFLIFE-200M.
3.2 Data Curation & Filtering
We retrieve data from data providers using distributed downdoddeed [45]. The initial retrieval gives 222,065.144 images and 1,359,405 unique taxonomic hierarchies. We then cleaned the data, focusing on (1) aligning taxonomic labels, (2) image quality, and (3) climinating duplication and data leakage. We summarize these efforts here, with more details provided in Ş1.
distribbatedddommloade
Taxonomic alignment. Curating large biological image collections from distributed data providers requires the alignment of noisy and inconsistent taxonomic labels both between and within providers. We develop a taxonomic alignment package TaxonoPy in consultation with taxonomists that resolves entries to both seven-rank Linnacan hierarchy and a common name. TaxonoPy queries GNVeri- fier [46] against the GBIF Backbone, Catalogue of Life, and OpenTree hierarchies (in that order). From the original 1.36M taxa, TaxonoPy filters 407K taxonomic hierarchies (such synonyms, provisional names, etc.) and yields 952K unique taxa.
Image-quality screens. Digital archives contain herbarium labels, empty camera-trap frames, and occasional people. None add biological signal, and faces raise privacy concerns, so we drop them to keep learning focused on organisms via three neural-network-bbre
filters: (i) Museum non- organism removal. A pre-trained CLIP-L/14 [12] is used for a nearest-centroic classifier spanning 25 fine-grained subtypes (10 collection areas, each split into categories such specimen, fossil, drawer-label, etc.). The classifier is fit to 8.5K mamually
examples and predicts a subtype for all museum images; we drop all non-organismal images. (ii) Camera-trap trimming. We apply pre- trained camera-trap model MegalDetector [47, 48) to filter for frames with visible animals. (iii) Face removal. We apply pre-trained face-detection model MTCNN [49] discard images containing human faces. We release the code used in processing the images in TreeOfLife-toor
Duplicate and leakage control. Exact duplicates in the training set removed with MD5 hashes. GBIF includes images from iNaturalist, popular source for computer vision ecology benchmarks. To prevent train-test leakage and inflated downstream scores, we compute both MD5 and perceptual PDQ [50] hashes for every test image and purge any near or exact duplicates from training.
3.3 Taxa Coverage
The International Union for Conservation of Nature (IUCN) estimates 2.14M species have been described [51]. Following curation, there are nearly 868K unique taxa labeled to the level of species in TREEOFLIFF-22OM.. Based on the most recent IUCN Red List assessment [52], TREEOFLIFE- 200M demonstrates a particularly strong representation of threatened species, with 77.1% coverage (36,370 species). This coverage establishes that the approach to integrating diverse data sources used in LIFEOOLLIFE-2OM substantial majority of species prioritized for global conservation action.
a valuable resource for conservation research, providing representation for
Notably, TREEOFLIFE-200M adds diverse clades that are extremely under-represented in prior work like TREEOFLIFE- 10M and BioTrove. Figure 2c compares the distribution of taxonomic names in the class of Cephalopoda between TIEEOFLIFE- 10M and TREEOFLIFE-22 200M. While they share overlaps, there are clades almost completely absent in TREEOFLIFE- 10M. These under-represented clades receive substantial influx of samples in TREEOFLIFE-200M (1,102 new taxa). Another
Not all images contain full 7-rank Linnaean taxa; for instance, 93% of BIOSCAN 5M images are not labeled to the species level. Thus, the unique taxa with non-null species is more appropriate comparison.

---

Table 1: Zero-, one-, and five-shot species classification top-1 accuracy across 10 tasks different models. Bold and underlined entries indicate the best and second best accuracies, respectively. BIOCLIP 2 outperforms both strong general- (CLIP, SigLIP, DINOv3) and domain- (BIOCLIP, BioTrove-CLL basclines. "Camera Trap" mean performance across 5 camera trap datascts; Appendix F.4 contains more details.
Animals
Plants & Fungi
Camem Trap
Plant Village
Rure Speciies
NABards
Plankium
Invects 2
PlantNer
Ineess
Fungl
Model
Random Guessing
0.2
1.2
1.0
1.0
3.5
4.0
4.0
2.6
4.0
0.3
2.2
Zeno-Shot Classification
CLIP (VII-L/14)
06.5
1.3
9.0
11.7
29.5
61.7
7.6
6.5
25.6
35.2
25.5
SigLIP
2.4
20.7
36.9
BioTrroee CLIP
39.4
1.0
20.5
15.7
11.0
64.4
38.2
15.7
31.6
24.6
26.2
B1OCLIP
58.8
34.9
20.5
40.9
19.0
37.6
B1OCLIP
74.9
3.0
55.3
27.7
53.8
96.8
83.8
25.1
57.8
76.8
55.6
Sho- Classification
CLIP (VIT-1/14)
42.7
28.0 28.0..o6
20.010.4
17.0
36.0
58.7
20. Tina
56.7
74.4
34.3
39.8
SigLIP
39.920.9
2
32.3
20.621.8
37.84a.
66.3
72..
64.1aa.
81.7=2.
38.8
43.9
Supervised-IN2IK
13.8+0.8
23.5+12
15,241.1
18.2+1.6
30.6+2.7
63.8+4.
26. 16.4
52.8
75.2=4.4
31.6e0.
38.1
DINOv3
48.3
36.5.0s
8.8
18.8
43.0
66.8
27.5
64.8
92.1-22
41.5a0.
44.8
BiaTrove CLIIP
61.920.6
26.4
67.1
20.9
31.21za
09.7.8.4
47.3+2.1
55.8144
83.5_1.1
34.9
48.9
BIOCLIP
67.4212
2
67.141.
20.410a
35.0ga.
67.72a.
44.622.
50.5g9.5
83.7
44.9
50.0
BIOCLIP
82.4
22.0.o
74.610.
28.41o7
48.1
85.8
70.3
67.611.1
92.0
59.5_o.
64.1
Sive-Shor Classification
CLIP (VI--144)
6
48.2g1
60.6
30.
53.91aa
76.9.1a
31.41aa
78.3
02.6_ao7
53.3
58.2
SigLIP
64.2g0.3
47.4
54.9
35.2
56.9
81.6214
45.5g16
21.1
94.1-0.7
57.820.
Supervise-IN2IK
57.5+0.4
40.1+0.6
30.1+0.
30.3402
48.3+2a
77.2+14
39.6+1.
78.0e1.1
92.8-0.9
48.8+6.4
5133
DINOv3
72.3:0.4
57.8
19.7aa.7
34.2gus
63.6,29
83.3s14
44.1g1a
8244..
98.8
62.6.0.a
61.9
78.5+002
44.6.0
77.01s
34.21a8
47.912a
86.0.1.
65.2+08
75.1
98.2.0.
51.3100
65.6
BIOCLIP
78.2:0.3
49.2.1.
78.0
33.9
54.3+22
85.7+1.
61.6119
81.7.1.1
96.7=00
65.7
68.5
BIOCLIP
92.41o.
50.5
89.31o4
44.3111
67.7
94.41ns
85.0114
$3.91as
98.4
77.2
example is 55,085 taxa of Fungi in TREEOFLIFE-20OM, close to 4x of that in TREEOFLIFE- 10M 793, The improved diversity facilitates accurate species classification of these clades, as evidenced in Table (42.9% absolute improvement over BIOCLIP zero-shot Fungi benchmark).
BIOCLIP 2 and Species Classification
BIOCLIP adopts a hierarchical multi-modal contrastive training framework, where images are asso- ciated with their corresponding hierarchical labels including taxonomic labels, scientific names, and common names [13]. Different from one-hot labels, taxonomic labels inherently encode hierarchical biological information from different levels [15]. In combination with auto-regressive text encoder, BIOCLIP yielded superior species classification performance on both zero- and few-shot settings. In this work, we stick with the hierarchical contrastive training recipe and focus the impact of scale.
Modifications. addition to the significantly larger and more diverse dataset, we also scale model capacity by adopting a larger vision transformer (VIT-L/IA L/4 pre-trained on LAION-2B [12, 53, 54]). An auxiliary replay mechanism introduced [55, 56] to maintain general-domain -ommain understanding for broader applications [33, 57]; a portion of CLIP training data (LAION-2B) is interleaved simultane- ously with species contrastive learning. We ablate this decision and find that the experience replay improves biological understanding and performance across diverse tasks in $6.
train BIOCLIP 2 on 32 NVIDIA H100 GPUs for 10 days on 21 4M organismal biology images with hierarchical labels and 26M randomly-sample image-text pairs from LAION-2B for 30 epochs. provide the training details in $D.
4.1 Species Classification Performance
We evaluate BIOCLIP on species classification tasks Table 1. We use the same benchmarks as BioCLIP [13], including seven tasks from Meta-Album [58] and Rare Species [59]. We substitute
5

---

Table 2: Biological visual tasks beyond species classification. Bold underlined entries indicate best
second best accuracies. See $11 for task and evaluation methodology details.
Animals
Plants
FishNet
Herh. 19
PlantDee
AwA2
Model
Mean
CLIP (VIT-L/14)
27.9 27.9+0.2
83.4+0.1
61.6+0.6
18.2+0.1
22.3+
42.7
SigLIP
31.940.1
83.240.1
67.310.c
18.610.2
28.2
45.8
Supervised-NN21K
29.440.1
75.8+0.2
52. 22.741.6
14.9
25.1+111
39.6
DINOv3
37.9
85.740.0
48.0
31.240.2
40.3
48.6
BioTrove-CLIP
22.1+0.0
82.5+0.1
45.7+0.7
20.4+0.2
37.7+1.2
41.7
BIOCLIP
30.110.2
82.71o.1
65.91o.3
26.810.4
39.5
49.0
BIOCLIP
39.8
89.1
69.5
48.620.6
40.4
57.5
Sex-FDR
Sex
Fisher Diecrion toe foont lartio (PDR)
Life sDge-DDR .ee
Life sagepp
Accurncy
Accurscy
63
NeWT
FishNett
Life stage Def
0.2
AwAA
Herbarium19
SexsAlign
Plun:Do
Life sisge-Allgg
2304
TM
seM
20/8
Number training samples
Number of training samples
Number traimingg samples
(a)
(b)
(c)
Figure 3: (a) The model performance on five downstream tasks under different scales of training data. (b) The model performance on differentiating and aligning different life stages and sexes. (c) The separation and orthogonality evaluation of models trained with different amounts of data.
Birds-525 [60] with NABirds [61] due to data inaccessibility. Additionally, we collect of IDLE-OO Camera Traps from the Labeled Information Library of Alexandria: Biology and Conservation (LILA-BC) [62, 63, 64, 65, 66, 67] illustrate more realistic species classification applications in the wild. The results suggest substantial improvements of BIOCLIP over BIOCLIP, Particularly, attributed to more comprehensive species and image type coverage of TREEOFLIFE- 200M, we observe over 20% zero-shot improvement on Camera Trap, Fungi and Rare Species. On average, BIOCLIP surpasses the second-best model by 16.1% and provides 30.1% improvement over the original CLIP model that serves as weight initialization. Information on baselines is in $E.
Emergent Properties from Scaling Hierarchical Contrastive Learning
5.1 Beyond Species Classification
Biology's 's organization extends beyond species taxonomies; if scaling truly induces emergent behavior, model representations learned through species-level supervision should transfer to problems far removed from species classification. We collect and benchmark models on five visual benchmarks that push past species ID: habitat classification (ccological context) [16], trait prediction (cvolutionary studies) [18, 17]. new-species identification (biodiversity monitoring) [19], and agricultural disease detection [20]. For each task, we keep the evaluated models frozen and extract the corresponding sample embeddings. The embeddings are subsequently processed using machine-learning techniques (e.g., support vector classifiers). Detailed evaluation procedures are listed in SH.
Table 2 presents the performance comparison among BIOCLIP 2, lassuanlanguage baselines, and vision-only models. Although no information on these tasks explicitly described during training, BIOCLIP 2 yields an average performance improvement of 14.8% over the original CLIP baseline. DINOv3 is commonly believed to capture fine-grained visual features and is adopted for diverse visual tasks [22, 68, 69]. Nevertheless, BIOCLIP 2 yields an 8.9% performance gap over DINOv3.

---

Training Samples
(CLIP)
1M
10M 50M
214M
2D Distribution
Freshwater
Non-Freshwater
Figure 4: t-SNE emhedding visualization of FishNet test for models trained with different amounts data. The leftmost plot is the original LAION-2B CLIP ViT-L/14, As the training data scales, freshwater fish become more distinct from saltwater fish and brackish fish, despite no explicit supervision, demonstrating that data scale contributes to emergent properties in model representations.
5.2 Scaling Trends
Better species classification with more species-labeled data is expected, but its effect on other, non- species classification tasks is unexpected. To better understand the relationship between training data scale and non-species classification performance, we apply the same hierarchical training with varying sizes of data: 1M. 10M, 50M, and 214M samples. The smaller datasets randomly sampled from the complete set without losing taxa representativeness. We compare the performance of baseline CLIP ViT-L/14 and the four obtained models on five non-species classification tasks in Figure 3a. A consistent improvement observed as the volume of training data increases from 1M to 214M. In AwA2, for example, the 1M model is worse than the baseline. However, the model gradually learns more generalized representations for different attributes across species and obtains improved performance as data scales up.
Next, we investigate how scale affects representations within species. We collect two groups of images with intra-species appearance variations: life stage variations from NeWT [18] and sex variations from NABirds [6]]. We ask whether scaling hierarchical contrastive training collapses all images of one species onto single prototype or still distinguishes juveniles from adults and males from females. We accordingly design two complementary tasks each type of variation: (i) alignment, where species classifier trained recognize the species on the other variant (e.g., adult images), and (ii) differentiation, where the task to tell the variants apart (e.g., juvenile vs. adult). As illustrated in Figure 3b, data scale steadily improves cross- variant species recognition. But at the same time, the model also becomes better at distinguishing the variants themselves.
variant (e.g.. juvenile images) is expected to
5.3 Emergent Properties and Qualitative Analysis
Why does scaling data boost tasks that are never supervised during hierarchical contrastive training? We look deeper into BIOCLIP 2's embedding space and identify two emergent properties that generalize beyond species classification.
First, the embedding distribution of different species aligns with their ecological and functional relationships. Figure shows t-SNE plots [70] of FishNet test set embeddings four training scales, colored by whether the fish can live in freshwater or not. In the baseline CLIP plot (left), freshwater and non-freshwater fish have a large portion of overlap. Larger training sets progressively separate two groups in the embedding space. We note that there is no explicit constraint to arrange meaningful distribution across species in contrastive loss, highlighting the emergence at scale.
Second, the intra-species variations are preserved and separated. Figure 5 shows t-SNE plots of BIOCLIP 2 embeddings of three species from NeWT exhibiting life-stage variation. In the 2D plots (top), different species (shown in different colors) tighten progressively from left to right, which is a direct consequence of scaled contrastive training. At the same time, the intra-species variations are preserved and better separated than the baseline CLIP model (leftmost sub- figure).. We further project the embeddings onto 3D spaces created by singular value decomposition (SVD, bottom), which reveals that the intra-species variations lie in subspaces roughly orthogonal to the species span. Therefore, the existence of intra-species variation does not interfere with inter-species distinctions. SF contains more empirical observations.
7

---

Training Samples
(CLIP)
1M
10M
50M 214M
2D Distribution
3D Distribution
p=0.33 ..0 Juvenile Images
p= Adult Images
p=0.26
0=0.17
p=0.14
Figure 5: The embedding distribution of life stage variations under different scales of training data. The 2D distributions are obtained using (-SNE. For the 3D distributions, first run SVD with the mean embedding of each species. The first two singular vectors used construct the gray plane that captures differences. The embeddings are then projected into straight lines point from the mean embedding of juvenile images to that of the adult images. As the training scales up. the iptra-species variations are preserved in the subspace orthogonal to the ipter-species differences. Orthogonality improves with data scale, evidenced by the decreasing varrincd-variance ratio p.
inter-species space with an additional orthogonal dimension. The
5.4 Formal Analysis
We next ask why these two properties of inter-species ecological alignment and intra-species variation separation emerge with scale.
Inter-species ecological alignment. As training data increases, species that share proximal taxonomic labels are pulled toward common textual prototypes at multiple levels. As related taxa typically share morphology, behavior, and ecological characteristics, this multi-level supervision aligns visual similarity with functional similarity [15]. With more samples per species providing supervision at scale, embeddings of species in the same family or genus form coherent macro-clusters. In effect, the embeddings extracted by BIOCLIP 2 are more separable for different ecological groups.
Intra-species variation separation. While the inter-species ecological alignment can be explained by hierarchical supervision, the intra-species variations are not encoded in taxonomic labels. The preservation of intra-species variations also contradicts the common intuition of contrastive training effects. Therefore, we investigate the optimization contrastive loss. We propose the following theorem suggest that subspace orthogonal inter-species differences is allowed after extensive training to accommodate intra-species variations.
Theorem 5.1. Let μ be the prototypes of species, with μts as the prototype of species 8. Let be temperature. If different μa are nearly orthogonal (i.e., species are well separated), the intra-species variation 6 for species s constrained by
WaMLE Motte ) 6, where
(op((TTPx)/T px)/T Σx (exp(H) (x)/T)
exp(p1
87
Σ &
2r2
Proof. See SC.
Thus, long as the variation 8 is distributed in subspace orthogonal the inter-species distinctions, the scale of 8 won't interfere with the overall contrastive optimization. The orthogonality is qualita- ratio, i.e. the ratio in the intra-species variation that is captured by the species span [71]. We first obtain an orthonor- mal basis for the species prototypes using QR decomposition. Let D [d,,... ........ d,,l E Rdxa be the matrix stacking intra-species variation difference vectors with dimension d. The explained- variance ratio calculates the energy fraction inside species span by م | D / DI ? where ||- l| denotes the Frobenius norm [72]. We show the ratio change as the data scales up in Figure 3c. The results suggest that the intra-species variations are increasingly orthogonal to the species differ- ences. Due to the orthogonality, the existence of intra-species variations will not interfere with the inter-species distinctions. The observation of smaller projection areas in Figure 5 at the species span also indicates better species classification accuracy after extensive contrastive training.
tively supported by Figure 5. To further quantify it, we calculate the explained-variae

---

Table 3: Ablation study with different training settings. BIOCLIP 2 adopts hierarchical contrastive training with taxonomic labels. We ablate using scientific names solcly without taxonomic labels, and one-hot labels with cross-entropy loss instead of the contrastive objective.
VIT-8/15
WIT-L/1
WIF-L/14 Beplay
Accuracy (80
Hicrarchical Contrastive
Contrastive w/ Scientific Name
Cross-cntropy Loss
Dataset
FishNet
35. 10.1
33.820.1
33.0
PlantDoc
38. 7 #3.7
37.3
30.9
Life stage-Diff
88.0
88.5 88.5
85.5
Rasespecies
FiahNet
Hacbarium1
FlantDoc
Life stage-Align
84.1
84.5+0.1
78.6+0.1
Sex-Diff
97.0+0.1
96.640.1
95.5
Figure 6:
Ahlation study on
Sex-Align
84.1
82.720.3
74.9
model size and experience replay.
Furthermore, we look at the separation between these variations using the Fisher Discriminant Ratio (FDR) metric [73]. Given two variant classes A and B, FDR is defined by their embedding mean and standard deviation a: FDR = μ /uBl²//o²" + oz)) The increasing trends in Figure 3c support that, in addition to orthogonality, the intra-species variations are more separable as the training scales up. This trend is also qualitatively evidenced by the top row of Figure 5. These theoretical and empirical insights validate that BIOCLIP 2 learns preserve and separate intra-species variations without explicit training constraints. We provide more qualitative analyses in $F.2.
Ablation Study and Analysis
The necessity of contrastive loss and taxonomic labels. The previous analyses highlight the effectiveness of hierarchical contrastive training. Table 3 ablates two key modeling decisions using TREEOFLIFE- 10M as a feasible testbed: scientific names solely vs. hierarchical labels cross-entropy loss on one-hot labels. The experiments
and contrastive learning TREEOFLIFE- 1 OM data with ViT-L/14 as the visual encoder. When trained solely with scientific names, the model loses some of the hierarchical supervision embedded in taxonomic labels. As a result, the performance on FishNet drops by 1.3%. However, contrastive training still leads to separation for both species and intra-species variations. Training 952K-class softmax classifier with and leads to inferior performance on all benchmarks. The
conducted on
cross-entropy loss is optimization-henn adopted hierarchical contrastive supervision leverages the advantages of both aspects and yields the best overall performance across benchmarks.
Architecture and replay. BIOCLIP 2 scales up the visual encoder of B1OCLIP from VIT-B/16 to ViT-L/14 4 and introduces experience replay of CLIP training data (LAION-2B). We ablate the effects of these two changes, again using TREFOFLIFE- 10M testhed. Figure 6 shows that increasing model capacity improves performance across all benchmarks. Comparatively, experience replay leads better species elassification accuracy and improved performance on some of the other visual tasks. We provide more detailed empirical study of experience replay in $F.1.
Conclusion
In this work, we curate TREEOFLIPE-200M, the largest and most diverse biological organism dataset to date, and train BIOCLIP 2 with hierarchical taxonomic labels. BIOCLIP 2 achieves state-of-the-art accuracy on species classification. More importantly, large-scale training gives rise to two emergent properties not described during training. At the inter-species level, the embedding distribution of different species aligns with their ecological relationships. the intra-species level, the appearance variations within species are preserved and well separated in the embedding space. demonstrate that combining the effort domain-specific scaling and structured supervision leads to effective generalization beyond the initial training objectives. BIOCLIP 2 serves as strong foundation model for biological research and simultaneously evidences the effectiveness of scale-driven scientific discovery.

---

Acknowledgments and Disclosure of Funding
We would like to thank Zhiyuan Tao, Shuheng Wang, Ziheng Zhang, Zhongwei Wang, and Leanna House for their help with the TREEOFLIFE-20OM dataset, Charles (Chuck) Stewart, Sara Beery, and other Imageomics Team members for their constructive feedback and Sergiu Sanielevici, Tom Maiden, and TJ Olesky for their dedicated assistance with arranging the necessary computational resources.
We are grateful to Kakani Katija and Dirk Steinke for helpful conversations regarding use and integration of NathomNet and BIOSCAN-5M, respectively, as well as Stephen Formel and Markus Döring for GBIF. We thank Marie Grosjean for comparative methods for filtering citizen science images and Dylan Verheul for assistance with acquiring images from observation.or from GBIF. We thank Suren Byna for helpful conversation on early dataset design decisions. We thank Doug Johnson for his collaboration in hosting this large dataset on the Ohio Supercomputer Center research storage file system.
Our research is supported by NSF OAC 2118240 and resources from the Ohio Supercomputer Center [74]. This work used the Bridges-2 system, which is supported by NSF award number OAC-19281 147 at the Pittsburgh Supercomputing Center (PSC) [75], under the auspices of the NAIRR Pilot program.
References
[1] Hanchen Wang, Tianfan Fu, Yuanqi Du Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, et al. Scientific discovery in the age of artificial intelligence. Nature, 620(7972) 60, 2023. 2
[2] Yongjun Xu, Xin Liu, Xin Cao, Changping Huang. Enke Liu, Sen Qian, Xingehen Liu, Yanjun Wu, Fengliang Dong, Cheng- Wei Qiu, et al. Artificial intelligence: powerful paradigm scientific research. The Innovation, 2(4), 2021. 2
[3] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figumov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-553 5885 2021. 2
[4] Zeming Lin. Halil Akin. Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-seale prediction of atomic-level protein structure with a language model. Science, 37966637) 2023.
[5] Amil Merchant, Simon Batzner, Samuel S Schoenholz, Muratahan Aykol, Gowoon Cheon, and Ekin Dogus Cubuk. Scaling deep learning for materials discovery. Nature, 624(7990):80 85, 2023. 2
Huan Tran, Rishi Gurmani, Chiho Kim, Ghanshyam Pilania, Ha-Kyung Kwon, Ryan Lively, and Rampi Ramprasad. Design of functional and sustainable polymers assisted by artificial intelligence. Nature Reviews Materials, pages 1-21, 2024.
[7] Remi Lam, Alvaro Banchez-Gonzalee Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, et al. Learning skillful medium-range global weather forecasting. Science, 382(6677):1416 1421, 2023. 2,31
[8] Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium-range global weather forecasting with 3d neural networks. Nature, 619(7970):533 538, 2023, 2
[9] Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities large language models. TMLR, pages 1-30, 2022. 2,
[10] Daniil A Boiko, Robert MacKnight. Ben Kline, and Gabe Gomes. Autonomous chemical research with large language models. Nature, 624(7992):570-557 578, 2023. 2
[11] Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle C Lawrence Zitnick, Jerry Ma, et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences. 118(15): e20162391 18, 2021.
10

---

[12] Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In ICML, pages 7763.8763. PMLR, 2021. 2, 4, S, 23, 24
[13] Samuel Stevens, Jiaman Wu, Matthew J Thompson, Elizabeth Campolongo, Chan Hee Song, David Ed- ward Carlyn, Li Dong, Wasila M Dahdul, Charles Stewart, Tanya Berger- Wolf, et al. Bioclip: A vision foundation model for the tree of life. In CVPR, pages 11412-19424) 2024. 2, 3, 5, 26,
[14] Chih-Hsuan Yang, Benjamin Feuer, Talukder Jubery, Zi Deng, Andre Nakkab, Md Zahid Hasan, Shivani Chiranjeevi, Kelly Marshall, Nirmal Baishnab, Asheesh Singh, et al. Biotrove: A large curated image dataset enabling for biodiversity. In NewrIPS, volume 37, pages 102101-102120, 2024. 2, 3, 24, 30
[15] José M Padial, Aurélien Miralles, Ignacio De la Riva, and Miguel Vences. The integrative future of taxonomy. Frontiers in zoology, 7:1-14, 2010. 2,
[16] Faizan Farooq Khan, Xiang Li, Andrew Temple, and Mohamed Elhoseiny. Fishnet: A large-scale dataset and benchmark for fish recognition, detection, and functional trait prediction. In ICCV, pages 20496-20506, 2023, 3, 6, 28
[17] Yongqin Xian, Christoph H Lampert, Bernt Schiele, and Zeynep Akata. Zero-shot learning compre- hensive evaluation of the good, the bad and the ugly. IEEE TPAMI, 41(9):2251-22 2265, 2018. 2, 6, 25, 29
[18] Grant Van Horn, Eljjah Cole, Sara Beery, Kimberly Wilber, Serge Belongie, and Oisin Mac Aodha. Benchmarking representation learning for natural world image collections. In CVPR, pages 12884-12893 2021. 2, 3, 6, 7, 29, 32
[19] Kiat Chuan Tan and Yulong Liu. Herbarium challenge 2019 competitions hwwrioooooeeebrrrrrreaeee 2019- 2019-fgvc6, 2019. Kaggle. 2. 6, 24, 29
fgvc6.
https://kaggle com/
[20] Davinder Singh, Naman Jain, Pranjali Jain, Pratik Kayal, Sudhakar Kumawat, and Nipun Batra. Plantdoc: A dataset for visual plant disease detection. In Proceedings of the 7th ACM IKDD CoDS and 25th COMAD, pages 249-253, 2020. 2, 3, 6, 24, 29
[21] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. ICCV, pages 11975-1196, 2023. 2, 24
[22] Oriane Siméoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Mare Szafranicc, Scungeun Yi, Michaël Ramamonjisoa, et al. Dinov3. arXiv preprint 10X1::2508.10104 2025. 2, 6, 24
[23] Sangeet Lamichhancy, Jonas Berglund, Markus Sällman Almén, Khurram Maqbool, Manfred Grabherr, Alvaro Martinez-Barrio, Marta Promerová, Carl-Johan Rubin, Chao Wang, Neda Zamani, al. Evolution of darwin's 's finches and their beaks revealed by genome sequencing. Nature, 518(7539) 2015. 2,
25
[24] Jinxin Zhou, Chong You, Xiao Li, Kangning Liu, Sheng Liu, Qing Qu, and Zhihui Zhu. Are all losses created equal: A neural collapse perspective. In NeurlPS, volume 35, pages 3 31697-31710, 2022. 2, 27
[25] Jianfeng Lu and Stefan Steinerberger. Neural collapse under cross-entropy loss. Applied and Computational Harmonic Analysis, 59:224 241, 2022. 2, 27
[26] Leonardo Berti, Flavio Giorgi. and Gjergji Kasneci. Emergent abilities in large language models: A survey. arXiv preprint arXiv:2503.05788 2025. 3
[27] Zheda Mai, Arpita Chowdhury, Zihe Wang, Sooyoung Jeon, Lemeng Wang. Jiacheng Hou, and Wei- Lun Chao. Ava-bench: Atomic visual ability benchmark for vision foundation models. arXiv preprint arXiv:2506.0900 09082, 2025. 3
[28] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégon, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, pages 9650-9660. 2021.
[29] Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Bycon, Thomas Breuel, Jan Kautz, and Xiaolong Wang. Groupvit: Semantic segmentation emerges from text supervision. In CVPR, pages 18134-18144 144, 2022. 3
[30] Morris Alper and Hadar Averbuch-Elon Emergent visual- semantcc hierarchies in image-text representations.
In ECCV, pages 220-238. Springer, 2024. 3

---

[31] Reza Abbasi. Mohammad Hossein Rohban, and Mahdieh Soleymani Baghshah. Deciphering the role of representation disentanglement: Investigating compositional generalization in clip models. In ECCV, pages 35-50. Springer, 2024.
[32] Valentin Gabeff, Mare Rušwurm, Devis Tuia, and Alexander Mathis. Wildclip: Scene and animal attribute retrieval from camera trap data with Nomain-adaptee vision- langugge models. IJC,, 132(9):377--3786 2024. 3
[33] Julian D Santamaria, Claudia Isaza, and Jhony Giraldo. Catalog: camera trap language-guided contrastive learning model. In WACV, pages (197-1206 IEEE, 2025. 3,5
[34] Srikumar Sastry, Subash Khanal, Aayush Dhakal, Adeel Ahmad, and Nathan Jacobs. Taxabind: unified embedding space for ecological applications. In WACV, pages 1765-1774. IEEE, 2025. 3
[35] Samuel Stevens, Jiaman Wu, Matthew J Thompson, Elizabeth Campolongo, Chan Hee Song, David Ed- ward Carlyn, Li Dong, Wasila M Dahdul. Charles Stewart, Tanya Berger- Wei-Lun Chao, and Yu Su. TreeOfLife- 10M 2023. doi: 10.57967/h/11972. URL https: httgggggggggggceeee////////// cttpeaaae/e///////////aaa weaggeomiccsTTreeffiiii 101.
[36] Grant Van Horn and Oisin Mac Aodha. inat challenge 2021 fgvc8, 2021. URL hwtps:://tst/////// com/ competitions/ wwwpitiiionnnnttnnnaniaea Lnnturaatuassstt 2021. 3, 32
[37] Z. Gharaee, Z. Gong. N. Pellegrino, I. Zarubiieva, J. B. Haurum, S. C. Lowe, J. T. A. McKeown, C. Y. Ho. J. McLeod, Y. C. Wei, Agda, S. Ratnasingham, D. Steinke, A. X. Chang, G. W. Taylor, and P. Fieguth. step towards worldwide biodiversity assessment: The BIOSCAN-IM insect dataset. In NeurIPS, volume 36, pages 13593-4361 Curran Associates, Inc., 2023.
[38] GBIForg. GBIF occurrence download. May 2024, doi: 10.15468/DL.BFV33. URL htL 10. 10.6648///1, vf33 3, 30
[39]
Encyclopedia Life (EOL). URL https:///eel.co org. Accessed August 2024.
[40] Zahra Gharaee, Scott Lowe, ZeMing Gong, Pablo Millan Arias, Nicholas Pellegrino. Austin T. Wang. Joakim Bruslund Haurum, Iuliia Zarubiieva, Lila Kari. Dirk Steinke, Graham Taylor, Paul Fieguth, and Angel X. Chang. BIOSCAN-5M: A multimodal dataset for insect biodiversity. In NeurlPS, volume 37, pages
2024. 3
[41] Kakani Katija, Eric Orenstein, Brian Schlining, Lonny Lundsten, Kevin Barnard, Giovanna Sainz, Oceane Boulais, Megan Cromwell, Erin Butler, Benjamin Woodward, and Katherine L. C. Bell. FathomNet: A global image database for enabling artificial intelligence in the ocean. Scientific Reports, 12(1):15114, Septemher 2022. ISSN 2045-2322. doi: 10. 1098//54558---022- 19939-2. URL htRp::// nature.
com/ com/articles/stt 598-1298-22222 19939- 2.
[42] iNaturalist contributors and iNaturalist. iNaturalist Research- grade Observations. Accessed via GBIForg (Occurrence Snapshot: 10. (0.15468/DL.BFV333) 2024-05-01. 4, 30
[43] Smithsonian Institution. URL wwt.hhtt:://// //wp:::://www... s1 edu/ Accessed May 15, 2025.
[44] Flickr. URL battps://vee http:s://wwwfflcckee www.ck.i..r.rrr cww/ Accessed May 15, 2025. 4
[45] Andrei Kopanev. Matthew J Thompson, and Elizabeth G Campolongo. Distributel-donnloaaerr 2025. doi: 10.5281/zenodo. 17418004 URL http:: htaeilhuuu.mmmmmmmmmeeeee hem//mmmaammmaaaaaaaaeaaaaeaaaaa http:eemmiie//isstiiiibateaeeeee disatrrbbbtdddddwoloddeee
[46] Dmitry Mozzhern. GNverifier reconeiler and resolver of scientific names against more than 100 data sources. November 2024. doi: htt.:///eeooo..000004000 URL https:// http://gttttutbbt com/ gnames/ gnverifier.
[47] Sara Beery, Dan Morris, and Siyu Yang. Efficient pipeline for camera trap image review, 2019. 4, 31
[48] Andres Hernandez, Zhonggi Miao, Luisa Vargas, Sara Beery, Rahul Dodhia, and Juan Lavista. Pytorch- wildlife: collaborative deep learning framework for conservation, 2024. 4,31
[49] Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE signal processing letters, 23(10))1499-1100 2016. 4, 32
[50] Facebook, Inc. Threatexchance.
http://w.........ooooooeeaaaaaaa 2019. 4, 33
[51] IUCN. IUCN Red List Summary Table 1a, March 2025. URL redlist/content/eee wtth///nttttttttttceammmam filpeff/oeeeoeeeeeeeeeeee BL_ Table_aaaaae 1a pdf. 21,
org/
12

---

[52] IUCN. The IUCN Red List of Threatened Species. version 2025-1. http:://ww....cccereeeeeee iucnredlistt org/ http://mmmmaiie44444-4---1-3---- May 2025. 4, 33
2025, Accessed on 14
[53] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, pages 1-21, 2021. 2,
[54] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski. Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarezyk, and Jenia Jitsev. LAION-S 5b: An open large-scale dataset for training next generation image-text models. In NewrIPS, pages 1-17, 2022. 5, 23
[55] Sylvestre-Alvis Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. CVPR, pages 2001-2010, 2017.
[56] David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. In NeurlPS, volume 32, 2019. 5
[57] Edward Vendrow, Omiros Pantazis, Alexander Shepard, Gabriel Brostow, Kate Jones, Oisin Mac Aodha, Sara Beery, and Grant Van Horn. Inquire: A natural world text-to-image retrieval benchmark. In NeurlPS, volume 37, pages 126500-126514 126514, 2024. 5, 25
[58] Ihsan Ullah, Dustin Carrion, Sergio Escalera, Isabelle M Guyon, Mike Huisman, Felix Mohr, Jan N Rijn, Haozhe Sun, Joaquin Vanschoren, and Phan Anh Vu. Meta-album: Multi-domain meta-dataset for few-shot image classification. In NeurIPS, 2022. URL https //meta//mmeeeeeeeeee album. gi gitthb...om to/. 5
[59] Samuel Stevens, Jiaman Wu, Matthew J Thompson, Elizabeth G Campolongo, Chan Hee Song. David Ed- ward Carlyn, Li Dong. Wasila M Dahdul, Charles Stewart, Tanya Berger- Wolf, Wei-Lun Chao, and Yu Su. Rare species. 2023. doi: 10.57967/h//1881. URL https: https:://utgtiggeeeeee httg//uggggggneeeeeee/eeeeeeeeee co/battasets/inggemiice rare- sprceese 5, 26, 32
[60] Gerald Piosenka. Birds 525 species- - image classification, 05 2023. URL http:///:/wwar kaggle. com/ datasets/ /atpottssgggooosaa///eo matsssnnnaan0---o 1000 speddesp 6
[61] Grant Van Horn, Steve Branson, Ryan Farrell, Scott Haber, Jessie Barry, Panos Ipeirotis. Pietro Perona, and Serge Belongie. Building bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In CVPR, pages 595-604, 2015. 7, 29
[62] Labeled information library of alexandria:
Biology
and conservation (LILA-BC).
https//lilaccience/daaassss 6, 26, 28
[63] Island Conservation. Island conservation camera traps. http://llasiiieeeccaaacc/aaaaaet
camera-traps/. 6, 26, 28
[64] Desert Lion Conservation. Desert lion conservation camera traps. http:://llaasscccccddassssssssss conservation-ccaee---tp... July 2024. 6, 26, 28
[65] Juliana Vélez, Paula J Castiblance-Camacho Michael A Tabak, Carl Chalmers, Paul Fergus, and John Fieberg. Choosing an appropriate platform and workflow for processing camera trap data using artificial intelligence. arXiv preprint arXivv2200,.02283, 2022. 6, 26, 28
[66] S. Balasubramaniam. Optimized classification in camera trap images: An approach with smart camera traps, machine learning, and human inference. Master's thesis, The Ohio State University, 2024. URL h/tae//rave.. ohiol ink.
6, 26, 28
[67] Hayder Yousif, Roland Kays, and Zhihai He. Dynamic programming selection of object proposals for sequence-level animal species classification in the wild. IEEF Transactions Circuits and Systems for Video Technology, 2019. 6, 26, 28
[68] Guangxing Han and Ser-Nam Lim. Few-shot object detection with foundation models. In CVPR, pages 28608-288 28618, 2024. 6
[69] Simon Damm, Mike Laszkiewicz. Johannes Lederer, and Asja Fischer. Anomalydino: Boosting patch- based sho- anomaly detection with dinov2. In WACV, pages 1319 1329, IEEE, 2025, 6
13

---

[70] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, ((86):2579-2605 2008.
[71] Hervé Abdi and Lynne J Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2(4):433 459, 2010.
[72] Gene H. Golub and Charles F. Van Loan. Matrix Computations. Johns Hopkins University Press, Baltimore, MD, 4 edition, 2013. ISBN 978 1-4414--49994444 8
[73] Ronald A Fisher. The use of multiple measurements in taxonomic problems. Annals of eugenics, 7(2): 179-188, 1936.
[74] Ohio Supercomputer Center. Ohio supercomputer center, 1987. URL http:://scc.cc// http:///sscceddukkk/11/////
15s1ph73. 10
[75] Shawn T. Brown, Paola Buitrago, Edward Hanna, Sergiu Sanielevici, Robin Scibek, and Nicholas A. Nystrom. Bridges-2: A platform for rapidly-cvolving and data intensive rescarch. In Practice and Experience Advanced Research Computing 2021: Evolution Across All Dimensions, PEARC '21, New York, NY. USA, 2021. Association for Computing Machinery. ISBN 9781450382922. doi: 10.1145/
3437359.3465593. URL https://dooi org/10. 10. /145/3437355 30443359....33599... 10
[76] Devis Tuia, Benjamin Kellenbenger, Sara Beery, Blair R Costelloe, Silvia Zuffi, Benjamin Risse, Alexander Mathis, Mackenzie W Mathis, Frank van Langevelde, Tilo Burghardt, et al. Perspectives in machine learning for wildlife conservation. Nature communications. 13(1):792, 2022. 21
[77] David I Lindenmayer and Ben Scheele. Do not publish. Science, 356(6340))800-801, 2017. doi: 10.1 12.1 science.aan 1362. 22
[78] Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. Pali: A jointly-scaled multilingual language- image model. In ICLR, pages 1 1-33, 2023. 24
[79] Jia Deng. Wei Dong, Richard Socher, Li-Jia Li. Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255. IEEE, 2009. 24
[80] Vignesh Kothapalli. Neural collapse: A review on modelling principles and generalization. Transactions Machine Learning Research, pages 1-43, 2022. 27
[81] Vardan Papyan, XY Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training, Proceedings of the National Academy Sciences, 117(40):24652 24663, 2020. 27
[82] Prannay Khosla, Piotr Teterwak. Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola. Aaron Maschinot. Ce Liu, and Dilip Krishnan. Supervised contrastive learning. In NeurlPS, volume 33, pages 18661-18673, 2020. 27
[83] Jaidev Gill, Vala Vakilian, and Christos Thrampoulidis. Engineering the neural collapse geometry of supervised-commrrstivv loss. In ICASSP, pages 7115-7119. IEEE, 2024. 27
[84] Zhihui Zhu, Tiayyu Ding, Jinxin Zhou, Xiao Li, Chong You, Jeremias Sulam, and Qing Qu. A geometric analysis of neural collapse with unconstrained features. In NewrlPS, volume 34, pages 29820-2982 2021.
[85] Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Generalized category discovery. In CVPR, pages 7492-7501, 2022. 29
[86] Yan Wang, Wei-Lun Chao, Kilian Weinberger, and Laurens Van Der Maaten. Simpleshot: Revisiting nearest-neighhor classification for few-shot learning. arXiv preprint arXiv: (adi::911..0422. 2019. 29
[87] GBIF Secretariat. GBIF Backbone Taxonomy. 2023. doi: 10.15468/39omei. https:/// 15448/39ommss 30
[88] J. Doubt, R. Blades, and A. Tyrell. Canadian Museum of Nature Herbarium. 2025. doi: 10. 1446//64ej.... Version 1.283. Canadian Museum Nature. Occurrence dataset https://doi.oogg11..5444466 accessed via GBIForg on 2025-05-22. 32, 33
[89] Natural History Museum and Benjamin Drew. Natural History Museum (London) Collection Specimens. 2025. Occurrence dataset https://doi.com org/ 10.55119/d...x4gff..... accessed via GBIForg on 2025-05-14 14. https://www. www.ggif...oooooo www... png//occureence/10563333 History Museum, London (licensed under http://creativecommmn.. htt.://mww.oooiiiiienssss///.// 33
Collected in Suriname by The Trustees of the Natural
14

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: This paper focuses on emergent properties in large-scale hierarchical contrastive learning for biological organismal data. We have theoretical and empirical results to support the claims.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contri- butions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
Justification:
We have limitation section in $A including the discussion of theoretical and data limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results violations of these assumptions (c.g., independence assumptions, noiseless settings, model well-specification asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken low lighting.
speech-to-tex system might not used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof? Answer: [Yes]
Justification: We propose theorem to support the orthogonality between the intra-species variations and inter-species differences. The proof is provided in $C.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
15

---

All ussumptions should be clearly stated or referenced in the statement of any theorems. The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification: We have provided detailed instructions on the model architecture and training details. The results are fully reproducible.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g.. large language model), then there should either be way access this model for reproducing the results way reproduce the model (c.g., with an open-soure dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users). but should be possible for other researchers have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions faithfully reproduce the main experimental results, as described supplemental material? Answer: [Yes]
Justification: PREEOFLIFE-200X is published on Hugging Face under the least restrictive licensing allowed by its constituent parts, accompanied by well-documented dataset card, and archived with DOI. The IDLE-OO Camera Traps dataset is also published on Hugging Face, under the Community Data License Agreement (permissive variant) used by LILA-BC datasets. The reconstruction of TREEOFLIFE
requires downloading the images from their original sources, instructions for which are included with the dataset and will be published alongside it. Source code is available on GitHub. Guidelines:
The answer NA means that paper does not include experiments requiring code.
16

---

Please see the NeurIPS code and data submission guidelines (ttt:e:///ips..ccppbbliic for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: asi asionPoolcy for more details.
Seepiic/guiee//oooee The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
/ //nips
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. If only subset of experiments are reproducible, they should state omitted from the script and why. At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
which
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The training hyperparameter settings are listed in $D. A detailed instruction of biological visual evaluation presented in SH.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, material.
supplemental
7. Experiment statistical significance
Question: Does the paper report eπror bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes]
Justification: Among the experiments presented in the paper, zero-shot classification is determin- istic, so we only report the accuracy for those experiments. Other than zero-shot classification, we conduct the other experiments-few-e. 5 runs and report the mean and standard deviation. Guidelines:
classification and other biological visual tasks for
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates). If error bars are reported in tables or plots. The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
17

---

For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have reported the required compute resources for the BIOCLIP 2 model training and the evaluation of other biological visual tasks.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https:/ hatpa:://euurippp cttppuuicc/thiicccuieeeellllllll Answer: [Yes]
Justification: The submission conforms to the NeurIPS Code of Ethics, where individual relevant adherence to components of the code are discussed in detail in other portions of the submission. For example, human faces are removed from the dataset to preserve privacy and data licenses and provenance are preserved. Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consideration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: We have included a Broader Impacts section in $B.
Guidelines:
The answer NA means that there is societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact
why the paper does address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance). fairness considerations (c.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there direct path any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mecha- nisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
18

---

Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)? Answer: [NA].
Question:
Justification: Images were gathered from established, publicly accessible biodiversity repositories and we used established procedures to ensure removal images with human faces. Additionally, there no GPS locations associated with the threatened species whose images are included. Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes].
Justification: The core data providers through whom we accessed images and their metadata for LIFFOFLIFFE-2OM. are directly cited in the paper. We provide and document provenance for all images through a dedicated Parquet file connecting each image license, creator, etc. and include explanation in dataset card. All remaining data, code, and models are cited appropriately, as indicated by their creators. Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset is used and, if possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes].
Justification: We have the detailed dataset cards on Hugging Face for TREEOFLIFE-200 200M and IDLE-OO Camera Traps. Documentation for code is also provided in the published GitHub repository.
Guidelines:
The answer NA means that the paper does not release new assets. Researchers should communicate the details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL include anonymized zip file.
14. Crowdsourcing and research with human subjects
19

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: We do not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with
human subjects.
Including this information in the supplemental material is fine, but if the main contribution the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: We do not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LIM usage
Question: Does the paper describe the usage of LLMs is an important, original, or non- standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration Answer: [NA]
required.
Justification: LLMs were not involved in developing the core method of this paper. Guidelines:
The answer NA means that the core method development in this research does not involve
LLMs as any important, original, or non-standard components.
Please refer to our LLM policy thttps: /ttp::////urrpss/e wwtpurrppsscccooffeaeeeeee wwt.c//ooooeeeeee///O/OOO /2025/LLM) for what should or should not be described.
20

---

Appendix
The Appendix is organized follows:
ŠA we discuss limitations of our work.
In BB we discuss the broader impacts of our work.
$C we present the proof of Theorem 5.1.
In SD we describe the training implementation for BIOCLIP 2 in detail.
Š we introduce the baselines employed for the numerical experiments.
In SF we demonstrate more empirical observations of BIOCLIP 2.
$G we discuss the relationship between the adopted training scheme and neural collapse. In $H we describe both the biological visual benchmarks and the implementation details.
In ŞI we provide the details of data processing for TREEOFLIFE-20OM, In JJ we list the contribution of each author.
A Limitations
Theoretical limitation. In this work, we have proved that the intra-species variations are preserved in subspaces orthogonal to the inter-species difference. We also have empirical observations that the variations are more separable as the training data scales up (See Figure 3c). However, we haven't t formally proved this. will be our future work to have deeper theoretical analyses of the separation of intra-species variations to better understand B1OCLIP 2's emergent properties.
Data limitation. TREEOFLIFE-200M is imbalanced dataset in both taxonomic coverage and image type. Specifically, the dataset exhibits a long-tailed distribution across taxa. This is to be expected when working with biological data-no all taxonomic ranks are represented evenly across the tree of life. For instance, though TREEOFLIFE-2001 has a balanced representation (at the kingdom level) between plants and animals, animals represent larger proportion of described species [51].
Some of this is due to the nature of the image type distribution, which provide for GBIF (Camera- trap, Citizen Science, and eleven Museum Specimen types: Fungi, Insect, Invertebrate Zoology, Microbiology, Plant, Vertebrate Zoology [Amphibians, Birds, Fishes, Mammals, Others], well as Unidentified). EOL. contains the same categories, but we do not have precise numbers; BIOSCAN-SM essentially all insect museum specimens, though the images skew toward their area of study; FathomNet contains mix. Citizen Science images are the vast majority (151M from GBIF alone); these will skew toward more charismatic species and plants. Our next largest category is museum specimen images (51.8M from GBIF), which are limited more to representatives of the species, so may not contain as many images per taxa, though more taxa are represented. Finally, camera trap images make up the smallest portion of the dataset (617.8K in GBIF), and when filtered, these are only images of animals and generally those large enough to trigger motion sensor or be detected in the primary provider's 's post-processee
taken by researchers, so will
Further emphasis the impact of citizen science images in amassing larger representations of species: the most prevalent taxonomic classes, flowering plants, insects, birds, mushrooms, and mammals, have millions of representative images, while the least-represented, microscopic organisms (e.g., bacteria, viruses), have a dozen or fewer.
B Broader Impacts
BIOCLIP and TREEOFLIFE-200M provide great potential to improve and enhance existing conservation efforts, in particular by facilitating recognition of threatened species. As noted in $3.3, TREEOFLIFE-
has expansive coverage of threatened species, as classified by IUCN. additionally builds on the coverage of species considered to be Data Deficient. Based on the emergent properties displayed by BIOCLIP 2, there potential to add to the effort to understand the risks facing these species that cannot currently be classified by IUCN due to lack of available information, as suggested [76]. These designations are crucial to the international effort to protect biodiversity across the planet.
21

---

Unfortunately, as with many open-source efforts to further conservation goals, there is potential for bad actors make use of these tools for malicious purposes. Though the improvement on threatened species could make it easier for poachers to identify protected species, these types of tools are a force-multiplier to monitor illicit trade and sales of these same species. The primary risk to endangered species comes from disclosure of precise location information rather than improved classification capability [77]. Our data does not provide geo-tagged information on the organisms included, minimizing the vulnerabilities that could be used in poaching.
C Proof of Theorem 5.1
Proof. The contrastive loss for one visual embedding = belonging to the class and the corresponding text embedding c, is:
exp(=
[(=, c.) = - log Σx exp(z Ca/T)
Let h (L.)) be the text encoder. Assume the representation is already close the species prototype H4 = ho(L(s)) = Cs. and there is a residual ó representing the intra-species variance:
= Ms.
|
<
l| Meselll.
Define:
m P2
exp(ax)
Z Σ exp(ax), wkk
Ok
Z
Substituting z - -1s + into Equation 1:
1m, + 6.c = (1t, + 8) Ms
(Ms + +0)T pak.
+ log
Σ
exp
(2)
k
Mg (1, +
T Mb
T u4
+ log Σ exp
eep
(3)
7
T
1._ 5
+ log
Σ exp
[/g 14
explut px/T)
exp
88 14
5
Σx exp(μ mx/T)
(4)
Mg us
T
log Z + log
Σ wk exp
8 14
(5)
Use the Taylor expansion the second order for the argument of the logarithm of the last term log |>x wa exp(0" p4 [
(8)
6 44
T 11k
(8T
2
Σ
wk exp
1
Wk
Wkk
+ 0(11813).
(6)
T
T2
27
obtaining (6)) =
Σ
Wk
8 L
+
1
Σ
Wk
(8 1p
Σ
wk
8
14k
+
1115)
2
T2
Insert (5) back into Eq.2:
11_
8 14
= 1(1ta, (())))))))
k
1
(8 T 1
8 #/
Σ
0118))3).
wk
Wk
72
T
k
22

---

Define m := Σx Whfts. Then collecting first-order terms around ó results in the following:
Ex WhIL p4s T (m
p4a)
Suppose the training resulted in w, s 1, leading to 14g as the embeddings of different species are better separated.
Then the first-order terms will vanish
We further rewrite the second-order term of Eq.6 as:
(8T
pbx
Σ
8" uk
8
(6 m) m)
Wk
Wk
Wk
T2
T
2
T2
T²
Wi - mmm )]0
| 8.
8
2r
Substituting m M4s, have an approximation as
T
Σ
8.
2-
The Hessian matrix lies in the span of species, as each term is outer product of prototype. Therefore, as long as the residual is distributed in subspace orthogonal to the species differences, the scale of 5 will not interfere with the overall contrastive optimization.
D Training Implementation Details
Table 4: The adopted ppermparreettr setting in train- ing BIOCLIP 2.
Table 5: The adopted hyper parameter setting in the ablation study.
Hyper-parameter
Value
Hyper-
Hyper-parameter parameter
Value
Architecture
ViT-L/14
Architecture
ViT-L/14
Optimizer
Adam
Optimizer
Adam
Batch size/GPU (organism)
2,816
Batch size/GPU (organism)
2,816
Batch size/GPU (replay)
320
Batch size/GPU (replay)
320
GPUs
32 H100s
GPUs
8 H100s
Epochs
30
Epochs
100
Max learning rate
x 10
Max learning rate
1 10
Warm-up steps
1,875
Warm-up steps
1,500
Weight decay
0.2
Weight decay
0.2
Input resolution
224
Input resolution
224
We list the adopted hyperparameters in Table and Table 5 for the BIOCLIP training and ablation study, respectively. TREEOFLIFE 10M is used for the ablation study. The batch size presented in both tables is the size per GPU. In addition to the larger GPU number and smaller batch size compared with the training in BIOCLIP due to the larger model size, another important modification is the introduction of experience replay. An additional visual projector is introduced as described in SF.1. Beyond the visual projector, the model architecture is kept the same as that of CLIP [12]. During the evaluation, all the embeddings are extracted with the visual projector for hierarchical label matching avoid extra influence.
E Baseline Details
In the quantitative experiments, we compare BIOCLIP with the following baseline vision-language models and vision-only models. Without specification, the input image size is 224.
CLIP (VTT-L/14), We compare BIOCLIP 2 with the CLIP model pre-trained on LAION-2B [54], which has the same architecture and patch size. The CLIP model is also used as the weight initialization of the B1OCLIP training. uses ViT-large visual encoder with patch size of 14. We load the weight from the OpenCLIP repository.
23

---

Table Performance comparison between different replay designs of CLIP training data. All models in the bottom three rows are initialized with CLIP (the first row) and trained with TREEOFLIPE-IOM data. The CLIP model is pre-trained with LAION-2B data, from which we randomly select 2M samples for this experiment. A represents the performance gap over the CLIP bascline. Bold entries indicate the best accuracy.
Species
Non-Species
Rarespecies
INQUIRE
NABirds
FishNet
NeWT
ANAA
llerh.
PlanDoo
Model
Mean (A)
CLIP (VIT-L/14)
66.5
35.2
27.9g0.2
83.4go.
61.6aaa
18.2
22.3
35.0
43.8
Replay
68.9
16.1
35
35.3
83.8+0.1
58 0.2.8
29.4+0.4
36.0+2.
31.4
49.0
5.2
Single-proj
68.8
44.8
34.4go
84.5.0a
58.1
31.010a
38.1
34.7
49.3
5.5
Sepurato-proj (Ouns)
71.2
47.2
35.
81.2go.
57 37.1ga
30.410a
38. 7837
37.1
50.2
6.4
SigLIP. In addition to the standard CLIP model, we also evaluate the performance of SigLIP [21]. We adopt the SigLIP model pre-trained WebLI data [78]. The adopted visual encoder is
ViT-large [53], the patch size is 16, and the input image resolution is 256. The model weight is also loaded from the OpenCLIP repository.
Sup For vision-only models, we first select a ViT-large model [53] trained in a supervised way ImageNet-21K dataset [79]. As it is vision-only model, we only run few-shot and non-species classification tasks with it. The patch size is 32. The model is publicly downloadable in Hugging Face.
DINOv3. Besides supervised pre-training, we also evaluate the performance of DINOv3, which is pre-trained in unsupervised way [22]. The backbone architecture is ViT-large, and the patch size is 16. The model can be downloaded from Hugging Face, Similarly, we only run few-shot and non-species classification evaluations with DINOv3.
Trove-C--LLIP. The above four models are trained on general knowledge covering a variety of topics. We also compare BIOCLIP 2 with domain-specific models. Bio trained with Bioerrove [14]. The model weights can be downloaded from Hugging Face. Among the provided three models, we use the model initialized with OpenAl CLIP [12] that yields the best average accuracy [14]. The visual backbone is ViT-base with patch size of 16.
BIOCLIP. BIOCLIP is trained on TREEOFLIFE- 10M. It adopts ViT-base as the visual encoder, with patch size of 16. The model weight is publicly available in Hugging Face.
F
More Empirical Observations
In this section, we present more empirical observations of the training design, detailed numerical results, and qualitative analyses.
F1 CLIP Training Data Replay
Together with the contrastive supervision of hierarchical labels, we also introduce the replay of CLIP training data to retain the understanding of general knowledge. For each experiment of different training scales, we randomly select subset from LAION-28 with 10%-20% of the corresponding biological image total. Specifically, for the largest run 21 214M biological images, we select 26M samples from LAION-2B. Each training batch consists of 69,312 biological images and 8, 192 replay samples. However, the text labels in T'REFOFLIFE-2000 names, which have a distributional gap from the CLIP training data. Therefore, we apply a separate visual projector specifically for the replay data to avoid the optimization conflict. Other than this difference, the biological data and replay data share the same visual backbone and text encoder.
are primarily different forms of taxonomic
We evaluate different replay settings quantitatively in Table 6. The "No Replay" row shows the baseline performance applying biological contrastive training on top of the pre-trained CLIP model, where a 5.2% performance improvement is achieved. When the replay data is added, which shares the visual projector with biological images, we observe a conflicting performance change. On Herb. 19 [19] and PlantDoc [20], more than 2% improvement is acquired. However, the species classification accuracy on Rare Species drops by 1.3%. We attribute the inconsistent performance change to the distribution gap between biological and replay text labels. After applying separate
24

---

Training Samples
(CLIP)
1M
10M
50M
214M
2D Distribution
65
22 ground finches
60
34 cactus finches
22
65
6 GR
tree finches
warbler finches
vampire finch
Cocos finch
vegetarian finch
Figure 7: The t-SNE distribution of Darwin's finches under different scales training data. Different colors represent different groups of finch species. As the training data scales up, the embeddings form biologically meaningful clusters that align with the phylogenetic tree and their functional traits.
visual projector for the replay data, we observe overall improved performance across multiple benchmarks. At the same time, also notice that replay has limited influence benchmarks like AwA2 [17]. Additionally, we evaluate the preservation of general knowledge understanding on AIQuIRE-Rerane [57]. Applying contrastive training with taxonomic labels slightly hurts the performance, while the single-projector replay fails to retain the understanding. Comparatively. the adopted separate- separate-projector replay improves the performance by 2.1%. Given that there is still distribution gap between the taxonomic labels and natural language, we do not treat INQUIRE as one of our main focuses in this work.
F.2 Qualitative Analyses
Embedding distribution of Darwin's finches. In Figure 1, we show that the embedding distribution of Darwin's 's finches aligns with the beak size. Here we further visualize the distribution under different training scales in Figure 7. Based on the genome- based phylogeny, warbler finches were the most ancient branches, while tree finches and ground finches form the recent branches [23]. Among these species, warbler finches have the smallest beak, convenient for extracting tiny arthropods from leaves. Comparatively, ground finches have larger beaks, which are more suitable for cracking seeds and nuts. In the original CLIP embedding space, the warbler finches and tree finches mixed. As the training data scales up, these two groups are separated, and the relative geometric relationship of all the finches aligns with their phylogeny tree. While the species separation is induced by taxonomic supervision, BIOCLIP 2's embedding space again illustrates emergent higher-level biological meaningfulness after extensive training.
Embedding distribution of Sex data. Similar to Figure 5, we visualize the 2D and 3D distributions of embeddings from 3 species of the Sex data in Figure 8. We can draw conclusions similar those obtained from Figure 5. When no training data is incorporated, the embeddings extracted by the original CLIP visual encoder (leftmost sub-figure) demonstrate large portions of overlap between male and female images. After extensive vision- Lanion- contrastive training, the embeddings present clear decision boundaries between the two variants. Furthermore, as evidenced in the 3D distribution, the embeddings within each species form more compact clusters when projected onto the species span (the gray plane). Comparatively, instead of being eliminated after contrastive training, the intra-species variations are embedded in the subspace orthogonal to the inter-species differences. The extensive training facilitates BIOCLIP to acquire a biologically meaningful embedding space, highlighting its value in serving as biology foundation model.
Embedding distribution of PlantDoc data. We further visualize the distribution of 6 classes from the PlantDoc dataset in Figure 9, When no training is processed (leftmost sub-figure), embeddings of different species, as well as diseases, are mixed. As the training scale increases, we observe two trends. First, the margin between different species is enlarged, and the embeddings belonging to the same species are clustered together. Second, the diseased leaves are easier to separate within each species, although not explicitly constrained during training. More interestingly, the embeddings of healthy apple leaves are distributed close to the healthy blueberry leaves. These observations again highlight the biologically meaningful embedding space of BIOCLIP 2 after extensive training.
25

---

Training Samples
(CLIP)
1M
10M
50M 214M
2D Distribution
3D Distribution
p=0.87
P=0.78
0=0.72
p=0.60
0.6.62
Male Images
Female Images
Figure The embedding distribution sex variations under different scales of training data. The distributions are obtained using t-SE. For the 3D distributions, we first run SVD with the mean embedding of each species. The first two singular vectors are used to construct the gray plane that captures most inter-species differences. The embeddings are then projected into the 3D space with additional orthogonal dimension. The straight lines point from the scales up. the intra-species variations are preserved in subspace orthogonal to inter-species differences. Orthogonality improves with data scale, as evidenced by the decreasing explained-variance ratio
embedding of male images that of the female images. As the training data
Training Samples
(CLIP)
1M
10M
50M
214M
Distribution
20
Apple Scab leaf Apple leaf
Apple rust leaf
Bell pepper leaf
Bell pepper leaf
Blueberry leaf
Figure 9: The I-SNE distribution of 6 classes from the PlantDoc dataset, including three species and three different diseases. As the training data scales up, not only are the species better separated, but intra-species variations also form clusters, making them easier to separate.
F.3 The decay of FDR numerator/dennominato along the data scale.
We observe that FDR between intra-species variation classes increasing as the training data scales up. More specifically, we look into the numerator term (the difference between two class centers) and the denominator term (the variation of features). We visualize the curve of the numerator and denominator terms in Figure 10a and Figure 10b for life stage data and sex data, respectively. The y-axes are scaled the same maximum ratio the minimum value. As the number of training samples scales from 1M to 214M. the denominator term goes through a larger decay than the numerator term. The numerical results further support the increasing FDR trend.
F.4 Camera Trap Results
In addition the species classification benchmarks adopted in [13], we further introduce balanced camera trap image benchmark for species classification, IDLE-OO Camera Traps, derived from LILA-BC datasets [62], construct a more realistic application scenario. Specifically, we select five datasets from LILA-BC that are labeled to the image level to avoid testing on noisy images those labeled as containing an animal when simply the animal's 's habitat. The Island Conservation Camera Traps [63] were of particular interest for their stated purpose of assisting in the prevention of endangered island species' extinction and varied ecosystems represented. This provides complement to the Rare Species test set [13, 59]. The Desert Lion Conservation Camera Traps dataset [64] similarly intended advance conservation efforts. With the Orinoquía Camera Traps [65], Ohio Small Animals [66], and ENA24 [67] camera trap datascts, test camera
26

---

0.00020
0. 0.00050
0.006
2,00035
Numerator
0,00009
Denominator
0. 6.00045
Delto monommaato (feature VaIlatI vallation)
Denomina atott (reature Vertturvvriiahoo vaniation))
Namerator (center distance)
Numerator (center distance)
0,0 0,00008
0. 0.0040
9,00025
0.004
0,03037
0. 0,00035
0.00006
0 0.00000
0.003
0,0 0,03005
0. 00022
0,00034
0. 0.00020
Numerator
Denominator
0.00003
0.
0.00015
0.0011
Number bember of training samples
Number of training samples
(a)
(b)
The decay curves for the numerator (difference between two class centers) and the denominator (feature variation) terms of the FDR metric along the incrcasing data scale. (a) The curves of life stage data. (b) The curves of sex data.
Figure 10:
trap images across varied settings. The Island Conservation set uses common names, while the remaining datasets are reduced to just those that are labeled to the species level, and then sampled to create balanced test set across the remaining classes (these are evaluated scientific names), The sampling and image manifests are provided in IDLE-OO Camera Traps. We report the detailed accuracy each camera trap dataset in Table 7.
G Discussion on the Relationship with Neural Collapse
Neural collapse is a status where the intra-class variations collapse to zero, and the embeddings col- lapse to their corresponding class prototypes [80, 81). The class prototypes collapse to vertices of simplex Equiangular Tight Frame (ETF). It has been empirically observed and theoretically proved that the commonly used loss functions- iiciluiiinnn cross-entrope loss and supervised contrastive loss [82]-lead to neural collapse at the terminal phase of training [24, 25, 83, 84]. If neural collapse happens, the intra-species appearance variations will be hardly separable. However, we reveal that after extensive training of BIOCLIP 2, the intra-species variations preserved in the subspace orthogonal to inter-species differences, indicating that BIOCLIP 2 does not suffer neural collapse.
Why BIOCLIP 2 does not lead to Neural Collapse. We summarize two key reasons that allow the existence the intra-species variation subspace in BIOCLIP
First, the class prototypes in standard classification tasks and supervised contrastive learning (SCL) can both be treated as fully trainable parameters [82]. In standard classification, the prototypes are the weights of the linear classifier. In SCL, they are simply the mean embedding of the corresponding classes. The class prototypes will form a simplex ETF after extensive training. Comparatively, the adopted contrastive training scheme in BIOCLIP 2 employs text embeddings of the taxonomic labels as class prototypes. During the creation of taxonomic labels, hicrarchical structures have been naturally embedded into them based ecological and functional evidence. Different species can share higher taxonomic levels, which makes them hard negatives. Therefore, even the text encoder is also being trained, the generated prototypes will not become ETF.
Second, TREEOFLIFE- 200M poses an enormous label space, where 952K classes are involved in training. Typical SCL is performed upon CIFAR or ImageNet-level datasets, consisting of 100-1.00000 1,000 classes. Previous works analyzing neural collapse usually assume the dimension embeddings is larger than the class number and the sample number is balanced across different classes [25, 81]. In contrast, the class number involved in BIOCI.IP is much larger than the embedding dimension (768 channels). These conditions restrict the possibility of prototypes forming ETF.
27

---

Table 7: Zero-, one-, and five-shot species classification accuracy of IDLE-OO Camera Traps from Labeled Information Library of Alexandria: Biology and Conservation (LILA-BC) [62, 63, 64, 65, 66, 67] for different models. Bold and underlined entries indicate the best and second best accuracies, respectively.
LILA-BC
OH Small Animals
Desert Lion
Orinoquia
BNA24
Island
Model
Mean
Random Guessing
3.1
5.0
3.2
3.6
2.6
3.5
Zero-Shot Classification
CLIP (ViT-L/14)
35.2
38.2
27.1
25.6
21.6
29.5
SigLIP
46.9
41.0
30.0
31.9
20.5
34.1
BioTrove-CUu CLIP
9.7
10.4
14.5
8.0
12.4
11.0
BIOCLIP
17.2
42.5
19.0
27.1
23.1
31.8
BIOCLIP 2
58.8
68.5
42.3
47.9
51.5
53.8
One-Shot Classification
CLIP (ViT-L/14)
37.612.8
35.9
44.7
31.312.7
30.7
36.0
SigLIP
41.1.118
39.1
48.512.9
32.711.6
27.612.5
37.8
Supervised-IN21K
32.412.
28.4+332
40.1
26.3a2.4
26.0a24
30.6
DINOv3
48.1
38.5
52.3
40.0+2.6
36.1
43.0
BioTrove-CLIP
30.5+1.7
30.5
37.942.1
29.042.6
28.3
31.2
BIOCLIP
39.942.2
34.4
45.7+3.0
27.544.7
27.5+22.
35.0
BIOCLIP 2
54.311.1
48.622.3
49.542.5
43.112.1
45.023.0
48.1
Five-Shot Classification
CLIP (ViT-L/14)
18.3
57.842.6
66. +2.4
43.942.
43.541.6
53.9
SigLIP
64.34222
60.0
71.6
46.2+1s
42.641.7
56.9
Supervised-IN211
51.3+2.3
48.5=2.
60.5+3.0
39.3+3.4
41.941.6
48.3
DINOv3
66.3
63.2
74.8
59.943.
53.8+24
63.6
Bio CLIP
47.643.0
46.7
62.2+1.1
41.1+1.
41.84224
47.9
BIOCLIP
62.541.7
57.0=a.2
57.0
63.2+3.8
44.64117
44.040.6
54.3
BIOCLIP
73.4+2.
73. 20.9
70.741.6
59.442.8
61.8
67.7
H Biological Visual Evaluation Details
Instead of training specialist models, we treat the evaluated models as frozen visual embedding extractors. Standard machine learning algorithms are applied on top of the acquired visual embeddings to predict the corresponding labels. Such a design is adopted to evaluate the quality of the embeddings while avoiding the influence of complicated optimization loops. the following, we introduce the details of the adopted benchmarks and the evaluation algorithms. All the experiments are conducted with NVIDIA A100 GPU, and the running time for each task within 30 minutes.
FishNet. FishNet focuses on recognizing, locating, and predicting species and their functional traits [16]. Specifically, 94,532 images are collected with annotations of habitat, ecological role, and nutritional value. In this work, we mainly focus on the prediction of habitats and ecological roles, involving groups of binary labels (e.g., whether the fish can live freshwater). Following the practice in the original paper, we train a two-layer linear classifier with binary cross-entropy loss to predict the labels. We count correct prediction only the labels are predicted correctly for the sample. The original train-test split is adopted, where 75,631 images are used in training, and the remaining 18,901 images are used for testing. This task evaluates whether the embedding distribution of different species is aligned with their ecological relationships.
28

---

NeWT. NeWT comprises 164 binary classification tasks in the natural world [18]. The tasks include appearance, gestalt, context, counting, and behavior concepts. In each task, 50-100 images are assigned per class per train-test split. After extracting visual embeddings, we apply support vector classifier for each of the tasks. The average accuracy is reported across all the evaluated tasks.
AwA2. AwA2 consists of 37,322 images of 50 animal classes, with annotations of 85 numeric attribute values for each class [17]. The dataset can be used for testing the capability of attribute- based classification and zero-shot transfer learning of trait prediction. In this work, we mainly focus on the transfer learning scheme. The training set of 45 classes used to train an attribute classifier, and the remaining 5 unseen classes are used for testing. Similar to FishNet, we incorporate linear classifier on top of the extracted features to predict the binary labels for the 85 attributes. The average Fl score over all the attributes is reported for this benchmark.
Herb. 19. Herbarium is task for discovering new species [19]. Specifically, given the images of known and unknown species, the model is required to predict the labels for both of them. As there are no fixed labels for unknown species, the task is implemented with form of semi-supervised clustering [85]. Given the total number of species, a semi-supervised K-means algorithm is conducted on top of the extracted embeddings to cluster images. Clustering accuracy is calculated for the predictions following the original practice [85].
PlantDoc. PlantDoc is a dataset targeting the incorporation of computer vision for scalable and early plant disease detection [20]. 2,598 images of 13 plant species and up to 17 classes of diseases are collected in uncontrolled natural settings. We evaluate the model on PlantDoc in few-shot learning style. One image per class is randomly selected from the original training split as the support set, and SimpleShot [86] is employed to predict the class labels for the test set. Accuracy over all the testing samples is reported the performance.
Life HageeDiif/AAlin. We use the data of Age tasks from NeWT [18] to construct the Life stage- Diff/Align benchmark, where images are labeled with juvenile and adult classes. Specifically, the differentiation tasks aim to separate the binary appearance variations within each species. Conversely, in the alignment task, we train species classifier with juvenile images while testing it using adult images. It requires the embeddings of two variations within one species to be closer than the embeddings of different species. For both of the tasks, we incorporate support vector classifier to predict the corresponding labels. Ideally, after extensive contrastive training, the embeddings of different variation classes expected to collapse to the species prototype. However, demonstrate that the intra-species variations are well preserved in the embedding space of BIOCLIP 2.
Sex-Diff/llign. NABirds consists of 48,000 images from 400 bird species [61]. Sex and life stage labels are provided for those species with large appearance variations. We manually examine the images and select 81 species with male-female differences to construct the Sex- Sex-Diff/Align benchmark, where 13.624 images are used in total. For the differentiation task, we filter out 20 male images and 20 female images per species as the test set. Among the remaining 10,384 images, we filter out at most 20 male and 20 female images for training. For the alignment setting, we use the images of female birds to train a species classifier and use the male images for testing. Similar to the life stage benchmark, we use a support vector classifier for both of the sub-tasks.
I Data Processing Details
In this section, we provide more details on the data curation process.
L1 Taxonomic Standardization
When GNVerifier returns result from a query, our taxonomic alignment package combines it with the input taxonomic hierarchy, along with the query parameter, to form a resolution attempt. The query response, based on the most specific taxonomic term available in the input data, along with the remaining input hierarchy (entry's 's resolution attempt), is then matched against pre-defined profiles. The algorithm uses these three components to fit against the series of profiles to determine whether a confident resolution found or if alternative query strategy is needed (such as using a different query term or data source), iterating until match is made alternative approaches are exhausted.
29

---

899,976
Tre0OfLife-200M Images
TreeOfLife-200M Biodiversity
INaturalist Source)
INaturalist Source)
134,900,000
356,362
350,229
37,455
5,150,850
5,258,477
1,418
18,711
FathomNet
BIOSCAN-5M
EOL
GBIF
FathomNet
BIOSCAN-SM
BOL
GBIF
Figure 11: The number images and unique 7-rank taxa by core data provider in TREEOFLIFE-2006 iNaturalist [42] is the largest GBIF data source (by image count) and the data provider used by BioTrove [14], so is included here for reference.
Table 8: Top taxa contributors in GBIF [38]. Here, "Distinct Taxa" is used to describe 7-rank taxonomic labels only provided to GBIF that publisher, while "Total Taxa" is the total number of unique labels within that publisher. Observe that, following iNaturalist [42], the next three most diverse sources are museums, which, together, account for more distinct taxa than iNaturalist alone, despite having significantly fewer images overall.
Publisher (GBIF Data Source)
Distinct Taxa
Total Taxa
Images
% GBIF Taxa
INaturalittorrg
81, 671
350, 229
134, 877, 019
9.07%
Natural History Museum
38, 535
148, 637
6,603, 256
4.28%
Museum national d' d'Histoire nat.
37, 201
258, 766
6, 185, 177
4.13%
Naturalis Biodiversity Center
25, 421
244, 697
5, 282, 428
2.82%
Table 9: The distribution of unique taxa and the covered images at different taxonomic ranks in TREEOFLIFE- 200M. 92. 22% images TREEOFLIFE-200M have taxonomic labels specific to the species level.
Rank
Total Images
Unique Taxa
Percent of Dataset
Species
297,382,624 628
967,455
92.26%
Genus
206,160,396
135,380
96.36%
Family
207,489,
13,790
96.99%
Order
210,063,488
1,683
98.19%
Class
211,236,966
382
98.74%
Phylum
212,416,362
127
99.29%
Kingdom
213,932,022
11
100.00%
If all attempts ultimately fail to match profile, the original input data is used for the entry's label in the final output. After hierarchical resolution, common names are annotated. For each entry, the common name corresponding to its most specific resolved taxonomic term is selected, when available, using the preferred English vernacular names from the GBIF Backbone Taxonomy [87]. In the output, the proportion of records changed for each data source is EOL: 98. 7%, FathomNet: 16.0%. BIOSCAN: 11.8%, and GBIF: 0.3%. The notably low modification rate for GBIF reflects our taxonomic alignment package's preference for the GBIF Backbone Taxonomy as its primary reference. After standardization. we summarize the image number and taxa number from each source in Figure 11. We also report the distinct taxa and image number provided by top publishers from GBIF in Table 8. We provide the source code for this part in TaxonoPy.
As claimed $3.3, not all taxa are specific to species level. We summarize the detailed taxa distribution in Table 9. Although some ambiguous labels were mapped to higher taxonomic ranks, our overall dataset still predominantly comprises images confidently labeled at species level. Specifically, out of the total 214M images, approximately 92.26% retain species-level labels.
30

---

1.2 Image-quality Screens
At download, all images were checked to be sufficiently large (224 pixels or more on the shortest side), and resized, where needed, so that the largest side does not exceed 720 pixels. The code, support set class embeddings, and further details of the image-quality screens and the subsequent duplicate control are provided in our dataset repository, TreeDfLife-tool1
1.2.1 Museum Specimen Processing
Museums often consider multiple specimens of the same species to be connected instead of differenti- ating them (as they are often collected from the same or similar location). Thus, these occurrences often include images of their metadata, duplicates of the same or different specimens under one occurrence ID, or less informative images. Some common complications to consider when working with museum specimen images that influenced our processing are that
1. Plant and fungi specimens may be stored in envelopes or folders. These are often photographed and digitized under the same occurrence ID as the image of the specimen itself, thus creating extraneous images we do not want to include in training. They also sometimes pair this with living specimen images (which would be worth keeping as well). See Figure 12, which demonstrates variety within single occurrence for fungi specimens. Plant specimens have secondary confounding factor in that they are often pressed for preservation with their metadata on the page (see Figure 13).
Fish, worms, and similar will be stored in jars. single jar is considered an occurrence, so only one specimen may be photographed perhaps at multiple peotographed-peehapss
jar may be photographed, multiple specimens photographed, etc. In any of these cases, the images will all be labeled with the same metadata that does not include this context.
For animal specimens, both of the above cases may occur: there may be simply a close-up of the tag (similar issue to envelopes with plants and fungi). There may also be multiple specimens photographed within the same occurrence (as noted with about fish and worms). Museums often consider multiple specimens of the same species to be connected instead of differentiating them. This is two-fold, in that they are generally, collected from the same or similar location at about the same time, and specimen is kept representative of its species, so there is not clear need for distinguishing between them. We also see multiple views of the same specimen within occurrence. Examples in Figure 14.
In order to appropriately separate these images, we treated them as museums do, specifically by first dividing them into 11 collection areas (Fungi, Insect, Invertebrate Zoology, Microbiology, Plant, Uncategorized, and five classes of Vertebrate Zoology: Amphibians, Birds, Fishes, Mammals, Others) inspired by the Smithsonian Institution's 's categorical subdivisions for their biological museum collections. From here, we further divided each category based on its image type (c.g., fossil or preserved specimen, specified in GBIF metadata).
For each museum specimen category, we manually curated a small "support set" of representative spec- imen and non-specimen images. We embedded these examples using CLIP VIT-L/144336pp [7]; we chose not to use BIOCLIP since museum specimen labels were not filtered from its training data (e.g., EOL contains many museum specimen images). To capture intra-class diversity, ran K-Means on each support set and retained the resulting cluster centers. During classification, we L2-normalized both the input image's embedding and each center, then assigned the image to the nearest center in Euclidean space. This processing was applied to all museum specimen images identified within GBIF. The support set embeddings are included in TreeDfLife-tooll
I.2.2 Camera Trap Images
Some occurrences include volumm camera-trap sequences, with up to 10,000 images. These occurrences have a single taxonomic label applied across all images, though there are different taxa (e.g., the first few frames may have a duck, while later images have swan, another a goose, but the label for all is duck). To reduce the risk of introducing such noise while still capturing relevant biodiversity, we filter the dataset to include only occurrences with 15 images or fewer. We then use MegaDetector [47, 48] to filter "empty" frames. In creating a dataset to train a foundation model on the entire tree of life, it is prudent to avoid introducing too many plant images labeled as animals.
31

---

10002004
10002004
COPALLO TOADH
TUADER
8801
49076
(a)
(b)
Figure 12:
These images belong to GBIF Occurrence 2301948912 188], and thus share metadata. (a)
Contains the specimen (Caloplaca cerina), (b) contains the file metadata from the specimen folder; only (a) should be retained. Collected in Canada by @Canadian Museum of Nature (licensed under CC BY-NC
1.2.3 Citizen Science Images
Similarly, citizen science occurrences may contain many images assigned single label. For each occurrence, we thus embedded the images using B1OCLIP [13], standardized them, calculated the pair-wise cosine similarity, and used the mean to indicate the occurrence's "distinctness" In these instances, we identified three broad categories into which to further subdivide them for reduction:
1. Mixed occurrences: multiple species all labeled as just one of them (low similarity). These were treated as noise and discarded.
2. Multiple image occurrences of the same species: observations suggest these are creature close- ups and environment images under a single occurrence. They may also be larger groups with close-ups. For these, we randomly sample down to 5 images.
Camera trap images: likely images collected backyards (high similarity). Occasionally, people upload images from camera traps to citizen science platforms like iNaturualist (also ex: MammalWeh citizen science platform for camera trap images). These are processed under camera trap protocols described above.
The final step our quality control processing pipeline was to filter out identifiable images of people from the training data. This was done by running MTCNN [49] on all images downloaded from GBIF and EOL.
I.3 Duplicate and Leakage Control
GBIF and EOL are large biodiversity data aggregators, sourcing images that are also used in biological benchmarks such as iNat21 [36] and NeWT [18]. Due to metadata provenance complications, the images these test sets cannot easily be matched their sources or copies downloaded from other sources. To prevent the introduction of test data sourced from iNaturalist or "other sources online", we perform two content deduplication steps. The first is to run MD5 hashes on all downloaded images. During our download, these hash sums are used to distinguish images that have already been downloaded. We record the hash both the original image and our resized image. These were used to ensure that Rare Species [59] images (sourced from EOL) were not included in the training data.
32

---

(a)
(b)
Figure 13: Both images contain specimens included in the dataset, their dominance frame quite different. The pressed flower sheets are the same size, hut (a) Osmorkiza longisrylis is dominant in frame, while (b) Chara globularis covers less pixel area than the text. These images belong to GBIF Occurrence 5132787320 and GBIF Occurrence 513583109 1095 [88], respectively, and were both collected in the United States of America by The New York Botanical Garden (licensed under CC BY 4.0).
M.M. 1948
PP
(a)
(b)
(c)
(d)
Figure 14: These images belong to GBIF Occurrence 1056329684 [89], and thus share metadata. (a) through (c) all contain the specimen (Pristimantis zeuctotylus), though (c) is primarily focused on its label or tag. Meanwhile, (d) is the label for another specimen of the same species from the collection; there are multiple views of this specimen well. (c) provides alternate view and can be retained, while (d) should be removed.
Traditional hash
are highly sensitive small changes-a one-pixel difference between two images will produce different hash. Hence, we applied perceptual hashing [PDQ 50], with distance less than 10, identify training images that may be in our desired test sets that could not otherwise be filtered out (i.e., by MDS hash sum or through metadata). Note that PDQ hash evaluation was only run on GBIF citizen science images and EOL images sourced from Flickr since they are not reliable for museum specimens.
IUCN Red List coverage. According the most recent IUCN Red List assessment[ [22] TREEOFLIFE-200M contains images of 69.5% (55,512) of all IUCN-assesee species in cate- gories characterized as rare species or data deficient. This coverage was determined by applying our taxonomic alignment package to the IUCN taxonomic data to enable direct comparison with our standardized dataset. TREEOFLIFE 200M demonstrates particularly strong representation of threatened species, with 77.1% coverage across threatened categories (36,370 species), including 79.7% of Vulnerable species (14,038), 79.4% of Endangered species (15,190), and 68.4% of Criti- cally Endangered species (7,142). Coverage extends to 81.7% of Near Threatened species (8,073) and 82.7% of species classified as Extinct in the Wild (67). Data-deficient species have lower representation at 48.4% (11,002), likely reflecting the challenges in imaging and identifying the
33

---

species in this group. Notably, these species are designated in this category because there is not sufficient information about them for IUCN evaluate their status; only 8% of the 2.14M described species have been evaluated [51]. Thus, including 48% of these species in SFFFOOLLIFF-20000 along with the threatened species coverage, establishes the approach to integrating diverse data sources used in TREEOFLLIFE-20OM as a valuable resource for conservation research, providing visual representation for a substantial majority of species prioritized for global conservation action.
J Author Contribution Statement
Jianyang Gu led the research project, ran the experiments, conducted analyses, and wrote the major part of the manuscript. For the dataset processing, he built the face detection pipeline to remove images with eecognizalleiidnnfffbbe people. He also retrieved the common names from GBIF backbone taxonomy alignment to TaxonoPy output.
Samuel Stevens constructed the initial benchmark for non-species classification tasks and signifi- cantly contributed to the paper writing. In addition, he also developed the content- -bsed de-duplication pipeline based on PDQ-hash to identify test images that were contained in training data.
Elizabeth G Campolongo analyzed, evaluated, planned, organized, and documented the dataset effort. She developed the data processing approach by categories and supervised the tool development to retrieve and organize the dataset onto the HPC file system and of dataset curation. She curated camera trap test sets for benchmarking and Darwin's 's Finches for embedding analysis. She also contributed significantly to the dataset part of the paper writing.
Matthew J Thompson planned, organized, and supervised the development of tools to retrieve and organize the dataset onto the HPC file system. He developed and executed the taxonomic standardization (TaxonoPy) and webdataset ML-ready format conversion. He also contributed to the paper writing of the dataset part.
Net Zhang played an essential role in cleaning the noisy data. He dealt with the museum specimen images, citizen science images, and data de-duplication. Besides, he also processed and analyzed the downloaded data, primarily focusing on GBIF.
Jiaman Wu provided detailed advice on designing the training experiments for BIOCLIP 2. She also contributed to the dataset processing by initiating specimen label filtering and informing the taxonomic standardization for hemihomonym cases.
Andrei Kopanev developed the distributedd-aee btwerbuuddddddmodeeeee..e. tool retrieve and organize the original images in the HPC file system and converted the dataset into ML-ready format (webdataset).
Zheda Mai provided constructive advice designing the experience replay of CLIP training data.
Alexander E. White provided expertise in consultation on the museum specimen image processing approach.
James Balhoff and Wasila Dahdul provided expertise in consultation on the algorithm for taxonomic hierarchy resolution relevant to Taxonopy development.
Daniel Rubenstein and Hilmar Lapp were involved in regular meetings and gave valuable feedback on results and experiments.
Tanya Berger-Woll Wolf and Wei-Lun Chao provided insightful discussions and directions throughout the development of the project. They offered constructive advice for both the training design and the model evaluation. They also provided detailed comments on the manuscript.
Yu Su is the senior lead that oversaw the project and was involved in every aspect. He conceived the idea of observing emergent properties from scaling hierarchical contrastive training. He gave insightful guidance in formalizing the research question and directions verify the capabilities of BIOCLIP 2. He provided critical feedback and helped shape the manuscript.
34


Reviews:
[{"review_id": "mnG42l7TB6", "reviewer": "Reviewer_A1KT", "summary": "The authors introduce Bio-CLIP-XL, a powerful CLIP model trained on the mega-scale TREEOFLIFE-200M dataset, comprising 214 million image-text pairs of living organisms from sources including GBIF, BIOSCAN, and EOL. By achieving state-of-the-art performance on multiple benchmarks, this work represents a significant contribution to the machine learning community. The paper also provides insightful discussions on the model's training strategy to avoid neural collapse and its resulting emergent properties.", "questions": "1. This work represents a significant contribution to morphology-based species identification using deep learning. In modern biology, however, there is a strong trend towards using genomic sequencing (e.g., DNA barcoding) for this task, often seen as providing a more direct \"ground truth\". Could the authors elaborate on the specific scientific and practical advantages of their image-based approach? In what scenarios is a model like Bio-CLIP-XL superior to, or a necessary complement to, genomic methods for understanding biodiversity?\n\n2. Could the authors clarify the composition of the 5 million images from the BIOSCAN dataset? The supplementary material suggests all 5M images were used, but the original BIOSCAN[1] dataset documentation indicates only ~473,000 images have expert-verified morphological labels, with others labeled via DNA barcoding.\n\n\n   2.1. What was the labeling methodology for the ~4.5 million images that lacked expert morphological IDs? \n\n\n   2.2. Could you also explain the reduction in class count from the 22,622 species in the source data to the 18,711 classes used in your final dataset?\n\n3. Back to weakness 3. I am thinking a possibile case it could become the strength. What if, the text embeddings of different names are too similar in Bio-CLIP-XL may suggesting a taxnonmy merger? And this merge is happened in academia. If this is confirmed, the model can be a guide for taxnonmy revisement.\n\n[1]. Gharaee, Z., Lowe, S. C., Gong, Z., Millan Arias, P., Pellegrino, N., Wang, A. T., ... & Chang, A. (2024). Bioscan-5m: A multimodal dataset for insect biodiversity. Advances in Neural Information Processing Systems, 37, 36285-36313.", "limitations": "See weakness.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. Scale and Diversity of the Training Data: The assembly of TREEOFLIFE-200M is a major achievement. \n\n2. Robust Model Training and State-of-the-Art Performance: The paper demonstrates a robust and technically sophisticated training strategy, effectively leveraging large-scale computational resources. The resulting model, BIOCLIP-XL, sets a new state-of-the-art on a wide range of biological benchmarks, showcasing its capabilities as a powerful foundation model for the life sciences.\n\n3. Avoided the Neural Collpsing often seen in constrast learning on fine grained data, and provided insightful discussion.\n\n4. Insightful Theoretical Framework: The theoretical analysis, particularly Theorem 5.1, provides a compelling mathematical justification for the observed preservation of fine-grained features. It elegantly explains how the model learns to represent different factors of variation (species identity vs. intra-species traits) in orthogonal subspaces, providing a solid theoretical foundation for the empirical results.\n\nWeakness\n1. Questionable Foundation of Taxonomic Standardization: The decision to standardize all taxonomic labels against the GBIF Backbone Taxonomy as a single source of truth is a significant concern from a biological standpoint. While pragmatic for automation, GBIF is a secondary aggregator, not a primary taxonomic authority. Biological classifications are dynamic and constantly revised in peer-reviewed literature. By anchoring the entire dataset to a single, potentially lagging source, the model may have learned outdated species concepts, which could lead to critical errors in downstream applications (e.g., misidentifying recently split species with different conservation statuses).\n\n2. Potential Mismatch with Core Scientific Needs for ecologists and conservationists: The model's exclusive focus on image identification (\"what\") overlooks the spatiotemporal metadata (location and date) that is often more critical for its target users. For ecologists and conservationists, core tasks like tracking species ranges, modeling habitats, or monitoring invasive species depend on knowing \"where\" and \"when\" an observation occurred. By discarding this essential context, the model's practical utility for a large portion of the biological community is significantly limited. The impact of the work on biology would be significantly increased if the dataset could provide high quality data on where and when the animals are spotted.\n\n3. Fragility of the Text-Embedding-Based Prototypes: The theory relies on class prototypes being defined by text embeddings of taxonomic names. This creates a brittle system that is highly vulnerable to taxonomic revisions. For example, if a species is reclassified (a common occurrence), its text-based prototype could become instantly obsolete. A name change could cause a prototype to point to an entirely different concept, a species merger could cause two distinct prototypes to collapse into one, and a species split would leave the model with no prototype for the newly described entity. This tight coupling of the model's core structure to a non-permanent string of text could be a vulnerability.", "was_revised": true, "final_justification": "I will keep my score of 4. Details in my reply.", "rebuttal": "Dear reviewer,\n\nWe sincerely thank you for the detailed and constructive comments, and are glad that you acknowledge the empirical and theoretical significance of this work. Please find the response below:\n\n***W1. Taxonomic standardization solely based on GBIF backbone***\n\nThank you for this insightful and important critique. We fully acknowledge that relying on the GBIF backbone taxonomy as a single taxonomic reference has potential limitations. As noted by the reviewer, biological classifications are inherently dynamic, and any taxonomy could be out of date with the ongoing scientific updates. Using GBIF provides a practical, scalable approach to unify taxonomic labels across vast datasets, accepting some degree of temporal lag as a trade-off for coverage and consistency.\n\nBuilt upon the unified taxonomy backbone, BioCLIP-XL illustrates capabilities beyond zero-shot classification. As illustrated in the reply to weakness 3, BioCLIP-XL learns generalizable visual features that can distinguish species and even intra-species variations without taxonomy labels. Moreover, as in the reply to question 3, BioCLIP-XL offers an opportunity to cross-validate the current taxonomic splits. Therefore, we humbly believe that the lagged taxonomy update won’t be a fatal limitation to BioCLIP-XL.\n\n***W2. Lacking spatiotemporal metadata in the dataset***\n\nThank you for the important observation. We agree that spatiotemporal metadata are critical for many ecological and conservation applications. However, incorporating such metadata at scale presents several challenges. \n* First, spatiotemporal data is often incomplete or unavailable for many images in large biodiversity datasets. \n* Second, for sensitive species (e.g., endangered or threatened), sharing precise location information raises ethical concerns regarding their protection. \n* Moreover, species ranges are changing. Relying strictly on expected location can limit practical utility in dynamic scenarios, such as poaching detection or invasive species detection.\n\nWith that being said, lacking spatiotemporal information during training does not limit its use for application. BioCLIP-XL is designed as a biological foundation model that can be fine-tuned to suit specific needs, including spatiotemporal context. Additionally, people can acquire a potential species list given geolocation, which has been validated in the field study to facilitate more accurate species classification.\n\n***W3. Dependency on text embedding of taxonomic names***\n\nThank you for raising this important point. However, we would like to clarify that although BioCLIP-XL is trained with taxonomic labels, its learned visual-semantic space is not strictly bound to these specific text embeddings. Importantly, BioCLIP-XL demonstrates strong performance not only in open-set and zero-shot classification but also in few-shot settings. Given only one example image per class, BioCLIP-XL achieves 64.1% average accuracy across the 10 species classification benchmarks, outperforming its zero-shot setting (55.6%). This indicates that BioCLIP-XL does not solely rely on taxonomic labels to predict species. The visual embeddings can be used directly to construct new prototypes or support few-shot adaptation for newly described or reclassified species. \n\nFinally, we note that BioCLIP-XL’s visual representations enable applications beyond species classification. The illustrated visual biological tasks in the paper do not depend on taxonomic labels. Thus, while taxonomic changes are an inherent challenge in biological research, we do not see them as a fundamental limitation for BioCLIP-XL. \n\n***Q1. The relationship between image-based and DNA-based methods***\n\nThank you for this insightful question. We fully recognize that DNA barcoding is a more fundamental way to identify species. At the same time, image-based approaches such as BioCLIP-XL have distinct scientific and practical advantages that make them indispensable in many real-world scenarios.\n* **Scalability and data availability**. \nGenomic data collection is often limited by cost and accessibility. In contrast, image data can be collected at scale from diverse sources, including field cameras, citizen science platforms, and historical archives. For example, our dataset comprises images of 952K unique taxa—a scale that is currently unattainable for DNA-annotated samples, especially for wild populations and rare or elusive species. \n* **Accessibility and non-invasiveness**.\nObtaining DNA samples requires specialized equipment, trained personnel, and often the physical collection or handling of organisms. However, this may not be feasible or ethical in many settings. In comparison, image-based methods allow rapid, non-invasive identification, making biodiversity monitoring more accessible to researchers and the general public. \n* **Complementary information to genomics**.\nAlthough DNA provides ground-truth identification, it does not capture all aspects relevant to biodiversity studies. For example, life stage, behavior, health, or phenotypic adaptations are often visually apparent but not easily inferred from genetic data alone. As demonstrated in the paper, BioCLIP-XL can distinguish between juvenile and adult individuals, complementing genetic approaches. \n\n***Q2. Clarification of the incorporation of BIOSCAN-5M***\n\nThank you for these questions. We clarify the following:\n\n*Q2.1: Labeling methodology for the images without expert morphological IDs*\n\nWe used the labels as provided in BIOSCAN-5M. To ensure quality and consistency, we performed additional cleaning, including resolving residual ambiguous labels such as “Malaise####” by mapping them to higher taxonomic ranks where appropriate. This cleaning was done in close consultation with the BIOSCAN-5M dataset creators, who subsequently updated the dataset to improve label accuracy.\n\n*Q2.2: Explanation for class reduction*\n\nWe are sorry that the discrepancy was not carefully explained in the manuscript. The 22,622 species in the BIOSCAN-5M paper refer to unique species-level entries. In our paper, we measure the number of unique 7-rank hierarchies instead of species. Additionally, we apply taxonomic standardization and alignment across all data providers, which resolves many irregular or placeholder labels. As a result, our resolved dataset includes 18,711 unique classes. We report in the supplementary material, line 64, that 11.8% of BIOSCAN-5M images were changed during the resolution. \n\nWe will clarify these points in the revised manuscript.\n\n***Q3. The potential of BioCLIP-XL as a guide for taxonomy revision ***\n\nIt is an insightful comment. We believe that similarity between text embeddings of different species names in BioCLIP-XL could potentially indicate taxonomic mergers or revisions. \n\nFor example, we observe cases where the text embedding of a mislabeled species (e.g., an insect labeled as a plant) clusters more closely with the correct group (other insects) than with its incorrect label (plants). While this is a different issue from taxonomic merger, it similarly shows how BioCLIP-XL’s embedding space reflects biological and semantic relationships that can guide taxonomy-related hypotheses. \n\nIn all cases, we emphasize that these hypotheses should be validated with DNA barcoding and expert review. Nevertheless, BioCLIP-XL demonstrates the possibility to support taxonomic research by identifying potential revisions such as mergers or misclassifications."}, {"review_id": "CWX5oyOIO4", "reviewer": "Reviewer_awBE", "summary": "The paper applies large-scale contrastive vision-language models to classify a dataset comprising 214 million images of living organisms. The objective is species classification, along with additional biological visual tasks such as habitat classification and trait prediction.", "questions": "I have no further questions.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n- The experiments leverage a large-scale dataset containing 214 million images, which is an impressive data scale that strengthens the validity of the reported results and supports robust model training.\n\n- The study performs classification over 10 distinct tasks, demonstrating the flexibility and broad applicability of the proposed approach.\n\n- The authors systematically investigate the impact of data scaling on model performance across these tasks, providing valuable insights into how increased data size translates into improvements in generalization and task transfer.\n\n- The presentation of ideas and results is well-organized and clear, supported by thoughtfully designed figures and tables that effectively communicate key findings to the reader.\n\n- The paper goes beyond empirical observations by introducing and proving a theorem that explains the ecological alignment across species while separating intra-species variation, offering a sound theoretical foundation for the proposed methods.\n\n- The experimental evaluation includes thorough comparisons against several baseline models on multiple benchmark datasets, helping to validate the effectiveness and superiority of the proposed approach through comprehensive empirical evidence.\n\n**Weaknesses**\n- The classification tasks evaluated in the paper may be considered relatively trivial downstream tasks, as they primarily rely on morphological data (images) to distinguish between organisms and their associated feature attributes. This could limit the broader applicability of the conclusions to more complex or less morphologically distinct domains.\n\n- In the BIOSCAN-5M project, which the authors themselves cite and leverage in their experiments, multimodal data were used in conjunction with a contrastive loss function to perform species classification. However, the current paper does not adequately compare its proposed approach to BIOSCAN-5M, nor does it sufficiently discuss how its architecture builds upon or improves over the BIOSCAN-5M framework.\n\n- The primary novelty of the paper appears to be centered on the scale of the dataset used in experiments. While this scale is impressive, there is a risk that the contribution could be perceived as incremental if the architectural innovations or methodological advances are not clearly distinguished beyond simply scaling up data.", "was_revised": true, "final_justification": "I thank the authors for their rebuttal and addressing my concerns.", "rebuttal": "Dear reviewer,\n\nWe sincerely thank you for the detailed and constructive comments, and are glad that you acknowledge our effort in scaling data and empirical observations. Please find the response below:\n\n***W1. Significance of morphological-based evaluation***\n\nThank you for the thoughtful comment. While BioCLIP-XL is intentionally focused on morphological/visual signals, we do not view the morphology-based evaluation tasks as “trivial.” Fine-grained species classification is inherently challenging due to the subtle visual differences between closely related organisms. While some biodiversity research necessitates more detailed examinations, such as X-ray or genetic sequencing, BioCLIP-XL offers the opportunity to propose hypotheses and a more targeted direction based on the morphological information.\n\nBeyond classifying known species, we also explore the novel and more complex task of new species discovery. Using the TerraIncognita dataset [1] as an example, we evaluate the model to map the previously unseen species to the correct family based on 264K insect species names in the TreeOfLife-200M dataset as the vocabulary. BioCLIP‑XL substantially improves top‑1 family accuracy over strong vision‑language baselines:\n\n| Model | CLIP (ViT-L-14) | SigLIP (ViT-L-16) | BioCLIP-XL |\n| --- | --- | --- | --- |\n| Accuracy (%) | 30.0 | 47.5 | 97.0 |\n\nAlthough assigning the exact name ultimately requires expert review and genetic examination, BioCLIP-XL offers a convenient and accurate way for hierarchical placement. This capability helps narrow the search space, prioritize comparing candidates, and accelerate the discovery. We will revise the paper to include these new experimental results. \n\n[1] Chiranjeevi, Shivani, et al. \"TerraIncognita: A Dynamic Benchmark for Species Discovery Using Frontier Models.\" arXiv preprint arXiv:2506.03182 (2025).\n\n***W2. Comparison with the model in BIOSCAN-5M***\n\nThanks for the question. We agree that BIOSCAN-5M is a valuable resource and a strong prior effort in multimodal species classification. Moreover, BIOSCAN-5M and BioCLIP-XL represent two orthogonal directions to improve biological foundation models—BIOSCAN-5M introduces extra DNA information, while BioCLIP-XL studies the data scaling.\n\nWe agree and believe that the extra DNA information is critical and helpful for accurate species classification. However, as we focus on comprehensive coverage across the entire tree of life, it is challenging to acquire the DNA information for the involved 952K different taxa and 214M images. Therefore, the employment of image and taxonomic labels is based on scalability and compatibility with large-scale data. Consequently, BioCLIP-XL can be broadly applied in a variety of settings where DNA information is not easily accessible, such as historical data, citizen science images, and field observations. We will include this discussion in the revised version. \n\n***W3. Contribution of the paper***\n\nThank you for your question. We humbly believe that “novelty” is not limited to creating new approaches or architectures. Novelty and contributions can come from demonstrating and discovering new insights. In our humble opinion, many recent breakthroughs in AI and machine learning stem from scaling up the data and learning how to train from it. With that being said, scaling up these is by no means trivial.\n\nWe also want to emphasize that TreeOfLife-200M is not a trivial aggregation of multiple data providers. As illustrated in Section 3.2 and supplementary material Section C, we design dedicated pipelines to unify taxonomic labels, filter out defective images, and remove duplicates. These efforts represent technical innovations in dataset construction and are essential to achieve a dataset of this size and the training of BioCLIP-XL.\n\nWith the scaled dataset, BioCLIP-XL achieves significant performance improvement in distinguishing species. Furthermore, this work demonstrates that BioCLIP-XL leads to broader scientific advances. We show that BioCLIP-XL serves as a general-purpose biological foundation model, providing state-of-the-art performance across a range of visual tasks beyond species classification. These properties were formally analyzed in the paper and would not be observable at smaller scales. These analyses reveal that scaling and structured supervision lead to improved and interpretable representations, providing insights for future biodiversity research and broader scientific domains."}, {"review_id": "kT1SahRRux", "reviewer": "Reviewer_2dWB", "summary": "This paper presents BioCLIP‑XL, a CLIP-based model that maps biological images and taxonomic labels into a shared embedding space. Trained with hierarchical contrastive learning on the new TreeOfLife‑200M dataset, it boosts zero-shot species classification accuracy by about 18% over the original BioCLIP. The authors also identify “emergent properties” in the embeddings: species naturally sort by ecological or functional traits, while subtle within‑species differences stay isolated in their own subspaces. Ablation tests and theoretical analysis confirm these effects and explain how they arise.", "questions": "1) Could you include visualizations of the sample distribution at each taxonomic rank to more clearly illustrate the dataset’s imbalance?\n\n2) Given the data imbalance, might you consider reporting class‑averaged or balanced accuracy alongside top‑1 accuracy for a more comprehensive evaluation?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1) The paper improves the hierarchical contrastive learning model based on BioCLIP, effectively improving the model's efficiency in utilizing hierarchical information.\n\n2) The TreeOfLife-200M dataset is proposed, providing high-quality data resources for subsequent biological vision research\nIt is very valuable for in-depth analysis of emergent properties.\n\nWeaknesses:\n\nAside from the dataset imbalance and incomplete theoretical analysis noted in the paper, another potential drawback is the simplicity of the text input: it only uses taxonomic labels (scientific or common names) and does not include ecological or appearance descriptions for each rank. Despite this limitation, due to data availability and missing fine‑grained descriptions, the ablation studies and downstream evaluations show that the model still generalizes well across tasks.", "was_revised": true, "final_justification": "My questions are answered by the authors. I will keep my score of 5.", "rebuttal": "Dear reviewer,\n\nWe sincerely thank you for the detailed and constructive comments, and are glad that you acknowledge our effort in scaling data and empirical observations. Please find the response below:\n\n***W1. Incorporation of ecological or appearance descriptions***\n\nThank you for the insightful comment. We agree that the ecological or appearance descriptions will likely facilitate the model generalization, which have been investigated at a preliminary stage of BioCLIP-XL. However, it is extremely challenging to collect such fine-grained textual information at the scale of TreeOfLife-200M that covers 952K unique taxa. Many species lack Wikipedia pages, and the descriptions often exist only in specialized taxonomic literature or original species descriptions that are not readily accessible in a structured form. Even if the species-level descriptions are available, there are inconsistencies between overall descriptions and instance images (e.g., juvenile and adult images). We view integrating richer ecological and morphological descriptions as an important direction for future work, which we expect will further improve the model’s performance and utility.\n\n***Q1. Sample distribution at each taxonomic rank***\n\nThank you for the suggestion. Due to the rebuttal restriction regarding figures, we provide the image number distribution at different taxonomy levels in the following table. For each taxonomic rank, we report the mean, median, minimum, and maximum image number per taxon. The largest taxon at each rank is also attached following the maximum image number. \n\n| Rank | Mean images per taxon | Median images per taxon | Min / Max images per taxon |\n| --- | --- | --- | --- |\n| Species | 227 | 8 | 1 / 679,902 (Anas platyrhynchos) |\n| Genus | 1,522 | 21 | 1 / 1,333,687 (Carex) |\n| Family | 15,318 | 62 | 1 / 10,810,236 (Asteraceae) |\n| Order | 123,285 | 295 | 1 / 18,279,859 (Lepidoptera) |\n| Class | 549,904 | 221 | 1 / 76,466,776 (Magnoliopsida) |\n| Phylum | 1,672,569 | 236 | 1 / 99,472,339 (Tracheophyta) |\n| Kingdom | 19,448,365 | 184,809 | 70 / 102,591,559 (Animalia) |\n\nThis imbalance reflects the natural prevalence and sampling biases typical in ecological data. We will include comprehensive visualizations of these distributions in the revised manuscript to better illustrate the dataset’s structure. \n\n***Q2. Balanced accuracy***\n\nThank you for the question. While the training data is imbalanced, most of our evaluation benchmarks feature balanced class distributions. \n* The Rare Species dataset has 400 classes, 395 of which have 30 unique images. The remaining 5 classes have 29, 29, 27, 27, and 21 images, respectively. \n* All the test sets from Meta-Album (Plankton, Insects, Insects 2, PlantNet, Fungi, PlantVillage, and Medicinal Leaf) have balanced classes, each with 40 images. \n* The Camera Trap datasets have a balanced class split. Each class in Desert Lion, ENA24, Island, Orinoquia, and OH Small Animals has 11, 56, 10, 12, 12 images, respectively. \n\nThe only exception is NABirds, where the largest class has 120 images, while the smallest one has only 13 images. We thereby report the micro (overall) and macro (class-averaged) accuracy of NABirds below. As expected, macro accuracy is lower across all models due to the imbalance. BioCLIP-XL maintains a significant performance advantage over the other models on all the tasks. We will include the test set information and comprehensive evaluation in the revised manuscript.\n\n| Model/Accuracy | 0-shot | 1-shot | 5-shot |\n| --- | --- | --- | --- |\n| CLIP/Micro | 66.5 | 42.7$\\pm$0.8 | 68.2$\\pm$0.3 |\n| CLIP/Macro | 63.3 | 41.7$\\pm$0.9 | 67.0$\\pm$0.4 |\n| SigLIP/Micro | 61.7 | 39.9$\\pm$0.9 | 64.2$\\pm$0.3 |\n| SigLIP/Macro | 60.0 | 39.1$\\pm$0.9 | 62.7$\\pm$0.5 |\n| BioCLIP/Micro | 58.8 | 57.4$\\pm$1.2 | 78.2$\\pm$0.3 |\n| BioCLIP/Macro | 54.7 | 55.8$\\pm$1.1 | 76.8$\\pm$0.3 |\n| BioCLIP-XL/Micro | 74.9 | 82.4$\\pm$1.1 | 92.4$\\pm$0.2 |\n| BioCLIP-XL/Macro | 72.2 | 80.7$\\pm$1.3 | 91.3$\\pm$0.2 |"}]

Author rebuttal:

Meta-review: This paper presents a new large scale biological organism image dataset (TreeOfLife-200M dataset) and a model (BioCLIP-XL) trained on these biological data with contrastive learning. The paper also presents several analysis and experiments on the learned embedding space of BioCLIP-XL.

All the reviewers are positive about the paper, valuing the new large scale dataset (with more than 2M images), the state-of-the-art performance of the BioCLIP-XL model on different biological tasks, and the theoretical insights discussed in the paper. The Author-Reviewer discussion was active, and the authors provided satisfactory answers to the questions of the reviewers, making the reviewers to keep their original positive score. Regarding weaknesses, there was some discussion on the discrepancy between the needs of biological researchers and the research focus of machine learning researchers. We recommend the authors to add some discussion regarding this aspect in the final version of the paper, to encourage future research that better aligns with the needs of biological researcher. 

I recommend this work for spotlight due to its potential impact in a relatively underexplored area. The work provides a large scale curated dataset, a trained embedding space, and extensive analyses and experiments on the resulting embedding space, providing valuable insights and tools for the research community. Assuming that the authors will release the dataset and the trained model, we believe these resources will open up new directions for future research in biological vision.
Decision: Accept (spotlight)