Title: BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity
Benchmark: BIOSCAN-5M
Summary: BIOSCAN-5M is a comprehensive dataset containing multi-modal information for over 5 million insect specimens, and it significantly expands existing image-based biological datasets by including taxonomic labels, raw nucleotide barcode sequences, assigned barcode index numbers, geographical, and size information.
Phenomenon type: Specific Application (A single use case, where the benchmark is likely to be examples of that use case)
Target phenomenon: Understand biodiversity, based on classification and clustering accuracy of multi-modal biological data
Phenomenon defined by authors: No
Definition scope: Comprehensive
Definition integrity: Single cohesive phenomenon
Definition integrity detail: Not applicable
Phenomenon contested: Contested
Task definition: Three tasks: 
DNA-based taxonomic classification: Use DNA barcode sequences to classify specimens at the species and genus levels.​

Zero-shot transfer learning: Applying self-supervised learning to images and DNA barcodes to generate embeddings, which are then clustered to assess the natural grouping of specimens. 

Multimodal contrastive learning: Training models to learn shared representations across images, DNA sequences, and taxonomic information to improve taxonomic classification and enable cross-modal retrieval.
Task item definition: Image of insect, DNA barcode, taxonomic label (species and genus labels)
Task source: Author-crafted task examples (e.g. hand-written examples, manual transformation of existing data into questions)
Task ecology: Representative task (e.g. answering medical licensing exam questions), Constructed task (e.g. predicting medical diagnoses from clinicians' notes)
Task ecology detail: Representative task: DNA-based taxonomic classification
Constructed tasks: Zero-shot transfer learning, Multimodal retrieval learning
Dataset size: 5,150,850 (5M)
Dataset metadata available: Yes
Dataset metadata detail: Geolocation of collection cite, insect size measurement, timestamp of specimen collection
Sampling method: Convenience sample (creators found a set of tasks that was readily accessible)
Split reviewed: Test
Response format: Short free response (e.g. single word or number), Free response (e.g. summary paragraph, executable code)
Response format detail: Short free response for classification. Free response for multi-modal retrieval
Metric definition: Exact Match (accuracy, F1, precision, recall)
Metric aggregation: Weighted Mean
Metric subscores: Yes
Metric subscores detail: Taxonomy classification accuracy by species and genus
Zero-shot clustering performance by model architectures
Constructive learning accuracy by model modality (DNA-to-DNA, image-to-image, image-to-DNA)
Metric access: Outputs alone
Few-shot evaluation: No
Authorship: Mix (multiple authors from industry and academia)
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No comparisons made
Results realism: No
Human baseline: No
Authors discuss validity: No
Taxonomy: Biology / 
New taxonomy: Specific Application / Physical Sciences