Arboretum: A Large Multimodal Dataset Enabling AI for Biodiversity
Authors: name: Chih-Hsuan Yang; name: Benjamin Feuer; name: Zaki Jubery; name: Zi K. Deng; name: Andre Nakkab; name: Md Zahid Hasan; name: Shivani Chiranjeevi; name: Kelly Marshall; name: Nirmal Baishnab; name: Asheesh K Singh; name: Arti Singh; name: Soumik Sarkar; name: Nirav Merchant; name: Chinmay Hegde; name: Baskar Ganapathysubramanian
Published: 2024-06-25 17:09:54

Abstract: We introduce Arboretum, the largest publicly accessible dataset designed to
advance AI for biodiversity applications. This dataset, curated from the
iNaturalist community science platform and vetted by domain experts to ensure
accuracy, includes 134.6 million images, surpassing existing datasets in scale
by an order of magnitude. The dataset encompasses image-language paired data
for a diverse set of species from birds (Aves), spiders/ticks/mites
(Arachnida), insects (Insecta), plants (Plantae), fungus/mushrooms (Fungi),
snails (Mollusca), and snakes/lizards (Reptilia), making it a valuable resource
for multimodal vision-language AI models for biodiversity assessment and
agriculture research. Each image is annotated with scientific names, taxonomic
details, and common names, enhancing the robustness of AI model training.
  We showcase the value of Arboretum by releasing a suite of CLIP models
trained using a subset of 40 million captioned images. We introduce several new
benchmarks for rigorous assessment, report accuracy for zero-shot learning, and
evaluations across life stages, rare species, confounding species, and various
levels of the taxonomic hierarchy.
  We anticipate that Arboretum will spur the development of AI models that can
enable a variety of digital tools ranging from pest control strategies, crop
monitoring, and worldwide biodiversity assessment and environmental
conservation. These advancements are critical for ensuring food security,
preserving ecosystems, and mitigating the impacts of climate change. Arboretum
is publicly available, easily accessible, and ready for immediate use.
  Please see the https://baskargroup.github.io/Arboretum/{project
website} for links to our data, models, and code.