# LifeCLEF 2023 teaser: Species Identification and Prediction Challenges

Alexis Joly, Stefan Kahl, Lukáš Picek, Christophe Botella, Diego Marcos, Milan Šulc, Marek Hrúz, Maximilien Servajean, Elijah Cole, Andrew Durso, et al.

► To cite this version:

Alexis Joly, Stefan Kahl, Lukáš Picek, Christophe Botella, Diego Marcos, et al. LifeCLEF 2023 teaser: Species Identification and Prediction Challenges. ECIR 2023 - 45. European Conference on Information Retrieval, Jaap Kamps; Lorraine Goeuriot; Fabio Crestani; Maria Maistro; Hideo Joho; Brian Davis; Cathal Gurrin; Udo Kruschwitz; Annalina Caputo, Apr 2023, Dublin, Ireland. pp.568-576, 10.1007/978-3-031-28241-6_65. hal-04204378

HAL Id: hal-04204378

https://inria.hal.science/hal-04204378v1

Submitted on 12 Sep 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
LifeCLEF 2023 teaser: Species Identification and
Prediction Challenges

Alexis Joly¹ ID, Hervé Goëau² ID, Stefan Kahl⁶ ID, Lukáš Picek⁹ ID, Christophe

Botella¹⁴ ID, Diego Marcos¹⁶ ID, Milan Šulc¹² ID, Marek Hrúz⁹ ID, Titouan

Lorieul¹ ID, Sara Si Moussi² ID, Maximilien Servajean⁷ ID, Benjamin

Kellenberger¹⁵ ID, Elijah Cole⁸ ID, Andrew Durso¹⁰ ID, Hervé Glotin³ ID, Robert

Planqué⁴ ID, Willem-Pier Vellinga⁴ ID, Holger Klinck⁶, Tom Denton¹¹, Ivan

Eggel⁵, Pierre Bonnet² ID, Henning Müller⁵ ID

¹ Inria, LIRMM, Univ Montpellier, CNRS, Montpellier, France

² CIRAD, UMR AMAP, Montpellier, Occitanie, France

³ Univ. Toulon, Aix Marseille Univ., CNRS, LIS, DYNI team, Marseille, France

⁴ Xeno-canto Foundation, The Netherlands

⁵ HES-SO, Sierre, Switzerland

⁶ KLYCCB, Cornell Lab of Ornithology, Cornell University, USA

⁷ LIRMM, AMI, Univ Paul Valéry Montpellier, Univ Montpellier, CNRS, France

⁸ Department of Computing and Mathematical Sciences, Caltech, USA

⁹ Department of Cybernetics, FAV, University of West Bohemia, Czechia

¹⁰ Department of Biological Sciences, Florida Gulf Coast University, USA

¹¹ Google LLC, San Francisco, USA

¹² Rossum.ai, Prague, Czechia

¹³ Listening Observatory for Hawaiian Ecosystems, Univ. of Hawai'i at Hilo, USA

¹⁴ Centre for Invasion Biology, Stellenbosch University, South Africa

¹⁵ Department of Ecology and Evolutionary Biology, Yale University, USA

¹⁶ Inria, TETIS, Univ Montpellier, Montpellier, France

**Abstract.** Building accurate knowledge of the identity, the geographic distribution and the evolution of species is essential for the sustainable development of humanity, as well as for biodiversity conservation. However, the difficulty of identifying plants, animals and fungi is hindering the aggregation of new data and knowledge. Identifying and naming living organisms is almost impossible for the general public and is often difficult, even for professionals and naturalists. Bridging this gap is a key step towards enabling effective biodiversity monitoring systems. The LifeCLEF campaign, presented in this paper, has been promoting and evaluating advances in this domain since 2011. The 2023 edition proposes five data-oriented challenges related to the identification and prediction of biodiversity: (i) PlantCLEF: very large-scale plant identification from images, (ii) BirdCLEF: bird species recognition in audio soundscapes, (iii) GeoLifeCLEF: remote sensing based prediction of species, (iv) SnakeCLEF: snake recognition in medically important scenarios, and (v) FungiCLEF: fungi recognition beyond 0-1 cost.

**Keywords:** biodiversity · machine learning · AI · species identification · species prediction · plant identification · bird identification · species distribution model · snake identification · fungi identification.
# 1 Introduction

Accurately identifying organisms observed in the wild is an essential step in ecological studies. Unfortunately, observing and identifying living organisms requires high levels of expertise. For instance, plants alone account for more than 400,000 different species and the distinctions between them can be quite subtle. Since the Rio Conference of 1992, this *taxonomic gap* has been recognized as one of the major obstacles to the global implementation of the Convention on Biological Diversity [1]. In 2004, Gaston and O'Neill [6] discussed the potential of automated approaches for species identification. They suggested that, if the scientific community were able to (i) produce large training datasets, (ii) precisely evaluate error rates, (iii) scale-up automated approaches, and (iv) detect novel species, then it would be possible to develop a generic automated species identification system that would open up new vistas for research in biology and related fields.

Since the publication of [6], automated species identification has been studied in many contexts [5,8,13,23,25,27,28,33]. This area continues to expand rapidly, particularly due to advances in deep learning [4,7,24,26,29,30,31,32]. In order to measure progress in a sustainable and repeatable way, the LifeCLEF [3] research platform was created in 2014 as a continuation and extension of the plant identification task [12] that had been run within the ImageCLEF lab [2] since 2011 [9,10,11]. Since 2014, LifeCLEF expanded the challenge by considering animals and fungi in addition to plants, and including audio and video content in addition to images [14,15,16,17,18,19,20,21,22]. LifeCLEF 2023 consists of five challenges (PlantCLEF, BirdCLEF, GeoLifeCLEF, SnakeCLEF, FungiCLEF), which we will now describe in turn. Table 1 provides an overview of the data and tasks of the five challenges.

<table><caption>**Table 1.** Overview of the data and tasks of the five LifeCLEF challenges</caption><thead><tr><th></th><th>Modality</th><th>#species</th><th>#items</th><th>Task</th><th>Metric</th></tr></thead><tbody><tr><td>PlantCLEF</td><td>images</td><td>80,000</td><td>4.0M</td><td>Classification</td><td>Macro-Average MRR</td></tr><tr><td>BirdCLEF</td><td>audio</td><td>100–500</td><td>10K–50K</td><td>Multi-Label Classification</td><td>F1 score</td></tr><tr><td>GeoLifeCLEF</td><td>images<br>time-series<br>tabular</td><td>11,340</td><td>5.3M</td><td>Multi-Label Classification</td><td>Jaccard index</td></tr><tr><td>SnakeCLEF</td><td>images<br>metadata</td><td>1,500</td><td>150–200K</td><td>Classification</td><td>ad-hoc metric</td></tr><tr><td>FungiCLEF</td><td>images<br>metadata</td><td>1,600</td><td>300K</td><td>Classification</td><td>ad-hoc metric</td></tr></tbody></table>
## 2 PlantCLEF 2023 Challenge: Identify the World's Flora

**Motivation:** It is estimated that there are more than 300,000 species of vascular plants in the world. Automatic identification has made considerable progress in recent years as highlighted during previous editions of PlantCLEF. Deep learning techniques now seem mature enough to address the ultimate but realistic problem of global identification of plant biodiversity despite many problems that the data may present (a huge number of classes, very strongly unbalanced classes, partially erroneous identifications, duplicate pictures, variable visual quality, diversity of visual contents, e.g., photos, herbarium sheets, etc).

**Data collection:** the training dataset that will be used this year can be distinguished into two main categories: trusted and web (i.e., with or without a strong revision of species names by human experts). The trusted training subset will be based on a dataset of more than 2.8 million images, covering more than 80,000 plant species, shared and aggregated by the Global Biodiversity Information Facility (GBIF) platform. The web training dataset will be based on a web crawl with Google & Bing search engines. All datasets provided in previous editions of PlantCLEF can also be used; the use of external data will be possible. Finally, the test set will contain more than 60k pictures verified by world class experts related to various regions of the world and taxonomic groups.

**Task description:** the task will be evaluated as a plant species retrieval task based on multi-image plant observations from the test set. The goal will be to retrieve the correct plant species among the top results of a ranked list of species returned by the evaluated system. Participants will initially have access to the training set, followed up a few months later by the whole test set. Self-supervised, semi-supervised or unsupervised approaches will be strongly encouraged and a starter package with pre-trained models will be provided.

## 3 BirdCLEF 2023 Challenge: Bird species identification in soundscape recordings

**Motivation:** Recognizing bird sounds in complex soundscapes is an important sampling tool that often helps reduce the limitations of point counts. In the future, archives of recorded soundscapes will become increasingly valuable as the habitats in which they were recorded will be lost. In the past few years, deep learning approaches have transformed the field of automated soundscape analysis. Yet, when training data is sparse, detection systems struggle with the recognition of rare species. The goal of this competition is to establish training and test datasets that can serve as real-world applicable evaluation scenarios for endangered habitats and help the scientific community to advance their conservation efforts through automated bird sound recognition.

**Data collection:** We will build on the experience from previous editions and adjust the overall task to encourage participants to focus on few-shot learning and task-specific model designs. We will select training and test data to suit this demand. As in previous iterations, Xeno-canto will be the primary source for
training data, expertly annotated soundscape recordings will be used for test-
ing. We will focus on bird species for which there is limited training data, but
we will also include common species so that participants can train good recog-
nition systems. In search of suitable test data, we will consider different data
sources with varying complexity (call density, chorus, signal-to-noise ratio, an-
thropophony...), and quality (mono and stereo recordings). We also want to focus
on very specific real-world use cases (e.g., conservation efforts in India) and frame
the competition based on the demand of the particular use case. Additionally, we
are considering including unlabeled data to encourage self-supervised learning
regimes.

**Task description:** The challenge will be held on Kaggle and the evaluation mode will resemble the 2022 test mode (i.e., hidden test data, code competition). We will use established metrics like F1 score and LwLRAP which reflect use cases for which precision is key and also allow organizers to assess system performance independent of fine-tuned confidence thresholds. Participants will be asked to return a list of species for short audio segments extracted from labeled soundscape data. In the past, we used 5-second segments, and we will consider increasing the duration of these context windows to better reflect the overall ground truth label distribution. However, the overall structure of the task will remain unchanged, as it provides a well-established base that has resulted in significant participation in past editions (e.g., 1,019 participants and 23,352 submissions in 2022). Again, we will strive to keep the dataset size reasonably small (<50 GB) and easy to process, and we will also provide introductory code repositories and write-ups to lower the entry level of the competition.

4 GeoLifeCLEF 2023 Challenge: Species presence
prediction based on occurrences data and
high-resolution remote sensing images

**Motivation:** Predicting which species are present in a given area through species distribution modeling is a central problem in ecology and a crucial issue for biodiversity conservation. Such predictions are a fundamental element of many decision-making processes, whether for land use planning, the definition of protected areas or the implementation of more ecological agricultural practices. The models classically used in ecology are well-established but have the drawback of covering only a limited number of species at spatial resolutions often coarse in the order of kilometers or hundreds of meters at best. The objective of GeoLifeCLEF is to evaluate models with orders of magnitude hitherto unseen, whether in terms of the number of species covered (thousands), spatial resolution (on the order of 10 meters), or the number of occurrences used as training data (several million). These models have the potential to greatly improve biodiversity management processes, especially at the local level (e.g. municipalities), where the need for spatial and taxonomic precision is greatest.

**Data collection:** A brand new dataset will be built for the 2023 edition of GeoLifeCLEF in the framework of a large-scale European project on biodiver-
sity monitoring (MAMBO, Horizon EU program). It will contain about 5 million
species occurrences extracted from various selected datasets of the Global Bio-
diversity Information Facility (GBIF) and covering the whole EU territory (38
countries including E.U. members). For the explanatory variables, we will pro-
vide both high resolution remote sensing data (i.e., Sentinel-2 RGB, Near-IR,
Red-Edge and SWIR, along with altitude) and coarser resolution environmen-
tal raster data (e.g., Chelsa climate, SoilGrids, land use, etc.). An important
change this year will be the evaluation and test set composition. We will evalu-
ate model ability to predict the whole set of species present in local sites using
presence-absence data.

**Task description:** Given a test set of locations (i.e., geo-coordinates) and corresponding high-resolution remote sensing images and environmental covariates, the goal of the task will be to return for each location the set of species that are were inventoried at that location. The test set will include only locations for which an exhaustive plant species inventory is available (i.e., in the form or presence/absence data).

5 SnakeCLEF 2023 Challenge: Snake Identification in
Medically Important scenarios

**Motivation:** Developing a robust system for identifying species of snakes from photographs is an important goal in biodiversity but also for human health. With over half a million victims of death & disability from venomous snakebite annually, understanding the global distribution of the > 4,000 species of snakes and differentiating species from images (particularly images of low quality) will significantly improve epidemiology data and treatment outcomes. We have learned from previous editions that “machines” can accurately recognize ($F_1^C \approx 90\%$ and Top1 Accuracy $\approx 90\%$) even in scenarios with long-tailed distributions and $\approx 1,600$ species. Thus, testing over real Medically Important Scenarios and specific countries (primarily tropical and subtropical) and integrating the medical importance of species is the next step that should provide a more reliable machine prediction.

**Data collection:** The dataset of the previous year will be extended up to ≈ 1,800% snake species from around the world (minimum 10 images per species). The images will be divided into observations that depict the same snake individual. Additionally, medical importance (i.e. how venomous the species is) and country-species relevance will be provided for each species. The evaluation will be done on various subsets of a newly created “secret” test set with around 50k images.

**Task description:** Given the set of authentic snake species observations and corresponding locations, the goal of the task is to create a classification model that, for each observation, returns a ranked list of predicted species. The classification model will have to fit limits for memory footprint (ONNX model with max size of 1GB) and prediction time limit (will be announced later) measured on the submission server. The model should have to consider and minimize the
danger to human life and the waste of antivenom if a bite from the snake in the image were treated as coming from the top-ranked prediction.

# 6 FungiCLEF 2023 Challenge: Fungi Recognition Beyond 0-1 Cost

**Motivation:** Automatic recognition of species at scale, such as in popular citizen-science projects, requires efficient prediction on limited resources. In practice, species identification typically depends not solely on the visual observation of the specimen but also on other information available to the observer, e.g., habitat, substrate, location and time. Thanks to rich metadata, precise annotations, and baselines available to all competitors, the challenge aims at providing a major benchmark for combining visual observations with other observed information. Additionally, since mushrooms are often picked for consumption, misclassification of edible and poisonous mushrooms is an important aspect for the evaluation of the practical prediction loss.

**Data collection:** The dataset comes from a citizen science project, the Atlas of Danish Fungi, where all samples went through an expert validation process, guaranteeing a high quality of labels. Rich metadata (Habitat, Substrate, Times-tamp, GPS, EXIF etc.) are provided for most samples. The training set will be the union of the training and test set (without out-of-scope samples) from the 2022 challenge (i.e. 295,938 training images belonging to 1,604 species observed mostly in Denmark). The test set will consist of new fungi observations from the citizen science project.

**Task description:** Given the set of real fungi species observations and corresponding metadata, the goal of the task is to create a classification model that, for each observation, returns a ranked list of predicted species. The classification model will have to fit limits for memory footprint (ONNX model with max size of 1GB) and prediction time limit (will be announced later) measured on the submission server. The model should have to consider and minimize the danger to human life, i.e., the confusion between poisonous and edible species. Baseline procedures of how metadata can help the classification and pre-trained baseline classifiers will be provided as part of the task description to all participants.

# 7 Timeline and registration instructions

All information about the timeline and participation in the challenges is provided on the LifeCLEF 2023 web page [3].

# 8 Discussion and Conclusion

To fully reach its objective, an evaluation campaign such as LifeCLEF requires a long-term research effort so as to (i) encourage non-incremental contributions, (ii) measure consistent performance gaps, (iii) progressively scale-up the problem
and (iv) enable the emergence of a strong community. The 2023 edition of the lab supports this vision and also includes the following innovations:

- The GeoLifeCLEF challenge will be entirely revisited towards running at the scale of the whole Europe thanks to a newly created dataset with millions of occurrences paired with high-resolution remote sensing data.

- The BirdCLEF challenge will include new data with a focus on Central African species.

- The world's coverage of PlantCLEF test set will be improved with a focus on tropical regions and biodiversity hotspots.

- The inclusion of time- and memory-limits within several challenges to encourage the use of frugal methods rather than ensembles of tens of models.

**Acknowledgements:** This work has received funding from the European Union's Horizon research and innovation program under grant agreement No 101060639 (MAMBO project).

## References

1. Convention on Biodiversity. https://www.cbd.int/

2. ImageCLEF. http://www.imageclef.org/

3. LifeCLEF. http://www.lifeclef.org/

4. Bonnet, P., Goëau, H., Hang, S.T., Lasseck, M., Šulc, M., Malécot, V., Jauzein, P., Melet, J.C., You, C., Joly, A.: Plant identification: experts vs. machines in the era of deep learning. In: Multimedia Tools and Applications for Environmental & Biodiversity Informatics, pp. 131–149. Springer (2018)

5. Cai, J., Ee, D., Pham, B., Roe, P., Zhang, J.: Sensor network for the monitoring of ecosystem: Bird species recognition. In: Intelligent Sensors, Sensor Networks and Information, 2007. ISSNIP 2007. 3rd International Conference on (2007). https://doi.org/10.1109/ISSNIP.2007.4496859

6. Gaston, K.J., O'Neill, M.A.: Automated species identification: why not? Philosophical Transactions of the Royal Society of London B: Biological Sciences 359 (1444), 655–667 (2004)

7. Ghazi, M.M., Yanikoglu, B., Aptoula, E.: Plant identification using deep neural networks via optimization of transfer learning parameters. Neurocomputing 235, 228–235 (2017)

8. Glotin, H., Clark, C., LeCun, Y., Dugan, P., Halkias, X., Sueur, J.: Proc. 1st workshop on Machine Learning for Bioacoustics - ICML4B. ICML, Atlanta USA (2013), http://sabiod.org/ICML4B2013_book.pdf

9. Goëau, H., Bonnet, P., Joly, A., Bakic, V., Barthélémy, D., Boujemaa, N., Molino, J.F.: The imageclef 2013 plant identification task. In: CLEF task overview 2013, CLEF: Conference and Labs of the Evaluation Forum, Sep. 2013, Valencia, Spain. Valencia (2013)

10. Goëau, H., Bonnet, P., Joly, A., Boujemaa, N., Barthélémy, D., Molino, J.F., Birnbaum, P., Mouysset, E., Picard, M.: The imageclef 2011 plant images classification task. In: CLEF task overview 2011, CLEF: Conference and Labs of the Evaluation Forum, Sep. 2011, Amsterdam, Netherlands. (2011)
11. Goëau, H., Bonnet, P., Joly, A., Yahiaoui, I., Barthélémy, D., Boujemaa, N., Molino, J.F.: Imageclef2012 plant images identification task. In: CLEF task overview 2012, CLEF: Conference and Labs of the Evaluation Forum, Sep. 2012, Rome, Italy. Rome (2012)

12. Goëau, H., Joly, A., Bonnet, P., Bakic, V., Barthélémy, D., Boujemaa, N., Molino, J.F.: The imageclef plant identification task 2013. In: Proceedings of the 2nd ACM international workshop on Multimedia analysis for ecological data. pp. 23–28. ACM (2013)

13. Joly, A., Goëau, H., Bonnet, P., Bakić, V., Barbe, J., Selmi, S., Yahiaoui, I., Carré, J., Mouysset, E., Molino, J.F., et al.: Interactive plant identification based on social image data. *Ecological Informatics* **23**, 22–34 (2014)

14. Joly, A., Goëau, H., Botella, C., Glotin, H., Bonnet, P., Vellinga, W.P., Müller, H.: Overview of LifeCLEF 2018: a large-scale evaluation of species identification and recommendation algorithms in the era of ai. In: Jones, G.J., Lawless, S., Gonzalo, J., Kelly, L., Goeuriot, L., Mandl, T., Cappellato, L., Ferro, N. (eds.) CLEF: Cross-Language Evaluation Forum for European Languages. Experimental IR Meets Multilinguality, Multimodality, and Interaction, vol. LNCS. Springer, Avigon, France (Sep 2018)

15. Joly, A., Goëau, H., Botella, C., Kahl, S., Servajean, M., Glotin, H., Bonnet, P., Planqué, R., Stöter, F.R., Vellinga, W.P., Müller, H.: Overview of LifeCLEF 2019: Identification of Amazonian Plants, South & North American Birds, and Niche Prediction. In: Crestani, F., Brascher, M., Savoy, J., Rauber, A., Müller, H., Losada, D.E., Bürki, G.H., Bürki, G.H., Cappellato, L., Ferro, N. (eds.) CLEF 2019 - Conference and Labs of the Evaluation Forum. Experimental IR Meets Multilinguality, Multimodality, and Interaction, vol. LNCS, pp. 387–401. Lugano, Switzerland (Sep 2019). https://doi.org/10.1007/978-3-030-28577-7_29, https://hal.umontpellier.fr/hal-02281455

16. Joly, A., Goëau, H., Glotin, H., Spampinato, C., Bonnet, P., Vellinga, W.P., Champ, J., Planqué, R., Palazzo, S., Müller, H.: LifeCLEF 2016: Multimedia Life Species Identification Challenges. In: Fuhr, N., Quaresma, P., Gonçalves, T., Larsen, B., Balog, K., Macdonald, C., Cappellato, L., Ferro, N. (eds.) CLEF: Cross-Language Evaluation Forum. Experimental IR Meets Multilinguality, Multimodality, and Interaction, vol. LNCS, pp. 286–310. Springer, Évora, Portugal (Sep 2016). https://doi.org/10.1007/978-3-319-44564-9_26, https://hal.archives-ouvertes.fr/hal-01373781

17. Joly, A., Goëau, H., Glotin, H., Spampinato, C., Bonnet, P., Vellinga, W.P., Lombardo, J.C., Planque, R., Palazzo, S., Müller, H.: LifeCLEF 2017 Lab Overview: Multimedia Species Identification Challenges. In: Jones, G.J., Lawless, S., Gonzalo, J., Kelly, L., Goeuriot, L., Mandl, T., Cappellato, L., Ferro, N. (eds.) CLEF: Cross-Language Evaluation Forum. Experimental IR Meets Multilinguality, Multimodality, and Interaction, vol. LNCS, pp. 255–274. Springer, Dublin, Ireland (Sep 2017). https://doi.org/10.1007/978-3-319-65813-1_24, https://hal.archives-ouvertes.fr/hal-01629191

18. Joly, A., Goëau, H., Glotin, H., Spampinato, C., Bonnet, P., Vellinga, W.P., Planque, R., Rauber, A., Fisher, B., Müller, H.: LifeCLEF 2014: Multimedia Life Species Identification Challenges. In: CLEF: Cross-Language Evaluation Forum. Information Access Evaluation. Multilinguality, Multimodality, and Interaction, vol. LNCS, pp. 229–249. Springer International Publishing, Sheffield, United Kingdom (Sep 2014). https://doi.org/10.1007/978-3-319-11382-1_20, https://hal.inria.fr/hal-01075770
19. Joly, A., Goëau, H., Glotin, H., Spampinato, C., Bonnet, P., Vellinga, W.P., Planqué, R., Rauber, A., Palazzo, S., Fisher, B., et al.: Lifeclef 2015: multimedia life species identification challenges. In: Experimental IR Meets Multilinguality, Multimodality, and Interaction, pp. 462–483. Springer (2015)

20. Joly, A., Goëau, H., Kahl, S., Deneu, B., Servajean, M., Cole, E., Picek, L., De Castaneda, R.R., Bolon, I., Durso, A., et al.: Overview of lifeclef 2020: a system-oriented evaluation of automated species identification and species distribution prediction. In: International Conference of the Cross-Language Evaluation Forum for European Languages. pp. 342–363. Springer (2020)

21. Joly, A., Goëau, H., Kahl, S., Picek, L., Lorieul, T., Cole, E., Deneu, B., Servajean, M., Durso, A., Glotin, H., et al.: Overview of lifeclef 2022: an evaluation of machine-learning based species identification and species distribution prediction. In: International Conference of the Cross-Language Evaluation Forum for European Languages. pp. 257–285. Springer (2022)

22. Joly, A., Goëau, H., Kahl, S., Picek, L., Lorieul, T., Cole, E., Deneu, B., Servajean, M., Durso, A., Bolon, I., et al.: Overview of lifeclef 2021: an evaluation of machine-learning based species identification and species distribution prediction. In: International Conference of the Cross-Language Evaluation Forum for European Languages. pp. 371–393. Springer (2021)

23. Lee, D.J., Schoenberger, R.B., Shiozawa, D., Xu, X., Zhan, P.: Contour matching for a fish recognition and migration-monitoring system. In: Optics East. pp. 37–48. International Society for Optics and Photonics (2004)

24. Lee, S.H., Chan, C.S., Remagnino, P.: Multi-organ plant classification based on convolutional and recurrent neural networks. IEEE Transactions on Image Processing 27(9), 4287–4301 (2018)

25. NIPS Int. Conf.: Proc. Neural Information Processing Scaled for Bioacoustics, from Neurons to Big Data (2013), http://sabiod.org/nips4b

26. Norouzzadeh, M.S., Morris, D., Beery, S., Joshi, N., Jojic, N., Clune, J.: A deep active learning system for species identification and counting in camera trap images. Methods in Ecology and Evolution 12(1), 150–161 (2021)

27. Towsey, M., Planitz, B., Nantes, A., Wimmer, J., Roe, P.: A toolbox for animal call recognition. Bioacoustics 21(2), 107–125 (2012)

28. Trifa, V.M., Kirschel, A.N., Taylor, C.E., Vallejo, E.E.: Automated species recognition of antbirds in a mexican rainforest using hidden markov models. The Journal of the Acoustical Society of America 123, 2424 (2008)

29. Van Horn, G., Mac Aodha, O., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., Belongie, S.: The inaturalist species classification and detection dataset. CVPR (2018)

30. Villon, S., Mouillot, D., Chaumont, M., Subsol, G., Claverie, T., Villéger, S.: A new method to control error rates in automated species identification with deep learning algorithms. Scientific reports 10(1), 1–13 (2020)

31. Wäldchen, J., Mäder, P.: Machine learning for image based species identification. Methods in Ecology and Evolution 9(11), 2216–2225 (2018)

32. Wäldchen, J., Rzanny, M., Seeland, M., Mäder, P.: Automated plant species identification—trends and future directions. PLoS computational biology 14(4), e1005993 (2018)

33. Yu, X., Wang, J., Kays, R., Jansen, P.A., Wang, T., Huang, T.: Automated identification of animal species in camera trap images. EURASIP Journal on Image and Video Processing (2013)