DOI: 10.5281/zenodo.2548629
title: Large-scale and fine-grained phenological stage annotation of herbarium specimens datasets

description: This upload is constituted of four datasets of specimens from American herbaria covering different levels of information precision and different floras - from temperate to equatorial.

Three of these datasets consist of selected specimens from herbaria located in different geographic and environmental regions. Each specimen of these three datasets was annotated with the following fields: family, genus, species name, fertile / non-fertile, presence / absence of flower(s), presence / absence of fruit(s). The resulting dataset was composed of 163,233 herbarium specimens belonging to 7,782 species, 1,906 genera, and 236 families. Specimens were annotated as “fertile” if any reproductive structures were present, such as sporangia (ferns), cones (gymnosperms), flowers, or fruits (angiosperms). Non-fertile specimens were those that lacked any reproductive structures.

The fourth dataset consists of 20,371 herbarium specimens from 11 genera in the sunflower family (<em>Asteraceae</em>). The main difference in this dataset is that it is annotated with fine-grained phenophase scores rather than presence/absence attributes (see description below).

Each of these datasets is described below:


	
	NEVP: this dataset of New England vascular plant (NEVP) specimens was produced by members of the Consortium of Northeastern Herbaria. The dataset comprises 42,658 digitized specimens that belong to 1,375 species and come from several North American institutions. Most of the specimens in this dataset are from the north-temperate region of the northeastern United States.
	
	
	FSU: this dataset was produced by the Florida State University's Robert K. Godfrey Herbarium (FSU), a collection that focuses on northern Florida and the U.S. Southeast Coastal Plain, one of North America's biodiversity hotspots. This dataset contains 54,263 digitized herbarium specimen records that belong to 3,870 species, making it the taxonomically richest dataset in this study. Most species in this dataset grow under subtropical or warm temperate conditions in the southeastern region of the United States.
	
	
	CAY: this dataset comes from the IRD’s Herbarium of French Guiana (CAY). CAY is dedicated to the Guayana Shield flora, with a strong focus on tropical tree species. This dataset is composed of 66,312 herbarium specimens that belong to 3,024 species. All digitized specimens of this herbarium are accessible online. Most specimens were collected in the tropical rainforests of French Guiana, with the remaining specimens coming mostly from Suriname and Guyana.
	
	
	PHENO: this dataset includes 20,371 herbarium specimens of 139 species in the <em>Asteraceae</em> produced in a study of phenological trends in the U.S. Southeast Coastal Plain. The dataset is composed of specimen records from 57 herbaria. Each recorded specimen was annotated for quartile percentages (0, 25, 50, 75, or 100%) of (i) closed buds, (ii) buds transformed into flowers, and (iii) fruits. According to the distribution of these three categories for each specimen, a phenophase code was computed.
	


 

<strong>Datasets format</strong>

These datasets are grouped in 3 tasks:


	fertility detection
	flowers and/or fruit detection
	phenophase classification


The first 2 tasks are carried on the first 3 previous datasets and thus are based on the same set of images, unlike the third task which has its own disjoint set of images. This is why the dataset is presented into two separated files, one for each set of images.

<em>Fertility detection & flower/fruit detection</em>

These tasks are contained into the <em>herbarium_fertility_annotations.zip</em> archive. It consists of 3 files:


	<em>metadata.csv</em>: general information about all the herbarium specimens for these tasks

	
		<em>id</em>: specimen identifier
		<em>collection</em>: which of NEVP, FSU or CAY does the specimen come from
		<em>herbarium</em>: institution of origin of the specimen, especially for NEVP collection
		<em>clade</em>,<em> family</em>,<em> genus</em>,<em> species</em>: classification of the specimen
		<em>URL</em>: URL of the scan
	
	
	<em>fertility_task.csv</em>: specific information regarding the fertility detection task
	
		<em>id</em>: specimen identifier
		<em>is_fertile</em>: <em>True</em> if the specimen has an expression of fertility, <em>False</em> otherwise
		<em>train_test_set</em>: which subset does the specimen belong to; possible values are: <em>train</em>, <em>random_test</em>, <em>species_test</em> and <em>herbarium_test</em>
	
	
	<em>flower_fruit_task.csv</em>: specific information regarding the flower/fruit detection task
	
		<em>id</em>: specimen identifier, note that in this case not all the specimen described in <em>metadata.csv</em> are included in this task
		<em>has_flower</em>: <em>True</em> if the specimen has at least one flower, <em>False</em> otherwise
		<em>has_fruit</em>: <em>True</em> if the specimen has at least one fruit, <em>False</em> otherwise
		<em>train_test_set</em>: which subset does the specimen belong to; possible values are: <em>train</em>, <em>random_test</em>, <em>species_test</em> and <em>herbarium_test</em>
	
	


<em>Phenophase classification</em>

These tasks are contained into the <em>herbarium_asteraceae_phenophase_annotations.zip</em> archive. It consists of a single file:


	<em>annotations.csv</em>:

	
		<em>id</em>: specimen identifier
		<em>URL</em>: URL of the scan
		<em>genus</em>: genus of the specimen
		<em>phenophase</em>: integer from 1 to 9 describing the phenophase of the specimen
		<em>train_test_set</em>: which subset does the specimen belong to; possible values are: <em>train</em> and <em>test</em>
	
	


 

<strong>Additional ressources</strong>

More information can be found in the related paper:<br>
<em>Lorieul, T., K. D. Pearson, E. R. Ellwood, H. Goëau, J.-F. Molino, P. W.  Sweeney, J. M. Yost, J. Sachs, E. Mata-Montero, G. Nelson, P. S. Soltis, P. Bonnet, and A. Joly. 2019. Toward a large-scale and deep phenological stage annotation of  herbarium specimens: Case studies from temperate, tropical, and equatorial floras. Applications in Plant Sciences 7(3): e1233.</em>

For an example of usage of these datasets as well as a baseline, see: http://doi.org/10.5281/zenodo.2549996