dataesr/scientific-paragraphs-categorization huggingface dataset:
Discussion If we are to achieve data-driven decision making based on accurate abundance trajectories for the conservation of threatened species, then we need the capacity to rapidly iterate population estimates to ensure regular evaluation and adjustment of our action plans. This requires reconciling disparate datasets, which for many species must occur across wide geographic regions and with the integration of opportunistic data from various stakeholders, in order to assess local, regional, and international population trends in support of national policy makers and multinational conservation agreements to protect those species. In order to achieve these goals, Flukebook intercedes with three different stakeholder groups. The platform most directly benefits scientists through tools for photo-identification and data management.
Automation of once-manual tasks enables greater efficiency in processing and curating photo-ID data. These lead to both cost savings in undertaking research and time savings in reporting on field studies. In addition, the collaborative platform allows questions to be asked on larger datasets than would otherwise be possible without significant logistical challenges, a noteworthy example of which can be seen in the collaborative global research done on the Wildbook for whale sharks  (Norman et al. 2017 ) and the early-stage collaborative work already underway on Flukebook by consortia such as the Arabian Sea Whale Network and  (Blount et al. 2020)  the Caribbean Marine Mammals Preservation Network (CARI'MAM  ; Vail and Borobia 2020). Larger labs or organizations can enable multiple graduate students, interns, or researchers to work on the same datasets from remote locations.
Data fields such as categorical behavior observations (e. g. , feeding, breaching, mating), metrics of photo quality or heterogeneity of identifiability (e. g. scales from 0 to 5 vs 0 to 10 or those which include markedness when other users do not), and individual naming conventions (e. g. individual animals in Flukebook may have different nicknames given by different research groups) can differ significantly between research teams. Flukebook offers tools for users to efficiently standardize these fields across one study or even a consortium of researchers, such as the Organizations described in the Data Ownership and Collaboration Model section above, but a user is not forced to accept conventions which they might not share. This has the benefit of allowing diverse data to be imported onto Flukebook, with the potential drawback that multiple data standards will co-exist on the platform as separate fields in the database.
In some senses, this is a limitation which is not new to this platform. It has always been true that if multiple catalogs of cetacean observations are to be combined into a single study, there may be some data reconciliation and standardization required to make that analysis possible given that field-wide standards do not always exist. Flukebook provides data sharing tools, a flexible database structure to import varied metadata fields, standardized export formats, and photo ID algorithms that empower and encourage researchers to collaborate in the first place. In some ways, Flukebook reflects the diversity of approaches which exist in cetacean photo-identification, and the analyses which rely upon it, while facilitating efforts without enforcing data standards which are not universal accepted within the field.
A final point regarding accuracy is that since every match suggested by Flukebook's algorithms must be confirmed by a user who is looking at both the query and candidate photos, an algorithm will not introduce errors into the data regardless of its purported accuracy rating. When a user confirms negative matches (potentially new individuals from photos queried which found no positive match in the database) using manual matching methods, these tools will save time with no accuracy detriment compared to previous fully manual methods. Flukebook enables the user to invest the majority of their time matching on these rare challenging matches by rapidly identifying the majority of well known, unchanged individuals which are identified regularly, thereby creating a more efficient use of expert time.
Both efficiency and accuracy improvements have been robustly documented using various automatic cetacean matching systems over the years (e. g. Mizroch et al. 1990; Whitehead 1990; Beekmans et al. 2005; Cheeseman et al. 2022). As with our discussion of the difficulties in extrapolating algorithm accuracy above, it is important that researchers measure these factors with the datasets, algorithms, and species relevant to their own work. In the context of Flukebook, this will require further study to quantify expected improvements in efficiency and accuracy for each specific pipeline on the platform, some of which is already underway. Automated identification is not a black box that produces scientific results on its own, but a tool that must be understood and used carefully by the scientist.
The first Wildbook intelligent agent deployed was a YouTube crawler integrated with Sharkbook. ai (then under the name "Whaleshark. org") in 2018. This agent searches YouTube for videos of the focal species, then processes key frames from the video to find clear photos of the animals and identify the individuals depicted. The agent also extracts available information from the video file and metadata; as well as reading the text available on YouTube to get the date, time, and location of the sighting. If the location, date and time cannot be automatically extracted, the bot actually asks the submitter for this information via the YouTube comment section, in multiple languages based on the language of the original post. This feature has yet to be implemented in Flukebook but is coming in a future release.
An intelligent agent active on Flukebook is 'Tweet-a-whale' which is currently in beta release (the phase of software development when a feature is fully implemented but undergoing usability testing). Twitter users can submit a photo for identification directly to Flukebook by tweeting a photo and tagging the account @tweetawhale. The agent running the account will extract available metadata from the text of the tweet and submit the image to Flukebook. The image goes through detection and identification, and resulting candidate matches are tweeted back to the original submitter. Users are then required to log in to Flukebook to approve these potential matches, meaning a member of the public who is not a Flukebook user cannot confirm match results on their own. Data submitted this way are then available to all researchers on Flukebook and labeled as data submitted by the public, unless the twitter user is already affiliated with a Flukebook user account, in which case the data are controlled by that user.
Conclusions By leveraging cutting-edge computer vision with a global data management platform, Flukebook allows for easier storage of data and more rapid photo-identification in cetacean studies while bringing together researchers and community scientists to create bigger datasets. Flukebook's position within the open-source family of Wildbook platforms provides major benefits for conservation technology development, where returns on technology investment are multiplied across every Wildbook platform. This computer vision pipeline works with real-world conditions, allowing for broad contributions from both research professionals and amateur naturalists. This process gives greater confidence that we know who the animals are and where they have been. Getting this baseline information correct allows government and managers to focus on policies that can bring about actionable change.
Acknowledgements The authors wish to thank the many volunteers, researchers, NGOs and government partners, whale watch companies, and community naturalists who make up the users of Flukebook. Dr. Ann Pabst of UNCW and Dr. Doug Nowacek of Duke University who led the BOEM cooperative agreement which funded the creation of this manuscript. Virgil Zetterlind and the team at Conserve. IO who developed the Ocean Alert app. Shauna Bingham and the Channel Islands Naturalist Corps at the Channel Islands National Marine Sanctuary for piloting the new Ocean Alert app-based pipeline. We also thank major collaborators and contributors of datasets which were foundational to algorithm development and website functionality: Arabian Sea Whale Network, CARI'MAM, Cascadia Research Collective, The Dominica Sperm Whale Project, Elding Whale Watch, IndoCet, Globice, MarEcoTel, New England Aquarium, NOAA, Norwegian Orca Survey, Oceania Project, Sarasota Dolphin Research Program, Wild Dolphin Project, and Whale Watching Akureyri.
, LTD, Wuhan, China). Prior to assembly and mapping, the raw sequencing data were first filtered by Trimmomatic (version. 36) (Bolger et al. , 2014) , and then the low-quality reads and the reads containing adapter were trimmed using default parameters (PE -phred33 ILLUMINACLIP:adaptor_file:2:30:5 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15 HEADCROP:0 MINLEN:36). Clean reads were further treated with inhouse scripts to eliminate duplication bias introduced in library preparation and sequencing. Briefly, clean reads with the same UMI sequence were first clustered together, and then they were compared to each other using pairwise alignment. Reads with sequence identity over 95% were extracted into a new subcluster. After all subclusters had been generated, multiple sequence alignment was performed to obtain one consensus sequence for each subcluster.
Analysis of D. officinale Transcriptomic Data Genomics mapping and alignment program analysis were performed based on the reference genome of Dendrobium catenatum (https://www. ncbi. nlm. nih. gov/assembly/GCF_ 001605985. 2) using STAR software (version 2. 5. 3a) with default parameters  (Dobin et al. , 2013; Dobin and Gingeras, 2015). Functional annotation of predicted genes was first performed using diamond Blastx in UniProt (Universal Protein) to obtain the protein ID, and then the annotation was searched in the database of Nr (NCBI non-redundant protein sequences), Pfam (Protein family), Rfam (RNA family), eggNog (evolutionary genealogy of genes: non-supervised orthologous groups), Gene Ontology (GO), and Kyoto Encyclopedia of Genes and Genomes (KEGG) using the same ID. Gene expression was qualified using the RSEM software package (RNA-Seq by Expectation-Maximization).
Genes that were differentially expressed between groups were identified using the edgeR package (version 3. 12. 1) (Li and Dewey, 2011). The threshold for significantly differentially expressed genes  (Robinson et al. , 2010 ) (DEGs) was set at a fold change of ≥2 and an FDR corrected p-value < 0. 05  (Robinson and Oshlack, 2010).
Transcriptomic Data Fungal de novo transcriptome assemblies were first reconstructed using Trinity based on the free-living mycelium of Tulasnella sp. (S6) and Serendipita sp. (  12825 ), and the longest transcript was chosen as the unigene. Next, the unmapped reads of symbiotic samples (after mapping to the plant genome) were subjected to mapping analysis using the unigenes by Kangce Technology Co. , Ltd. , Wuhan, China, followed by  Grabherr et al. (2011). Because we focused our analysis on the gene scale, further analysis was performed on Trinity unigenes, i. e. , the longest isoform per Trinity gene. TransDecoder (version 5. 5; http://transdecoder. sourceforge. net/) was used to identify putative coding regions from the assembled unigenes, and only the best-predicted protein was retained for each transcript (single_best_only option).
, 2014). To analyze fungal gene expression, RNA-seq data were generated from fungi in symbiotic status, namely, colonized during three symbiotic germination stage samples (seed germination, stage 2; protocorm formation, stage 3, and seedling development stage 4) and compared to free-living mycelium (FLM). Assembly of the fungal transcriptome was based on our de novo sequencing of Tulasnella sp. (S6) and Serendipita sp. (12825) due to the low species similarity with the published genome, but gene annotation was also performed with the reference genome of Tulasnella calospora AL13/4D (version 1; https://www. ncbi. nlm. nih. gov/assembly/GCA_000827465. 1/) and Serendipita vermifera MAFF 305830 (version 1; https://www. ncbi. nlm. nih. gov/assembly/GCA_000827415. 1/). In the global expression analysis of Tulasnella sp.
Automating the extraction of meaningful temporal information from sequences of microscopy images represents a major challenge to characterize dynamical biological processes. So far, strong limitations in the ability to quantitatively analyze single-cell trajectories have prevented large-scale investigations to assess the dynamics of entry into replicative senescence in yeast. Here, we have developed DetecDiv, a microfluidic-based image acquisition platform combined with deep learning-based software for high-throughput single-cell division tracking. We show that DetecDiv can automatically reconstruct cellular replicative lifespans with high accuracy and performs similarly with various imaging platforms and geometries of microfluidic traps. In addition, this methodology provides comprehensive temporal cellular metrics using time-series classification and image semantic segmentation.
Editor's evaluation In this work, the authors describe a novel method, based on deep learning, to analyze large numbers of yeast cells dividing in a controlled environment. The method builds on existing yeast cell trapping microfluidic devices that have been used for replicative lifespan assay. The authors demonstrate how an optimized microfluidic device can be coupled with deep learning methods to perform automatic cell division tracking and single cell trajectories quantification. The overall performance of the method is impressive: it allows to deal with large image datasets generated by timelapse microscopy several order of magnitudes faster than what manual annotation would require.
e. budded, unbudded, etc. ) trapped in a microfluidic device  (Ghafari et al. , 2021). However, due to the limited accuracy of the model, it has not demonstrated its ability to perform an automated division counting, let alone determine the RLS of individual cells. This is likely due to the fact that the misclassification of a single frame during the lifespan can dramatically compromise the accuracy of the RLS measurement. Here, we report the development of DetecDiv, an integrated platform that combines highthroughput observation of cell divisions using a microfluidic device, a simple benchtop image acquisition system, and a deep learning-based image processing software with several image classification frameworks. Using this methodology, one can accurately track successive cell divisions in an automated manner and reconstruct RLS without human intervention, saving between 2 and 3 orders of magnitude on the analysis time.
We selected this classification scheme -namely, the prediction of the budding state of the cell -over the direct assessment of cell division or budding (e. g. 'division' versus 'no division') because division and budding events can only be assessed by comparing successive frames, which is impossible using a classical CNN architecture dedicated to image classification, which takes a single frame as input. To train and evaluate the performance of the classifier, we generated a manually annotated dataset (referred to as 'groundtruth' in the following) by arbitrarily selecting 250 traps (split into a training and a test set, see Methods) containing situations representative of all cellular states from different fields of view and independent experimental replicates. Benchmarking the classifier consisted of three steps: first, we computed the confusion matrices (Figure  2 -figure supplement 2A) as well as the classical metrics of precision (i.
e. the fraction of correct predictions among all predictions for each class), recall (i. e. the fraction of detected observations among all observations for each class), and F 1 -score (i. e. the harmonic mean of precision and recall). The F 1 -score was found to be higher than 85% for all classes (Figure  2 -figure supplement 2C). Next, the predictions of budding events were compared to the manually annotated data. Despite a good visual match between the groundtruth and the CNN predictions, the distribution of cell-cycle durations revealed that the model tends to predict 'ghost' divisions of abnormally short duration (Figure  2B ). In addition, sporadic misclassification could falsely assign a cell to the 'dead' state, thus decreasing the number of total generations predicted based on the test dataset (N=1127 for the groundtruth versus N=804 for the CNN model, see Figure  2C ).
For scale reference, each image is 19. 5µm wide. (B) Comparison of the different methods used for six sample cells. The gray bars represent the groundtruth data made from manually annotated image sequences. Colored lines indicate the corresponding predictions made by CNN+LSTM (orange), the CNN+postprocessing (magenta), and the CNN (blue) networks (see Methods and supplementary text for details). The red segments indicate the position of new generation events. (C) Left: histogram of cell-cycle durations representing groundtruth data and predictions using different processing pipelines. The p-value indicates the results of a rank-sum test comparing the predictions to the groundtruth for the different pipeline variants. The total number of generations annotated in the groundtruth or detected by the networks is indicated in the legend.
Right: Scatter plot in log scale representing the correlation between groundtruth-calculated cell-cycle durations and those predicted by the CNN+LSTM network. R 2 represents the coefficient of correlation between the two datasets. Precision and recall are defined in the Methods section. (D) Left: cumulative distribution showing the survival of cells as a function of the number of generations (N=50 cells). The numbers in the legend indicate the median replicative lifespans. The p-value indicates the results from a statistical log-rank test. Right: Scatter plot representing the correlation of the replicative lifespans of 50 individual cells obtained from the groundtruth with that predicted by the CNN+LSTM architecture. Inset: same as the main plot, but for the CNN and CNN+Post-Processing pipelines.
R 2 indicates the coefficient of correlation between the two datasets. (E) Replicative lifespans obtained using the CNN+LSTM network for longevity mutants (solid colored lines, genotype indicated). The shading represents the 95% confidence interval calculated using the Greenwood method  (Pokhrel et al. , 2008). The median RLS and the number of cells analyzed are indicated in the legend. The dashed lines with shading represent the hazard rate (i. e. the instantaneous rate of cell mortality in the population of cells at a given replicative age) and its standard deviation estimated with a bootstrap test (N=100). Results from log-rank tests (comparing WT and mutant distributions) are indicated on the left of the legend. (F) Same as E but for WT cells grown in 2% glucose or 2% galactose (colored lines).
, 2000)  and we could compute the related death rate with a high-confidence interval thanks to this large-scale dataset (Figure  2E ). Model predictions were further evaluated by comparing the predicted replicative lifespans to manually generated test sets for each of the mutants (Figure  2 -figure supplement 5A). In addition, using glucose and galactose as carbon sources, we performed comparative measurements of cell-cycle durations (N=38,205 events and N=15,495 events for glucose and galactose, see Figure  2F  inset) and RLS (median = 26 generations, N=1174 events and median=24, N=565 events, for glucose and galactose, respectively, Figure  2F ). Our results were in line with previous measurements  (Liu et al. , 2015; Frenk et al. , 2017). To further check the performance of the model, we used an additional manually generated test set obtained with cells growing in galactose to compare to the corresponding predictions (Figure  2 -figure  supplement 5B-D ).
Application of the division counting and lifespan prediction model to different imaging platforms and microfluidic devices To further test the robustness of our analysis pipeline, we proceeded to the analysis of several datasets obtained under various imaging conditions. First, we performed experiments with the same microfluidic system but using a commercial microscope with ×60 magnification. After training the classifier on 80 ROIs and testing on 40 independent ROIs, we observed similar results to those obtained with the RAMM system and a ×20 objective (compare the 'specialist' columns for the panels in Figure  3A  and B): the classification benchmarks were greater than 90%, the error rate on the number of generations detected was a few percents, and the cell-cycle length distributions were similar between prediction and groundtruth.
This first demonstrated that neither the RAMM imaging system nor the ×20 magnification is required to guarantee successful division counting and lifespan reconstruction with our analysis pipeline. In addition, we gathered time-lapse microscopy datasets from several laboratories using microfluidic cell trapping systems with different geometries and various imaging conditions (Figure  3C  and E;  Crane et al. , 2014; Liu et al. , 2015; Granados et al. , 2018). We also included data generated in our lab based on a device similar to that used in  Jo et al. , 2015 (Figure 3D). For each trap geometry, we manually evaluated the retention rate of a mother cell during a lifespan. Indeed, high retention is key to getting a reliable measurement of the RLS (i. e. to ensure that mother cells are not eventually replaced by their daughters).
This analysis revealed that a 'semi-open' geometry (as in the design shown in Figure  3D , Figure  1 -figure supplement 1G, and Figure  3 -figure supplement 1) did not prevent large mother cells from being sporadically expelled during the budding of their daughters, unlike other cell trap shapes. Of note, the geometry proposed by  Crane et al. , 2014; Figure   3E  was not tested on an entire lifespan, but only on about even generations, hence leading to an overestimation of the retention rate (it was reported to be below 50% in the original paper). For each dataset, we trained a specific classifier (or 'specialist') on 80 ROIs and validated it on 40 independent ROIs. The different benchmarks (i. e. classification performance and division predictions) showed that each specialist performed very well on each specific test dataset, thus confirming further that our analysis pipeline is robust and applicable to different cell trapping and imaging configurations.
Last, instead of training the classifiers separately on each dataset, we asked whether a unique classifier would have sufficient capacity to handle the pooled datasets with all imaging conditions and trap geometries used in Figure  3. Strikingly, this "generalist" model showed comparable performance to the different specialists. This approach thus further highlighted the versatility of our methodology and demonstrated the interest in aggregating data sets to ultimately build a standardized reference model for counting divisions, independently of the specific imaging conditions.