dataesr/scientific-paragraphs-categorization huggingface dataset:
Polymer showed the lowest CV(560) in comparison to the other ACs. This additional sensitivity study revealed no strong trends between physical and biogeochemical parameters and RE (at any wavelengths) such as observed for SeaWIFS by  [20]. This suggests that ACs are less sensitive to the biogeochemical parameters. However, it suggests that Polymer can be affected by high values of turbidity where Chl-a and CDOM do not dominate the marine signal. CDOM, Chl-a, and their resulting turbidity (turbidity indirectly resulting from CDOM) could be limiting for NASA due to the use of Chl-a estimates in the Case-2 iteration scheme. The same could not be observed for the BlAC. The lack of atmosphere and more seawater parameter data limited our discussion. The AC negative values were taken into account in the match-ups exercise and were included in the scatterplots and the statistics calculations except for BlAC between 412 and 665 nm, as applied by the its recommended flags.
Acknowledgments: The authors would like to thank the  Université du Littoral-Côte d'Opale  and the  Hauts-de-France region  for the PhD scholarship fellowship and the  CNRS  for the short-term contract funding. The  European Organization for the Exploitation of Meteorological Satellites (EUMETSAT)  is acknowledged for support of the Sentinel-3A data delivery through the  Sentinel-3 Validation Team for Ocean Color  ( S3VT-OC ). The authors thank the crew of Wayki village for their help and patience during the field campaigns in French Guiana. The Sepia II and  Côte de la Manche  ship crews are acknowledged for their help and kindness during sea cruises in the Eastern English Channel. The authors are grateful to  David Dessailly  for the help in coding the SeaDAS processing loop. Furthermore, the authors would like to thank  François Steinmetz  for his help in handling the Polymer software and  Sean Bailey  for his help with the SeaDAS software.
Funding: The field campaigns were funded by the Centre National d'Etudes Spatiales (CNES) through the TOSCA program, by the Centre National de la Recherche Scientifique (Guyane), and the European Space Agency for the "MERIS validation and algorithm 4th reprocessing" (Contract No. ARG/003-025/1406/LOG). The PhD scholarship fellowship of M. A. M. was funded by the Université du Littoral Côte d'Opale and Région Hauts-de-France. An additional short-term contract of M. A. M. was funded by CNES. The ASD FieldSpec4 was purchased thanks to Université du Littroral-Côte d'Opale through "Bonus Qualité Recherche" and to CNES.
For instance, dopamine (CHEBI:18243), a chemical compound with several important roles in the brain and body, can be characterized as being a catecholamine (CHEBI:33567), an aralkylamino compound (CHEBI:64365) and an organic aromatic compound (CHEBI:33659) (Fig. 1 ). When predicting if a certain drug interacts with dopamine, its ancestors will provide additional information that is not usually directly expressed in the text. While the reader can consult additional materials to better understand a biomedical document, current relation extraction models are trained solely on features extracted from the training corpus. Thus, ontologies confer an advantage to relation extraction models due to the semantics encoded in them regarding a particular domain. Since ontologies are described in a common machinereadable format, methods based on ontologies can be applied to different domains and incorporated with other sources of knowledge, bridging the semantic gap between relation extraction models, data sources, and results  [12].
LSTM units account for this vanishing gradient problem through a gated architecture, which makes it easier for the model to capture long-term dependencies. Recently, LSTMs have been applied to relation extraction tasks in various domains. Miwa and Bansal  [14]  presented a model that extracted entities and relations based on bidirectional tree-structured and sequential LSTM-RNNs. The authors evaluated this model on three datasets, including the SemEval 2010 Task 8 dataset, which defines 10 general semantic relations types between nominals  [15]. Bidirectional LSTMs have been proposed for relation extraction, obtaining better results than one-directional LSTMs on the SemEval 2010 dataset  [16]. In this case, at each time step, there are two LSTM layers, one that reads the sentence from left to right, and another that reads from right to left.
Textpresso is a project that aims at helping database curation by automatically extracting biomedical relations from research articles  [32]. Their approach incorporates an internal ontology to identify which terms may participate in relations according to their semantics. Other approaches measure the similarity between the entities and use the value as a feature for a machine learning classifier  [33]. One of the teams that participated in the BioCreative VI ChemProt task used ChEBI and Protein Ontology to extract additional features for a neural network model that extracted relation between chemicals and proteins  [34]. To the best of our knowledge, our work is the first attempt at incorporating ancestry information from biomedical ontologies with deep learning to extract relations from text.
Moreover, we improved the F1-score of an existing DDI extraction model by 0. 022 by adding our proposed ontology information, and demonstrated its applicability to other domains by generating a corpus of gene-phenotype relations and training our model on that corpus. The code and results obtained with the model can be found on our GitHub repository (https://github. com/lasigeBioTM/ BOLSTM), while a Docker image is also available (https:// hub. docker. com/r/andrelamurias/bolstm), simplifying the process of training new classifiers and applying them to new data. We also made available the corpus produced for gene-phenotype relations, where each entity is mapped to an ontology concept. These results support our hypothesis that domain-specific information is useful to complement data-intensive approaches such as deep learning.
Therefore, after tokenizing each sentence, we obtain the Shortest Dependency Path (SDP) between the entities of the pair. For example, in the sentence "Laboratory Tests Response to Plenaxis e1 should be monitored by measuring serum total testosterone e1 concentrations just prior to administration on Day 29 and every 8 weeks thereafter", the shortest path between the entities would be Plenaxis -Response -monitoredby -measuring -concentrations -testosterone. For both tokenization and dependency parsing, we use the spaCy software library (https://spacy. io/). The text of each entity that appears in the SDP, including the candidate entities, is replaced by the generic string to reduce the effect of specific entity names on the model. For each element of the SDP, we obtain the WordNet hypernym class using the tool developed by Ciaramita and Altun  [35].
Then, we assumed that every gene-phenotype pair that co-occurred in the same sentence was a positive instance if this relation existed in the file. While the phenotype entities were manually mapped to the Human Phenotype Ontology, we had to employ an automatic method to obtain the most representative Gene Ontology  [41, 42]  concept of each gene, giving preference to concepts inferred from experiments. We applied the same pre-processing steps as for the DDI corpus, except for entity matching and negative instance filtering. This corpus is available at https://github. com/ lasigeBioTM/BOLSTM/tree/master/HP%20corpus.
Results We evaluated the performance of our BO-LSTM model on the SemEval 2013: Task 9 DDI extraction corpus  [49]. This gold standard corpus consists of 792 texts from DrugBank  [50] , describing chemical compounds, and 233 abstracts from the Medline database  [51]. DrugBank is a cheminformatics database containing detailed drug and drug target information, while Medline is a database of bibliographic information of scientific articles in Life and Health Sciences. Each document was annotated with pharmacological substances and sentence-level DDIs. We refer to each combination of entities mentioned in the same sentence as a candidate pair, which could either be positive if the text describes a DDI, or negative otherwise. In other words, a negative candidate is a candidate pair that is not described as interacting in the text.
We implemented the BO-LSTM model in Keras, a Python-based deep learning library, using the TensorFlow backend. The overall architecture of the BO-LSTM model is presented in Fig. 2. More details about each layer can be found in the "Methods" section. We focused on the effect of using different sources of information to train the model. As such, we tuned the hyperparameters to obtain reasonable results, using as reference the values provided by other authors that have applied LSTMs to this gold standard  [18, 19]. We first trained the model using only the word embeddings of the SDP of each candidate pair (Fig. 2a ). Then we tested the effect of adding the WordNet classes as a separate embedding and LSTM layer (Fig. 2b ) Finally, we tested two variations of the ChEBI channel: first using the concatenation of the sequence of ancestors of each entity (Fig.
The first row of each table shows the results obtained using an LSTM network trained solely on the word embeddings of the SDP of each candidate pair. Then, we studied the impact of adding each information channel on the performance of the model, and the effect of using all information channels, as shown in Fig. 2. For the detection task, using the concatenation of ancestors results in an improvement of the F1-score in the Medline dataset, contributing to an overall improvement of the F1-score in the full test set. The most notable improvement was in the recall of the Medline dataset, where the concatenation of ancestors increased this score by 0. 246. The usage of ontology ancestors did not improve the F1-score of detection of DDIs in the DrugBank dataset. In every test set, it is possible to observe that the concatenation of ancestors results in a higher recall while considering only the common ancestors is more beneficial to precision.
Combining both approaches with the WordNet channel results in a higher F1-score. Regarding the classification task (Table  2 ), the F1-score was improved on each dataset by the usage of the ontology channel. Considering only the common ancestors led to an improvement of the F1-score in the DrugBank dataset and on the full corpus, while the concatenation improved the Medline F1-score, similarly to the detection results. To better understand the contribution of each channel, we studied the relations detected by each configuration by one or more channels, and which of those were also present in the gold standard. Figures  4  and 5  show the The difference between the values of this figure and the sum of their respective values in Fig. 5  is due to the system being executed once for each dataset.
While we studied the potential of domain-specific ontologies based only on the ancestors of each entity, there are other ways to integrate semantic information from ontologies into neural networks. For example, one could consider only the ancestors with the highest information content, since those would be the most helpful to characterize an entity. The information content can be estimated either by the probability of a given term in the ontology or in an external dataset. Alternatively, a semantic similarity measure that accounts for non-transitive relations could be used to find similar concepts to the entities of the relation  [59] , or one that considers only the most relevant ancestors  [60]. The quality of the ontology embeddings could also be improved by pre-training on a larger dataset, which would include a wider variety of concepts.
Conclusions This work demonstrates how domain-specific ontologies can improve deep learning models for classification of biomedical relations. We developed a model, BO-LSTM which combines biomedical ontologies with LSTM units to detect and classify relations in text. In this manuscript, we demonstrate that ontologies can improve the performance of deep learning techniques for biomedical relation extraction, in particular for situations with a limited number of annotations available, which was the case of the Medline dataset. Furthermore, we explored how it can be adapted to other relation extraction domains, for example, gene-phenotype relations. Considering that biomedical ontologies are openly available and regularly updated as the knowledge on the domain progresses, they should be considered important information sources for relation extraction.
We present a tutorial that introduces high school students to the Gene Ontology classification system which is widely used in genomics and systems biology studies to characterize large sets of genes based on functional and structural information. This classification system is a valuable and standardized method used to identify genes that act in similar processes and pathways and also provides insight into the overall architecture and distribution of genes and gene families associated with a particular tissue or disease. By means of this tutorial, students learn how the classification system works through analyzing a gene set using DAVID the Database for Annotation, Visualization and Integrated Discovery that incorporates the Gene Ontology system into its suite of analysis tools. This method of profiling genes is used by our high school student interns to categorize gene expression data related to behavioral neuroscience.
Introduction Genomics is the branch of biology concerned with the study of genes and their functions (see the National Institutes of Health Frequently Asked Questions about Genetic and Genomic Science). Genomics arose from the acceleration of genetic research which was fueled by the development of rapid and affordable DNA sequencing technologies  (Shendure et al. , 2017). This opened the door to the sequencing of entire genomes. Presently, the DNA codes for thousands of genomes from diverse species have been sequenced and studied (see the National Center for Biotechnology Information Genome database). The goals in genomics research are to address all genes and their inter-relationships in order to understand the combined influence on the function of an organism. With this newfound knowledge of the staggering number of genes that make up an organism, the Gene Ontology (GO) classification system was created to organize genes by their similarities and differences (see Gene Ontology Consortium 'About' page).
"Ontology" is not a commonly encountered term and there are several definitions that are related to philosophical concepts. In the context of information science, as described here, "ontology" is concerned with the representation, formal naming and classification system with the purpose of describing the relationship categories and properties of the data. This classification system provides the scientific community with a structured vocabulary for defining genes  (Ashburner et al. , 2000; du Plessis et al. , 2011; Hastings, 2017; Thomas, 2017). GO terms are commonly used in most, if not all, databases and analysis tools relevant to bioinformatics, systems biology  (Wanjek, 2011) , and genomics studies  (du Plessis et al. , 2011). GO terms are species specific and are continuously revised and expanded as biological knowledge is obtained  (Gaudet et al.
, 2017). The importance of the GO term system becomes apparent when analyzing the organization of genomes and coding regions, the distribution of genes involved in specific processes and the conservation of genes across species  (Gaudet et al. , 2017). This classification system is also quite powerful when analyzing data from large scale gene expression studies  (du Plessis et al. , 2011)  that consider co-expression data from specific tissues obtained under defined circumstances such as treatment with pharmaceutical agents, or with neurodevelopmental disorders, cancer, or diabetes as examples. GO terms are instrumental for understanding the functions of these genes. Introducing GO terms and the gene classification system to high school students will bring them up to speed on a commonly used research tool in current genomics methods and expose them to the vast amounts of data that have been derived from genomics and systems biology studies.
In the subsequent sections we show an example of how to extract information about a gene from its associated GO terms and then provide instruction for a practical exercise which will enable students to profile a list of genes using GO terms in the bioinformatics resource DAVID, The Database for Annotation, Visualization and Integrated Discovery. This is a protocol that we teach to our high school student interns when they are evaluating gene expression data for their summer projects  (Crusio et al. , 2017 , see BioScience Project student posters). The student research internship projects are in the context of behavioral neuroscience. Students typically work with gene expression data associated with a specific brain region or brain disorder. As an example for projects related to learning and memory, gene expression data for the hippocampus would be used.
Gene Profiling in DAVID DAVID is primarily a clustering program that groups genes based on different criteria related to GO terms. DAVID links to other databases that contain complementary information like The Gene Ontology. In this exercise, students will use the sample gene lists (DEMOLIST1 or DEMOLIST2) that are accessible from the DAVID database to see how the Gene Ontology classification partitions a set of genes based on GO Terms. Screenshots and videos are provided for step by step instruction. We also provide a video to instruct students on profiling a gene list in DAVID obtained from a random gene list generator. Protocol Screenshot 1 (Figure  2 ). DAVID landing page. The start analysis link is accessed here and is circled in red in this image. (  Video 1, Delprato et al. , 2019c)  Screenshot 2 (Figure  3 ).
Learning assessment We polled 12 student interns from our summer program for feedback on their experience working with the GO system and DAVID. The survey consists of 5 direct yes or no response questions and two open ended questions. The responses are show in Table  1, Table 2,  and Table 3. For the direct response questions, 100% of the students had not worked or heard of the DAVID database (Ques1, Table  1. ) 83% of the students had not heard of the GO system. Two students had heard of the GO system in an advanced placement biology class but had not explored it further (Ques2, Table1). 100% of the students responded that they benefited intellectually from working with the GO system and DAVID tools (Ques3, Table1) and also that they had enjoyed the experience (Ques 4, Table  1 ). 96% of the students thought that the experience would better prepare them for future database use.
One student indicated that this was not applicable (Quest 5, Table  1 ). For the open ended survey questions 3a and 4a (Table  2  and Table 3 ), students were asked to explain how they benefited from working with the databases and tools and what about the Through this internship using DAVID, I was exposed for the first time to the field of bioinformatics and the gene ontology system. I learned more about the role genes play in neurological disorders in humans. I also got a glimpse of what biomedical researchers and neuroscientists actually do and the type of resources that they work with. I learned about current methods used by researchers and was able to apply those techniques to my own project. The exposure and experience I gained with bioinformatics gave me a better understanding of gene interactions and how researchers analyze them.
For larger gene sets, a computer program may be used. I learned different ways genes are classified and analyzed (kegg pathway, etc. ). I can use this database for future research projects to get a sense of different pathways these genes are involved in. I learnt a lot about genomics and how interconnected the different genes in our body are. It also helped me work on my project on Alzheimer's and the APOE 4 allele. I was able to learn more about PTSD and identifying candidate genes for PTSD. I performed this project with DAVID and learned a lot about the Gene Ontology System. The DAVID tools helped me cluster annotation terms based on keywords associated with neurological disorders. Also, the KEGG Pathway map allowed me to visualize genes and how they interact. This was the first time that I worked with either of these tools and I think I benefited mostly because I got an idea of what working in the field of Biology/Neuroscience would be like.
I was able to learn more about the GO System, different databases such as the Allen Brain Atlas and StringDB, which I used in conjunction with DAVID and learned about how DAVID analyzes the data. I found that working with the DAVID tools and Gene Ontology System I was exposed to a real world experience in science that gave me a better understanding of where we are in research now, and what is still to be done. The benefits I found in myself were a capital in scientific nomenclature, new skills in analysis of data, and a wholesome exposure into the field of genomics research. I learned a lot about gene databases and how to look at specific sets of data while ignoring information that may be necessary. I felt like working with the DAVID tools exposed me to an area of research that I wasn't previously familiar with in an engaging and fascinating way.
Learning how to find specific information about gene interactions/gene functions through an online database is a valuable tool that I believe will benefit me in my future research. Working with DAVID and the other databases gave me a chance to learn about the field of bioinformatics and genetics, beyond our school curriculum. It was extremely interesting to learn about the hundreds of genes in the human genome, their various functions both at the molecular and biological level and how they affect the neurological characteristics of human beings. The databases were also very interactive and allowed me to explore the other parts of the database myself. Although it was initially difficult to navigate the DAVID tools, I found the experience rewarding in the end. The process became easier as I persisted in using the database, and I enjoyed being able to explore the realm of gene profiling.
I enjoyed working with the DAVID tools and learning about gene profiling because from a biological point of view, DAVID is very good at finding relevant information, like other correlates, to the keyword I'm looking for. I felt like what I was learning about through DAVID and the Gene Ontology system was applicable to my project and I was able to utilize and apply my knowledge of these tools effectively, making me more excited to use it. I gained insight into gene analysis and gained knowledge that I can use in other situations. I got practical hands-on experience working with a scientific database which was quite different from the textbook learning taught in schools. I found this quite refreshing and enjoyable. Although it was pretty confusing for me at the time, I think what made it enjoyable was that I found using these tools interesting especially since this was all new to me! Also it was not too difficult or overwhelming since the provided instructions for the internship walked me through each step.