dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgements We would like to acknowledge  Almut Schüz  for her council on German animal welfare regulations. We would like to thank  Tadashi Isa  and  Katsuki Nakamura  for consultation on animal welfare regulations in Japan. We additionally thank  Vladimir Popov  from the  Faculty of Medicine at Lomonosov Moscow State University and the Russian Laboratory Animal Science Association  for consultation on animal welfare regulations in Russia. ASM received funding from the  Wellcome Trust  ( 110157/Z/15/Z ). AS is supported by the  Canadian Institute of Health Research  ( PJT-162141 ) and the  Natural Sciences and Engineering Research Council of Canada  ( RGPIN-2020-06930 ). AWR is supported by  China Brain Project, China National Natural Science Foundation of China , and  China National Key Research and Development Program.
To support the use of alternative methods in regulatory assessment of chemical risks, the concept of adverse outcome pathway (AOP) constitutes an important toxicological tool. AOP represents a structured representation of existing knowledge, linking molecular initiating event (MIE) initiated by a prototypical stressor that leads to a cascade of biological key event (KE) to an adverse outcome (AO). Biological information to develop such AOP is very dispersed in various data sources. To increase the chance of capturing relevant existing data to develop a new AOP, the AOP-helpFinder tool was recently implemented to assist researchers to design new AOP. Here, an updated version of AOP-helpFinder proposes novel functionalities. The main one being the implementation of an automatic screening of the abstracts from the PubMed database to identify and extract event-event associations.
, 2017; Knapen et al. , 2015; Pittman et al. , 2018). Nevertheless, this led to a new challenge regarding the integration of existing knowledge. Technical advances allow the generation of large volume of mechanistic data, that are dispersed in different sources. Given the enormous amount of existing knowledge in the scientific literature, the identification of relevant biological information to build AOP is a complex and time-consuming task. Previous studies have applied automated approaches, using artificial intelligence (e. g. text mining, frequent item-set mining) to extract fragmented information from texts and data. For example, Scholten et al. have developed network with mechanistic evidence extracted from the literature to support cancer risk assessment  (Scholten et al. , 2022). Various tools exist to automatically screen large datasets, such as SWIFT-Review for systematic review  (Howard et al.
, 2016) , ComptoxAI (https://comptox. ai. ) for graphical representation of integrated open sources, Abstract Sifter  (Baker et al. , 2017) , Reach for machine reading of biomedical papers  (Valenzuela-Escárcega et al. , 2018) , or Compass for association rule mining from clinical data, biobank and electronic patient records  (Jiang et al. , 2020; Krysiak-Baltyn et al. , 2014). To assist in the AOP development process, a tool, called AOP-helpFinder was recently developed  (Carvaillo et al. , 2019; Jornod et al. , 2021). The AOP-helpFinder method (server 1. 0 available at https://aop-helpfinder. u-paris-sciences. fr/index. php) allows to link prototypical stressors to biological events of AOPs (MIE, KE, AO). This tool automatically screens the abstracts from the PubMed database by combining artificial intelligence with graph theory algorithm.
AOP-helpFinder, being a third-party tool of the AOP-Wiki database (the central web-based application compiling OECD-validated and submitted AOPs), can also be acceded via https://aopwiki. org/info_pages/8. As example, AOP-helpFinder 1. 0 was already successfully applied for the identification of links between different stressors and events  (Gundacker et al. , 2022; Jornod et al. , 2020; Kaiser et al. , 2022; Rugard et al. , 2020). Moreover, it was used to create two AOPs, both submitted to the AOP-Wiki database, one linked to AhR activation to breast cancer (AOP 439)  (Benoit et al. , 2022)  and another one related to increased DNA damages during embryonic development lead to microcephaly (AOP 441)  (Jaylet et al. , 2022). Here, we proposed a new version of the tool, AOP-helpFinder 2. 0, that bring a wide range of new features extremely useful for AOP development.
AOP-helpFinder AOP-helpFinder is a tool developed in 2019, that allows to automatically screen available knowledge in a text format to identify and extract data (i. e. association between a stressor and an event) in order to help to build new AOP  (Carvaillo et al. , 2019). All developed codes, under the python programming language, are freely available. It has the advantage of using text mining algorithms and graph theory. Recently, an updated version of the tool was published and was made available via the webserver AOP-helpFinder 1. 0 to facilitate the researcher's and regulator's uses (https://aop-helpfinder. u-paris-sciences. fr/index. php) (Jornod et al. , 2021). This version 1. 0 provides an easy way to identify and compile information automatically extracted from the literature, considering always the latest version of the PubMed database.
Changes in AOP-helpFinder 2. 0 The proposed version of the tool has several improvements, the main one being the possibility to search for identification of event-event linkage from published scientific abstracts, information that are complementary to the ones existing in the various toxicological databases. Others features are also proposed to the user, such as the scoring of the identified linkage, therefore supporting the WoE of the associations, the visualization of the results and the workflow framework of the tool. Therefore, the interface of the tool has been updated with the possibilities to search, either for stressor-event or for event-event (Fig. 1 ). These news features were evaluated and are presented with case studies in the results part, with all AOPs included in the OECD workplan, open for comment & citation (WPHA/WNT Endorsed).
The event-event module The event-event module, that allows to identify MIE-KE, KE-KE and KE-AO linkage, only requires one list of specific biological events of interest, defined by the user (e. g. obesity, oxidative stress…). AOP-helpFinder 2. 0 will screen abstracts from the PubMed database to identify co-occurrence of events mentioned in abstracts. The pre-processing of the PubMed abstracts to be screened is based on the same methodology developed previously for the stressor-event module  (Carvaillo et al. , 2019; Jornod et al. , 2021). Evaluation of the search method for the event-event module. To determine if it was preferable to search 1) on a subset of abstracts, the ones related to the defined list events of interest or 2) directly on the entire PubMed database (≈ 35 million articles), a comparison of these two searches methods was performed and tested to ensure that no important information was missing.
Development of the confidence score to support WoE The current version of AOP-helpFinder provides all the information and links from the identified abstracts co-mentioning a stressor and an event for the query lists in a table format. It is also possible to obtain in the results table the summary of the publications, if available. In order to provide additional useful information to guide the user, a Confidence score (Cs) was developed and added in both modules (stressor-event and even-event), that is helpful for the identification of the most relevant linkages. The Cs metric is based on a Fisher's exact test, a statistical test used to determine whether the proportions of categories between two variables differ significantly from each other. The test was implemented under Python using the 'fisher_exact' function of the SciPy package (v 1.
8). This metric was used to determine whether an event is found to be more frequently associated with a stressor (stressor-event) or with another event (event-event) in comparison to the remaining articles on PubMed, referring to the full PubMed database excluding the articles that have already been analyzed by AOP-helpFinder for a given stressor or event (example on Fig. 2 ). The Cs, which provides a measure of confidence on the associations identified by AOP-helpFinder, was evaluated with three case studies, and a categorical ranking was proposed for easy interpretation.
Visualization of the results In the proposed new version of AOP-helpFinder, a visualization module has been added using the matplotlib python package (v. 3. 4. 3). It produces user-friendly figures from the obtained results, that can be very largely dependent of the query. The plain text data tables (results output from AOP-helpFinder 1. 0) have also been modified by colorful and interactive tables. Raw data will always be available to users who wish to access them, but we recommend using the colored tables for a quicker, more pleasant and easier reading. Indeed, in these new styles of tables, data are interactive (e. g. by clicking on a PMID users are instantaneously forwarded to the PubMed page of the abstract hosting the link and they can quickly check a putative link by reading the associated PubMed abstract at a glance or the full text if needed).
Improvement of the AOP-helpFinder workflow framework In the first version of the web server, a bash pipeline running AOP-helpFinder was used. In order to easily parallelize, and to have a more reproducible and robust pipeline, we adapted this bask pipeline to a Nextflow one (v. 22. 04. 5)  (Di Tommaso et al. , 2017) , available on https ://github. com/systox1124/AOP-helpFinder. This change is associated to an increase of the number of central processing units (CPUs) on our information technology (IT) infrastructure.
Evaluation of the search procedures for the event-event module To determine if the AOP-helpFinder module will be based on cooccurrence searches from a pre-selected set of abstracts (related to the input list) or on the full PubMed database, both procedures were tested, and various criteria evaluated on the obtained results. The AOP "Cyp2E1 Activation Leading to Liver Cancer" (AOP 220) was randomly chosen for the evaluation. This AOP, validated by OECD, describes the toxicity pathway induced by prolonged exposure to Cyp2E1 substrate leading to liver cancer. As of October 03, 2022 in the AOP-Wiki database it is composed of five events present in different levels of the biological organizations (macro-molecular, cell/tissue and organ/organ system): "Activation of Cyp2E1", "Oxidative stress", "Hepatotoxicity", "Increased proliferation", and "Liver cancer", which served as input list for AOP-helpFinder.
First, AOP-helpFinder was run for co-occurrences between each pair of events (among the five presents in the AOP 220) by screening the full PubMed database (≈ 35 millions of abstracts). Then, AOP-helpFinder screened independently pre-selected abstracts from the PubMed database. These pre-selected abstracts, that mention at least one the event present in the list, were compiled using the following search: unique (Cyp2E1 activation AND Oxidative stress AND Hepatotoxicity AND Increased proliferation AND Liver Cancer). A total of 733,413 abstracts as of October, 03, 2022 were retained. Identified event-event linkage from the two runs were independently analyses. Both methods provided exactly the same number of results, and were able to retrieve articles mentioning all the defined pairs of KEs present in the AOP 220 (Table  1 ).
The difference between the two methods was only in the calculation time, with a calculation time about 20 times faster for the method screening the pre-selected abstracts (26 min against 11 h for the method analyzing the full PubMed). This difference is easily explained by the number of abstracts to be analyzed (35 millions for PubMed against 733,413 for the pool of pre-selected abstracts). The method analyzing the whole of PubMed is therefore of no interest here, either for the user or for a science that wants to be respectful for the environment. The method based on pre-selected abstracts was retained and included in the event-event module of AOP-helpFinder 2. 0. Since we chose to use the event-event module on a pool of preselected articles of interest extracted from the PubMed database, we investigated whether the use of AOP-helpFinder had an advantage over the use of a simple PubMed query online.
An alternative solution would be to perform a more restrictive search on the PubMed web site (Table  1 ). For example, the query "Cyp2E1 activation" requires the presence of both words. However, this method poses several problems. First, it imposes a precise order on the words (e. g. "Cyp2E1 activation" will give different results than "Activation of Cyp2E1 ′′ ), greatly reducing the number of results obtained. Also, when searching between two events (e. g. "Cyp2E1 activation" AND "Oxidative stress"), PubMed does not take into account the distance between the events. Therefore, the first event and the second one can be found in different parts of the abstract, without any link between both. The use of module event-event from AOP-helpFinder 2. 0 on pre-selected abstracts from the PubMed.
Evaluation of the confidence score (Cs) To evaluate the value of the developed Cs, several tests were performed. AOP-helpFinder was used to identify putative existing linkage between obesity and three stressors, which are Orlistat (a drug prescribed for obesity), Lidocaine (a local anesthetic used to relieve pain and inflammation and a cardiac depressant), and Adepal (a 2nd generation contraceptive pill). The calculation of the one-sided Fisher exact test on the obtained results provided a significant p-value (p-value < 0. 05) for Orlistat and Adepal (Fig. S1a  and S1b ), whereas the p-value obtained for Lidocaine was non-significant (p-value≈1; Fig. 2 ). For example, among the 1989 abstracts mentioning Orlistat in the PubMed database, 1158 were identified by AOP-helpFinder as co-mentioning obesity and Orlistat (Fig.
Visualization of the results Several figures and an interactive table of the obtained results are proposed to the user, both for the stressor-event, and for the event-event searches. Automatic figures will be generated with the results (evidence retrieved from abstracts from the PubMed database). Several type of figures representing various information are proposed (Fig. S3 ): the distribution of articles according to their publication date (Fig. S3a ), the distribution of links according to (i) the most common stressors (Fig. S3b ) and (ii) the most common events (Fig. S3c ), the distribution of (i) stressor-event links (Fig. S3d ) and (ii) event-event links (Fig. S3e ) with taking into account the confidence score (Fig. S3e , Fig. 3 ) and, a heatmap to assess accurately what are the links between stressors and events (Fig.
3 ). For example, links with High Evidence on AOP-Wiki such as "Cyp2E1 activation → Oxidative stress", "Oxidative stress → Hepatotoxicity" or "Cyp2E1 activation → Hepatotoxicity" have a high Cs, while links with a moderate Evidence such as "Oxidative stress → Liver cancer", "Hepatotoxicity → Liver cancer" and "Increased proliferation → Liver cancer" have a low Cs (Table  S1 ). Nevertheless, the Cs of AOP-helpFinder is based only on the PubMed bibliographic data, which may explain some differences as for example for the link "Hepatotoxicity → Increased proliferation" which has high evidence on AOP-Wiki show only a low Cs (Table  S1 ). Moreover, AOP-helpFinder is based on artificial intelligence, more precisely natural language processing using keyword for the searches, i. e. the results and the scoring can be different depending on how the event has been written by the user (e.
g. "Cyp2E1 activation" will not be considered in the same way as "Cyp2E1 induction"). In addition to retrieving all KERs from AOP 220, AOP-helpFinder identifies additional information with non-adjacent links such as "Oxidative stress → Increased proliferation" and "Cyp2E1 activation → Liver cancer" (Fig. 3 , Table  S1 ), that could be very useful to improve the existing AOPs. Following the identification of the event-event links of the AOP 220, we sought, in a second step, to link the stressors triggering this AOP to its different events using the 'stressor-event' module of AOP-helpFinder 2. 0. The AOP 220 is triggered by different substrates of Cyp2E1 such as ethanol, acetone, benzene, acetaminophen, chloroform, carbon tetrachloride or furan. After running AOP-helpFinder, links with a 'Very High' Confidence score for different Cyp2E1 substrates to oxidative stress (ethanol, carbon tetrachloride, acetaminophen, benzene) and hepatotoxicity (acetaminophen, carbon tetrachloride, ethanol) were retrieved (Fig.
S3f ). In addition to the effects of stressors on the different KEs (oxidative stress; hepatotoxicity), AOP-helpFinder also allowed the identification of other putative linkage associating stressors (e. g. ethanol) with the Adverse Outcome 'Liver Cancer' (Fig. S3f , Fig. S4 ). The use of AOP-helpFinder 2. 0 with the AOP 220 illustrates very well the high interest and potential of the proposed tool. Nevertheless, to go further and verify that AOP-helpFinder is adapted to a wide range of biological events (and not only to the tested AOP 220), analyses were independently performed on the other 20 AOPs validated by the OECD (WPHA/WNT Endorsed) that are present in the AOP-Wiki database (as of October 2022). As results, AOP-helpFinder identified more than 70 % of the KERs, and for 9 AOPs 100 % of the links were successfully retrieved  (AOP ID 3, 10, 12, 15, 40, 48, 220, 263, 360)  (Table  2 ).
For only few AOPs, the number of KERs found was quite low (40 % for AOP ID 23, 54 and 6) (Table  2 ). This can be explained by the complexity of the events defined in these AOPs in the AOP-Wiki database. The developed tool relies on text mining, which implies certain limitations in its ability to retrieve some events from abstracts, given the diversity in writing styles among researchers around the world. AOP-helpFinder is reliant on the input events provided, and if the keywords used are too specific or misspelled, the tool is likely to fail to identify any links to these terms in abstracts. For instance, most of the relationships between the complex events of AOP 23 describing reproductive dysfunction following androgen receptor agonism (e. g. , "Reduction, Gonadotropins, circulating concentrations," "Reduction, Testosterone synthesis by ovarian theca cells," "Reduction, 17beta-estradiol synthesis by ovarian granulosa cells," etc.
Discussion In this paper we present a major update of AOP-helpFinder, the event-event module, that allows to identify and extract KER to help building AOPs. To emphasize the results and support the WoE, we also integrated a scoring system, and propose an innovative way to rank the scores. Finally, to facilitate the interpretation of the results, several automatic figures are generated and an interactive table is created. Case studies, with the AOPs validated by the OECD, showed a very good ability of the tool to retrieve existing KERs (e. g. , all described links Fig. 3. Distribution of event-event linkage belonging to the AOP 220. Linkages are sorted according to the number of occurrences (i. e. the identified abstracts), and colors were added to fit the confidence score. A total of 1135 evidence, representing the 9 event-event links (KERs), were identified by AOP-helpFinder using the PubMed database.
The propose method is complementary to the existing ones that use knowledge from texts, as it uses graph theory and provides scores to rank the co-occurrence  (Carvaillo et al. , 2019) , and to other integrative data mining approaches that predict AOP signatures  (Oki and Edwards, 2016). Some limitations of the tool exist as by today it is based only on one source of information, the literature available in the PubMed database. Nevertheless, the main idea was to be able to identify and extract in an automatic manner complementary information that are published and not present in toxicological databases. A future work will therefore be to implement other data sources of information to identify as much as possible knowledge for AOP development. In the presented version of the tool, the text mining is performed on the PubMed database, more precisely on the abstracts that are peer reviewed.
PubMed database (≈35 M • abstracts) AOP-helpFinder on pre-selected PubMed abstracts (733,413 abstracts) PubMed query: (Event 1) AND (Event 2) Restrictive PubMed query: ("Event 1") AND ("Event 2") Cyp2E1 activation → 46 results 46 results 834 results 3 results Oxidative stress Cyp2E1 activation → 21 results 21 results 524 results 2 results Hepatotoxicity Oxidative stress → 753 results 753 results 4171 results 3761 results Hepatotoxicity Oxidative stress → Liver 76 results 76 results 4564 results 574 results cancer Hepatotoxicity → Increased 3 results 3 results 566 results 1 result proliferation Hepatotoxicity → Liver 28 results 28 results 2549 results 123 results cancer Increased proliferation → 29 results 29 results 11 952 results 54 results Liver cancer Calculation time 11 h 26 min instantaneous instantaneous.
Manual techniques used for home cage monitoring (HCM) include live observations or analyses of videos recorded by camera-based systems, which can be complemented or substituted by automated HCM techniques. Automated HCM techniques have been implemented in various commercial systems. A range of systems were described in a review by Richardson et al. [43]. For example, a combination of light emitting diodes (LEDs), infrared light sensors, and a camera allows for analyzing, among others, physical activity or learning behavior  [44, 45]. Electrical capacitance technology makes it possible to measure physical activity or rest  [46, 47]. As animals move about in a cage, changes in an emitted weak electromagnetic field can be used to track their movements. However, these systems do not allow the acquisition of individual data from group-housed animals.
Search strategy and screening Primary databases were searched through PubMed and Web of Science on February 24, 2021. PubMed search terms were refined using the search refiner tool QueryVis  [60]. The following search strings were used for obtaining relevant studies. Pubmed: (("mice"[MeSH Terms] OR mice OR mouse) OR ("rats"[MeSH Terms] OR rat OR rats)) AND ((home cage) OR (housing cage)) AND (monitoring OR ("observation"[MeSH Terms] OR observation)). The filter "other animals" was applied. Web of Science: ALL FIELDS: (((mice OR mouse) OR (rats OR rat)) AND (("home cage") OR ("housing cage")) AND (monitoring OR observation)). The data extraction template can be found in S1 Supporting Information. Further detailed information on the objective and materials & methods is provided in S2 Supporting Information.
In brief, only primary studies (published in English language) in which behavioral, physiological, and/or external appearance-related parameters were monitored in mice and/or rats in their home cage or a testing apparatus connected to the home cage were included. Studies in which no other parameters than calorimetric measurements were examined in a home cage were excluded. In phase 1, titles and abstracts were screened. Thereafter, full texts were screened (phase 2) and data were extracted (phase 3) simultaneously. In all phases, all papers were screened by two independent reviewers and discrepancies were resolved by a third reviewer.