dataesr/scientific-paragraphs-categorization huggingface dataset:
The smallest oS value being 3 (for example, ESR1 and ESR2 are targeted by the 3 POPs bHCH, chlordane, and chlordecone); the highest score was 8 (ESR1 and NR1I2 are targeted by endosulfan, dieldrin, bHCH, endrin, DDT, chlordane, chlordecone, and aldrin). In order to evaluate the POP-PPAN in the toxicological space, we went one step further and integrated biological events from the known AOPs into the protein complex network using both the CompTox and AOP-wiki databases. We were able to link 7 of the 18 proteins to 12 MIEs and 7 KEs (the event ID 122 is defined as a MIE and a KE in AOPwiki) that are involved in 21 different AOPs, mainly ones related to reproductive dysfunctions (see Tab. S3 4 for full names of the AOP events, and links to MIEs and KEs). Only one AOP (AOP 167, Early-life estrogen receptor activity leading to endometrial carcinoma in the mouse) is connected to two events (MIE 1064, Pre-pubertal increase, estrogen receptor (ER) activity and KE 1065, Activation, estrogen receptor alpha).
Disease-protein enrichment For disease enrichment, the DisGeNET data source, which contains human protein-disease annotations, was used to identify disorders potentially connected with the studied chemicals (Piñero et al. , 2017) (last access November, 2019). DisGenNET is the largest publicly available platform integrating knowledge on gene-disease linkages from diverse databases and published articles. The current version (v6. 0) contains 628,685 gene-disease associations between 17,549 genes and 24,166 diseases and abnormal human phenotypes.
Results Based on the CompTox and ToxCast databases, a PPAN model representative of the POP biological space was generated in order to predict potential toxic effects of suspected POPs (sPOPs). We extracted the 28 POPs from the Stockholm Convention list. Among them, five are families of compounds; therefore, we selected specific chemicals that are good representatives of these chemical families to facilitate further analysis, resulting in a total of 35 compounds used to create the PPAN model (Tab. 1). Protein information could only be retrieved for 20 chemicals when their cytotoxicity thresholds, as defined by  Judson et al. (2016) , were taken into consideration (Tab. S1 4 ). The cytotoxicity thresholds allow distinction between toxicity due to the disruption of a spe-an sPOP and proteins.
Global mapping of the 28 sPOPs in biological and disease spaces Potential toxicities were explored using the POP-PPAN model for the other 26 listed sPOPs using the approach described above for dicofol and PFOA. First, sPOP-protein data were compiled from the CompTox database (as of November 2019), including the cytotoxicity thresholds (Tab. 2). Among the 28 sPOPs, no data was retrieved for three compounds, i. e. , PCB 118, dechlorane plus, and perfluorohexane sulfonic acid (PFHxS). Three other chemicals did not pass the cytotoxicity threshold (BDE 47, BDE 99 and BDE 153), and fewer than 5 proteins were identified for 11 compounds. Therefore, only 11 sPOPs could undergo the systems toxicology analyses. Using the POP-PPAN model, first order interactor proteins were added individually to the 11 sPOP-proteins to form protein complexes (Tab.
In total, 20 significant pathways were identified with the Reactome ORA analysis, most of them involving the different cytochromes associated with dicofol. The enrichment analysis performed with the GO terms revealed that 27 of the 28 proteins are involved in metabolic processes (GO:0008152) and 10 are linked to reproduction (GO:0000003) (all biological processes, molecular functions, and cellular components are shown in Tab. S5 4 ). As a last step, disease enrichment was performed on the identified protein complex using the DisGeNET database. This allows us to highlight potential links between dicofol and disorders of the nervous system (depressive symptoms, unipolar depression, major depressive disorder, and brain ischemia with respective FDRs of 1. 11 e-6 , 3. 22 e-3 , 3. 40 e-3 , and 0.
For example, PFOA, which is quickly absorbed, can be transferred to the fetus through the placenta, and to infants via breast milk 8. Potential toxicity events were explored using the POP-PPAN model. First, data were compiled from the CompTox database (as of July 18, 2019). Considering the cytotoxicity threshold (5. 35 µM), nine proteins were extracted (Tab. S6 4 ). Using the POP-PPAN model, eight proteins were added, allowing the formation of a protein complex of 17 proteins linked to PFOA (Tab. S6 4 ) that Seven chemicals were statistically associated with the reproductive disorder female infertility (2. 56 e-06 < FDR < 7. 98 e-04 ). Some links between the reproductive system and sPOPs are already known (methoxychlor), others have not been characterized yet (diazinon, dimethomorph, pyrimethanil) (PubMed, as of December 9, 2019).
Discussion The proposed computational network-based model is different and complementary to existing molecular modelling approaches such as quantitative structure activity relationships (QSAR) and quantitative structure property relationships (QSPR) that are based on structural information of chemicals. For example, efficient QSAR and QSPR models have been developed to predict, in a quantitative manner, eco-toxicological, half-life or chemical properties of environmental interest that can be used for regula-ious metabolism pathways were retrieved as statistically significant, such as linoleic acid, retinol, arachidonic acid or tryptophan metabolism. Within the Reactome enrichment, mainly lipid metabolism-related pathways were identified. The last analysis was performed using the DisGeNet database in order to decipher linkage between the sPOPs and diseases.
Linkages between chemicals and disorders were obtained by an over-representation analysis using the DisGeNet database on the protein complexes associated to each chemical that were identified using the developed POP-protein network. The width of the edges is according to statistical significance. With the advances in data generation and bioinformatics technology, more heterogeneous data will be publicly accessible and could be integrated, such as qualitative and quantitative information and know-how from animal, clinical, epidemiological, exposure and biomonitoring studies, allowing the development of emerging approaches to help understand the effects of POPs on human health. The challenge will be to map and integrate all these data into a common platform  (Pawar et al. , 2019). The use of innovative informatics technology is also useful as this will allow the identification, in a systemic manner, of already known and published information.
Tab. 2: List of the 28 chemicals defined as POP candidates Chemicals that have proteins associated to them in the ToxCast database considering the cytotoxicity threshold are shown in grey. Chemicals that have more than 4 proteins associated to them, for which we performed an ORA analysis, are in bold font. NA, not analyzed for enrichment (proteins < 4) Inital POPs CAS rn Newer POPs CAS rn Aldrin 309-00-2 Insecticide Alpha Hexachlorocyclohexane 319-84-6 Insecticide Chlordane 57-74-9 Pesticide Beta Hexachlorocyclohexane 319-85-7 Insecticide Dichlorodiphenyltrichloroethane (DDT) 50-29-3 Pesticide Chlordecone (=Kepone) 143-50-0 Insecticide Dieldrin 60-57-1 Insecticide Decabromodiphenyl ether (DecaBDE = BDE 209) 1163-19-5 Flame retardant Endrin 72-20-8 Pesticide Hexabromobiphenyl (PBB 153) 36355-01-8 Flame retardant Heptachlor 76-44-8 Pesticide Hexabromocyclododecane 25637-99-4 Flame retardant Hexachlorobenzene (HCB) 118-74-1 Fungicide, additive to fireworks, ammunition & synthetic rubber Hexabromodiphenyl ether 36483-60-0 Mirex 2385-85-5 Pesticide, fire retardant Heptabromodiphenyl ether 68928-80-3 Toxaphene 8001-35-2 Pesticide Hexachlorobutadiene 87-68-3 Polychlorinated biohenyls (PCB)* Insulating fluid; paint; additive; lubricant Lindane 58-89-9 Insecticide -2,2'-Dichlorobiphenyl 13029-08-8 Pentachlorobenzene 608-93-5 Pesticide -2,2',3,4',5,5',6-Heptachloro-1, 1'-biphenyl 52663-68-0 Pentachlorophenol 87-86-5 -2,2',4,4',5,5'-Hexachlorobiphenyl 35065-27-1 Perfluorooctane sulfonic acid (PFOS) 1763-23-1 Protective coating Polychlorinated dibenzo-p-dioxins (PCDD)* Herbicide Perfluorooctane sulfonyl fluoride (PFOSF) 307-35-7 Protective coating -2,3,7,8-Tetrachlorodibenzodioxin (=TCDD) 1746-01-6 Pentabromodiphenyl ether 32534-81-9 Flame retardant Polychlorinated dibenzofurans (PCDF)* Herbicide Tetrabromodiphenyl ether 40088-47-9 -2,3,4,7,8-Pentachlorodibenzofuran 57117-31-4 Polychlorinated naphthalenes* -1,2,3,7,8-Pentachlorodibenzofuran 57117-41-6 -1-Chloronaphthalene 90-13-1 Endosulfan 115-29-7 Pesticide Short chain chlorinated parafins (SCCPs)* -chlorinated paraffins: C12, 60% chlorine no CAS rn (PubChem ID: 5284361) 2.
However, AAT is known to be endemic and only a specific cattle race (trypanotolerant Ndama) is adapted in the area. T. vivax, T. congolense and, in a lesser extent, T. brucei are also recorded in neighbouring countries. Based on the data available from 48 different health centers within the Kédougou region  [15] , six villages were purposively selected based on P. falciparum prevalence infection outcomes from outpatients (Fig. 1 , Additional file 1). Mako, Bantaco and Tomboronkoto exhibited the lowest Plasmodium prevalence (respectively 46. 2%, 36. 5% and 36. 1%) while Silling, Ndébou and Boundoucondi displayed the highest Plasmodium prevalence (respectively 73. 2%, 73. 5% and 75. 4%).
Discussion The objective of this study was to update the epidemiological profile of AAT in South-East Senegal and to investigate a possible link between Trypanosoma infection and Anopheles vectorial capacities for malaria. The presence of Trypanosoma was investigated in domestic animals from each of the 6 selected villages. Our results showed the presence of T. vivax in ten animals out of 284 in Bantaco, Mako and Tomboronkoto; the three sites that showed the lowest Plasmodium infections in human at the beginning of the study. In these sites, no Plasmodium infection was observed in mosquitoes. Several studies conducted in the area have shown the presence of trypanotolerant animals  [9, [21] [22] [23]. The detection of T. vivax in animals from the Kédougou region confirms the data already obtained by Touré  [9]  who showed that this species was mainly found in cattle and horses, not only in this area but also across all the country and also the presence of T.
i. d. Gaussian ones. We show analytically that the generalisation performance deteriorates as the manifold dimension D grows. We show that the learning rate has a very minor influence on the asymptotic error, and analyse how the error final error of the network changes as a function of the width of the hidden layer. Sec. V is devoted to comparison of learning on the HMM and on real data sets such as MNIST  [10] , fashion-MNIST  [24]  or CIFAR10  [11]. In particular, we demonstrate that neural networks learn functions of increasing complexity over the course of training on both the HMM and real data sets. We also compare the memorisation of some samples during the early stages of training between the HMM to real data. These comparisons provide strong evidence that the HMM captures the properties of learning with one-pass SGD and two-layer neural networks on some of the standard benchmark data sets rather faithfully.
Recently, Yoshida and Okada  [36]  analysed the dynamics of online learning for data having an arbitrary covariance matrix, finding an infinite hierarchy of ODEs. Their study implicitly assumes that inputs have a Gaussian distribution, while our approach handles a more general data structure. The importance of the spectral properties of the data was recognised for learning in deep neural networks by Saxe et al. [37]  in the special case of linear neurons, where the whole network performs a linear transformation of the data. Relation to random feature learning. The hidden manifold model has an interesting link to random feature learning with unstructured i. i. d. input data. The idea of learning with random features goes back to the mechanical perceptron of the 1960s  [38]  and was extended into the random kitchen sinks of Rahimi and Recht  [39, 40].
The principles underlying the analytic solution of this paper as well as  [41] [42] [43]  rely on the Gaussian Equivalence Property, which is stated and used independently in those papers. Gaussian equivalence and random matrix theory. Special cases of the Gaussian Equivalence Property were in fact derived previously using random matrix theory in  [44] [45] [46] [47] , and this equivalent Gaussian covariates mapping was explicitly stated and used in  [42, 43]. This formulation has recently been extended to a broader setting of concentrated vectors encompassing data coming from a GAN in  [48, 49] , a version closer to our formulation.
B. Memorisation of random and realistic data An interesting difference between random and realistic data was demonstrated in a recent paper by Arpit et al. [86]. They trained 100 two-layer networks (K = 4096 hidden units with ReLU activation, 10 output units with softmax activation) for a single epoch on the ten-class image classification task on the CIFAR10 data set, starting from different initial conditions each time. At the end of training, they measured the frequency with which each individual image was classified correctly by the network across runs, which we will call the memorability of an image, which should be thought of as a function of the image and the data set that contains it. We repeated this experiment on CIFAR10, and added three different synthetic data sets: (colour codes refer to Fig.
9 ): CIFAR10: x µ : CIFAR10 images; y * µ ∈ [0, 9]: CIFAR10 label giving the class of that image. Gaussian: teacher acting on Gaussian inputs: x µ i. i. d. standard Gaussians; y * µ = arg max φ(x µ , θ * ) TeacherS: teacher acting on structured inputs x µ = f (F c µ ); y * µ = arg max φ(x µ , θ * ) HMM: x µ = f (F c µ ); y * µ = arg max φ(c µ , θ * ) The labels for the synthetic data sets were generated by two teacher networks, one with input dimension N for the Gaussian and TeacherS data sets, and another with input dimension D for the HMM. The teachers were two-layer fully connected networks having M = 2K hidden units with ReLU activation function, and 10 nodes in the last readout layer. Thus the teacher's output φ(•, θ * ) ∈ R 10 , and the class for a given input was obtained as the index of the output node with the highest value for that input.
We plot the memorabilities for all images in the training set, sorted by their memorability, in Fig. 9. On the left, we first reproduce the memorability curve for CIFAR10 that was found by Arpit et al. [86]  (solid blue), which demonstrates that many examples are consistently classified correctly or incorrectly after a single epoch of training. The memorability curve for a data set containing the same images with random labels (dashed blue) demonstrates that randomised CIFAR10 doesn't contain images that are particularly hard or easy to memorise. The smaller variation in memorability for the randomised data set is largely due to the fact that it takes it more time to fit randomised data sets  [87]. After one epoch, the network thus has a lower training accuracy on the randomised data set (cf.
the inset of Fig. 9 ), which leads to the smaller area underneath the curve. We verified that no easy or hard samples appear when training the randomised data sets to comparable training accuracy (not shown). In fact, the memorability of data sets with random labels seem to coincide after accounting for differences in the training error, regardless of whether the inputs are CIFAR10 images, Gaussian inputs or structured inputs X = f (CF ) (4) (dashed lines in Fig. 9 a ). The memorability curves for the Gaussian, TeacherS and HMM data sets in Fig. 9  (b) reveal that hard and easy examples exist for TeacherS and HMM, which both contain structured inputs X = f (CF ), but not in the Gaussian data set. The number of easy examples, but not their existence, correlates well with the training accuracy on these data sets, shown in the inset.
In that sense, the hidden manifold model is thus a more realistic model of image-like data than the canonical teacher-student setup. Note that by making the teacher network larger than the students (M = 2K), the learning problem is unrealisable for all three synthetic data sets, i. e. there is no set of weights for the student that achieve zero generalisation error. The absence of easy examples in the Gaussian data set thus suggests that unrealisability alone is insufficient to obtain a data set with easy examples. Our results also demonstrate that memorability is not just a function of the input correlations: CIFAR10 images, Gaussian inputs and structured inputs yield the same memorability curves when their labels are randomised. We leave it to future work to identify some criterion, statistical or otherwise, that predicts either whether a sample (x µ , y * µ ) is easy (or hard!) to memorise or whether a training set contains easy examples at all.
VI. CONCLUDING PERSPECTIVES We have introduced the hidden manifold model as a generative model for structured data sets that displays some of the phenomena that we observe when training two-layer neural networks on realistic data. The HMM has two key ingredients, namely high-dimensional inputs which lie on a lower-dimensional manifold, and labels for these inputs that depend on the inputs' position within the low dimensional manifold. We derived an analytical solution of the model for online SGD learning of twolayer neural networks. We thus provide a rich test bed for exploring the influence of data structure on learning in neural networks. Let us close this paper by outlining several important directions in which our work is being (or should be) extended. Comparison to more deep learning phenomenology In the spirit of our experiments in Section V, it is of great interest to identify more properties of learning that are consistently reproduced across experiments with realistic data sets and network architectures, and to test whether the HMM reproduces these observations as well.
Of particular interest will be those cases where learning on realistic data deviates from the HMM, and how we can extend the HMM to capture these behaviours. Beyond online SGD Our analytical results on online SGD rely on the assumption that each new sample seen during training is conditionally independent from the weights of the network up to that point. In practice, samples are seen several or even many times during training, giving rise to additional correlations. Taking those correlations into account to analyse those cases is an important future direction. First steps towards a solution to this challenging problem were made using the dynamical replica method  [88, 89]  for two-layer networks, and for single-layer neural networks trained using full-batch gradient descent, where all the samples in the training set are used at every step of the algorithm  [42, 43, 90].
Conditioning the inputs on the labels In the HMM, the true label y * of an input x is conditioned on its latent representation c, i. e. its coordinates in the manifold. It may be more realistic to consider models where instead, the latent representation is conditioned on the label of the input, i. e. p(c|y). A simple case of such a model that reduces to a Gaussian mixture of two clusters was explored recently  [91]. This is also the point of view taken implicitly in  [33]. More generally, exploring different approaches to modelling realistic inputs will allow us to better understand how data structure influences learning.
Yet, we can find closed equations by changing basis for S. FIG. 11. Two-layer ReLU neural networks also learn functions of increasing complexity, show more run-to-run variance. We plot the mean-squared error as a function of training time for sigmoidal networks with increasing number of nodes K when trained on three different data sets. For all plots, g(x) = max(0, x), gaussian initial weights with std. dev. 10 -3 , batch size 32. For online learning, we chose a teacher with g(x) = max(x, 0), M = 10, ṽm = 1 /M. CIFAR10: air plane, bird, deer, frog, ship vs. automobile, cat, dog, horse and truck. We first demonstrate in Fig. 10  that sigmoidal networks show the same learning of functions of increasing complexity discussed in Sec. V A for CIFAR10 when trained on MNIST or FMNIST. Note that for CIFAR10 in particular, we see the effects of over-training set in after several epochs, when the generalisation error starts to increase again (we use plain SGD without any explicit regularisation in these experiments).
ACKNOWLEDGMENTS We would like to thank  Bruno Loureiro  and  Federica Gerace  for useful discussions. We thank  Stanisław Jas -trzębski for discussing the experiments of [86]. We are grateful to the  Kavli Institute For Theoretical Physics  for its hospitality during an extended stay, during which parts of this work were conceived and carried out. We acknowledge funding from the  ERC  under the European Union's  Horizon 2020 Research and Innovation Programme Grant Agreement 714608-SMiLe , from " Chaire de recherche sur les modèles et sciences des données ",  Fondation CFM pour la Recherche-ENS , and from the  French National Research Agency (ANR)  grant PAIL. This research was supported in part by the  National Science Foundation  under Grant No. NSF   PHY-1748958.
First, we validated that data of benthic invertebrates from bottom trawl surveys could be used to estimate an index of benthic prey production, by comparing data collected by grabs and trawls at the same sampling locations. Using this index on an extended trawl dataset, we estimated inter-annual variability of benthic prey production among several nurseries along the coast of the Bay of Biscay. Estimates of benthic prey production index were similar among nurseries, although, these nurseries displayed different local hydrological patterns (currents and residence time). The index was finally used to investigate whether benthic prey production limits young-of-the-year fish density using quantile regressions. We found a significant and positive relationship between the benthic prey production index and young-of-the-year fish densities, including flatfish and round fish species.
Although not demonstrating a causal relationship, the analysis supported the hypothesis that when food supply is low, birds are unlikely to allocate energy to breeding. Similarly, along the North Sea coast, a longterm dataset showed a decrease in nutrient loading, which presumably caused a change in the spatial distribution of Pleuronectes platessa juveniles  (Støttrup et al. , 2017). Joint analysis of predator and prey could provide arguments in the debate on food limitation, especially when it is performed in a causation context  (Kato and Sasaki, 2017). Quantile regressions used in this context may be very helpful in identifying limiting factors  (Cade and Noon, 2003; Planque and Buffaz, 2008) , since the food-limitation hypothesis can be tested by examining several quantiles of the predator-prey relationship  (Tableau et al.
We investigated whether benthic prey production limits the density of juvenile fish in coastal and estuarine nurseries using data collected solely from trawl surveys. First, we verified that data from bottom trawl surveys could be used to estimate an index of benthic prey production, which is traditionally assessed using grab samples. Second, using this index, we estimated inter-annual variability in benthic prey production among several nurseries. Third, we used quantile regressions to test the hypothesis that trophic limitation occurs in young-of-the-year (YOY) bentho-demersal fish density in nurseries. This potential limitation was tested for a variety of YOY fish, including flatfish and round fish species. The study included nurseries located along the French coast of the Bay of Biscay (Figure  1 ; Table  1 ) that were previously identified as important grounds for juvenile flatfish (Le  Pape et al.