dataesr/scientific-paragraphs-categorization huggingface dataset:
Complexity analysis for training and prediction Computational complexity is of paramount importance in GPs as it quickly scales with large datasets. The classical cost to train a GP is O(N 3 ), and O(N 2 ) for prediction (Rasmussen and  Williams, 2006)  where N is the number of data points (see aforementioned references in Sec. 1 for sparse approximations). Since Magma uses information from M individuals, each of them providing N i observations, these quantities determine the overall complexities of the algorithm. If we recall that N is the number of distinct timestamps (i. e. N ≤ M i=1 N i ), the training complexity is O M × N 3 i + N 3 (i. e. the complexity of each EM iteration). As usual with GPs, the cubic costs come from the inversion of the corresponding matrices, and here, the constant is proportional to the number of iterations of the EM algorithm.
The dominating term in this expression depends on the values of M , relatively to N. For a large number of individuals with many common timestamps (M N i N ), the first term dominates. For diverse timestamps among individuals (M N i N ), the second term becomes the primary burden, as in any GP problem. During the prediction step, the re-computation of µ 0 's hyper-posterior implies the inversion of a Ñ × Ñ (dimension of t p * ) which has a O( Ñ 3 ) complexity while the final prediction is O(N 3 * ). In practice, the most computing-expensive steps can be performed in advance to allow for quick on-the-fly prediction when collecting new data. If we observe the training dataset once and precompute the hyper-posterior of µ 0 on a fine grid on which to predict later, the immediate computational cost for each new individual is identical to the one of the single-task GP regression.
Experimental results We evaluate our Magma algorithm on synthetic data, and two real datasets. The classical GP regression on single tasks separately is used as the baseline alternative for predictions. While it is not expected to perform well on the dataset used, the comparison highlights the interest of multi-task approaches. To our knowledge, the only alternative to Magma is the GPFDA algorithm from  Shi et al. (2007) ,  Shi and Choi (2011) , described in Sec. 3. 4, and the associated R package GPFDA, which is applied on the examples. Throughout the section, the standard Exponentiated Quadratic kernel (see Eq. (  5 )) is used both for simulating the data and for the covariance structures in the three algorithms. Hence, each kernel is associated with θ = {v, }, v, ∈ R + , a set of, respectively, variance and length-scale hyper-parameters.
Each simulated dataset has been drawn from the sampling scheme below: 1. Define a random working grid t ⊂ [ 0, 10 ] of N = 200 timestamps, and a number M of individuals. 2. Define the prior mean for µ 0 : m 0 (t) = at + b, ∀t ∈ t, where a ∈ [ -2, 2 ] and b ∈ [ 0, 10 ]. 3. Draw uniformly hyper-parameters for µ 0 's kernel : θ 0 = {v 0 , 0 }, where v 0 ∈ 1, e 5 and 0 ∈ 1, e 2. 4. Draw uniformly µ 0 (t) ∼ N m 0 (t), K t θ 0. 5. For all i = 1,. , M , θ i = {v i , i }, where v i ∈ 1, e 5 , i ∈ 1, e 2 , and σ 2 i ∈ [ 0, 1 ]. 6. For all i = 1,. , M , draw a subset uniformly at random t i ⊂ t of N i = 30 timestamps, and draw y i ∼ N µ 0 (t i ), Ψ t i θ i ,σ 2 i. This procedure provides a synthetic data set {t i , y i } i , and its associated mean process µ 0 (t). Those quantities are used to train the model, make predictions with each algorithm, and then compute errors in 0 estimation and forecasts.
We recall that the Magma algorithm enables two different settings depending on the model's assumption over hyper-parameters (HP), and we refer to them as Common HP and Different HP in the following. In order to test these two contexts, differentiated datasets have been generated, by drawing Common HP data or Different HP data for each individual at step 5. We previously presented the idea of the model used in GPFDA, and, although the algorithm has many features (in particular about the type and number of input variables), it is not yet usable when timestamps are different among individuals. Therefore, two frameworks are considered, Common grid and Uncommon grid, to take this specification into account. Thus, the comparison between the different methods can only be performed on data generated under the settings Common HP and Common grid, and the effect of the different settings on Magma is analysed separately.
Illustration on a simple example To illustrate the multi-task approach of Magma, Fig. 2  displays a comparison between single GP regression and Magma on a simple example, from a dataset simulated according to the scheme above. Given the observed data (in black), values on a thin grid of unobserved timestamps are predicted and compared, in particular, with the true test values (in red). As expected, GP regression provides a good fitting close to the data points and then dives rapidly to the prior 0 with increasing uncertainty. Conversely, although the initialisation for the prior mean was also 0 in Magma, the hyper-posterior distribution of µ 0 (dashed line) is estimated thanks to all individuals in the training dataset. This process acts as an informed prior helping GP prediction for the new individual, even far from its own observations.
More precisely, 3 phases can be distinguished according to the level of information coming from the data: in the first one, close to the observed data (t ∈ [ 1, 7 ]), the two processes behave similarly, except a slight increase in the variance for Magma, which is logical since the prediction also takes uncertainty over µ 0 into account (see Eq. (  9 )); in the second one, on intervals of unobserved timestamps containing data points from the training dataset (t ∈ [ 0, 1 ] ∪ [ 7, 10 ]), the prediction is guided by the information coming from other individuals through µ 0. In this context, the mean    [ 10, 12 ] ), the prediction behaves as expected, with a slow drifting to the prior mean 0, with highly increasing variance. Overall, the multi-task framework provides reliable probabilistic predictions on a wider range of timestamps, potentially outside of the usual scope for GPs.
Performance comparison on simulated datasets We confront the performance of Magma to alternatives in several situations and for different datasets. In the first place, the classical GP regression (GP), GPFDA and Magma are compared through their performance in prediction and estimation of the true mean process µ 0. In the prediction context, the performances are evaluated according to the following indicators: • the mean squared error (MSE) which compares the predicted values to the true test values of the 10 last timestamps: 1 10  • the ratio of CI 95 coverage, i. e. the percentage of unobserved data points effectively lying within the 95% credible interval defined from the predictive posterior distribution p(y * (t p )|y * (t * ), {y i } i ): 30 k=21 y pred i (t k i ) -y true i (t k i ) 2 , 100 × 1 10 30 k=21 1 {y true i ∈ CI 95 }.
3  that Magma consistently provides lower errors as well as less pathological behaviour, as it may sometimes occur with the B-splines modelling used in GPFDA. To highlight the effect of the number of individuals M on the performance, Fig. 3  provides the same 100 runs trial as previously, for different values of M. The boxplots exhibit, for each method, the behaviour of the prediction and estimation MSE as information is added in the training dataset. Let us mention the absence of discernible changes as soon as M > 200. As expected, we notice on the right panel that adding information from new individuals improves the estimation of µ 0 , leading to shallow errors for high values of M , in particular for Magma. Meanwhile, the left panel exhibits reasonably unchanged prediction performance with respect to the values of M , excepted some random fluctuations.
This property is expected for GP regression, since no external information is used from the training dataset in this context. For both multi-tasks algorithms though, the estimation of µ 0 improves the prediction by one order of magnitude below the typical errors, even with only a few training individuals. Furthermore, since a new individual behaves independently through f * , it is natural for a 10-points-ahead forecast to present intrinsic variations, despite an adequate estimation of the shared mean process. To illustrate the advantage of multi-task methods, even for M = 20, we display on Fig. 4  the evolution of MSE according to the number of timestamps N that are assumed to be observed for the new individual on which we make predictions. These predictions remain computed on the last 10 timestamps, although in this experiment, we only observe the first 5, 10, 15, or 20 timestamps, in order to change the volume of information and the distance from training observations to targets.
Running times comparisons The counterpart of the more accurate and general results provided by Magma is a natural increase in running time. Table  3  exhibits the raw and relative training times for GPFDA and Magma (prediction times are negligible and comparable in both cases), with varying values of M on a Common grid of N = 30 timestamps. The algorithms were run under the 3. 6. 1 R version, on a laptop with a dual-core processor cadenced at 2. 90GhZ and an 8Go RAM. The reported computing times are in seconds, and for small to moderate datasets (N 10 3 , M 10 4 ) the procedures ran in few minutes to few hours. The difference between the two algorithms is due to GPFDA modelling µ 0 as a deterministic function through B-splines smoothing, whereas Magma accounts for uncertainty. The ratio of computing times between the two methods tends to decrease as M increases, and stabilises around 2 for higher numbers of training individuals.
Data and problematic We consider the problem of performance prediction in competition for french swimmers. The French Swimming Federation (FFN) provided us with an anonymised dataset, compiling the age and results of its members between 2000 and 2016. For each competitor, the race times are registered for competitions of 100m freestyle (50m swimming-pool). The database contains results from 1731 women and 7876 men, each of them compiling an average of 22. 2 data points (min = 15, max = 61) and 12 data points (min = 5, max = 57) respectively. In the following, age of the i-th swimmer is considered as the input variable (timestamp t) and the performance (in seconds) on a 100m freestyle as the output (y i (t)). For reasons of confidentiality and property, the raw dataset cannot be published. The analysis focuses on the youth period, from 10 to 20 years, where the progression is the most noticeable.
In order to get relevant time series, we retained only individuals having a sufficient number of data points on the considered time period. For a young swimmer, observed during its first years of competition, we aim at modelling its progression curve and make predictions on its future performance in the subsequent years. Since we consider a decision-making problem involving irregular time series, the GP probabilistic framework is a natural choice to work on. Thereby, assuming that each swimmer in the database is a realisation y i defined as previously, we expect Magma to provide multi-task predictions for a new young swimmer, that will benefit from information of other swimmers already observed at older ages. To study such modelling, and validate its efficiency in practice, we split the individuals into a training and testing datasets with respective sizes: • M F train = 1039, for the female training set, • M F test = 692, for the female testing set, • M M train = 4726, for the male training set, • M M test = 3150, for the male testing set.
Inference on the hyper-parameters is performed thanks to the training dataset in both cases. Considering the different timestamps and the relative monotony of the progression curves, the settings Uncommon grid /Common HP has been used for Magma. The overall training lasted around 2 hours with the same hardware configuration as for simulations. To compute MSE and the CI 95 coverage, the data points of each individual in the testing set has been split into observed and testing timestamps. Since each individual has a different number of data points, the first 80% of timestamps are taken as observed, while the remaining 20% are considered as testing timestamps. Magma's predictions are compared with the true values of y i at testing timestamps. As previously, both GP and Magma have been initialised with a constant 0 mean function.
As previously observed in the simulation study, the simple GP quickly drifts to the prior 0 mean, as soon as data lack. However, for both men and women, the Magma predictions remain close to the true data, which also lie within the 95% credible interval. Even for long term forecast, where the mean prediction curve tends to overlap the mean process (dashed line), the true data remain in our range of uncertainty, as the credible interval widens far from observations. For clarity, we displayed only a few individuals from the training dataset (colourful points) in the background. The mean process (dashed line) seems to represent the main trend of progression among swimmers correctly, even though we cannot numerically compare µ 0 to any real-life analogous quantity. In a more sportrelated perspective, we can note that both genders present similar patterns of progression.
Discussion We have introduced a unified non-parametric multi-task framework integrating a mean Gaussian process prior in the context of GP regression. While we believe that this process is an interesting object in itself, it also allows individuals to borrow information from each other and provide more accurate predictions, even far from data points. Furthermore, our method accounts for uncertainty in the mean process and remains applicable no matter the observational grid of data. Both on simulated and real-life datasets, we exhibited the adequacy of such an approach, and studied some of its properties and possible settings. Magma outperforms the alternatives in estimation of the mean process as well as in prediction, and gives a reliable quantification of uncertainty. We also displayed evidence of its predictive efficiency for real-life problems and provided some insights on practical interpretation about the mean process.
For each of these datasets, common (minor allele frequency (MAF) ≥0. 01) high-quality variants were extracted from the whole-genome sequence data. An unrelated subset of individuals of genetically defined European (EUR) ancestry (Supplementary Fig. 2 ), was identified by kinship coefficient estimation and principal component analysis. After quality control, the NIHR-RD dataset comprised 216 cases and 8436 controls with 8,291,828 variants, and the 100KGP dataset comprised 182 cases and 45,585 controls with 9,429,238 variants. A GWAS was performed in each cohort separately using a generalized mixed model association test with saddle-point approximation to adjust for case-control imbalance using SAIGE  13  with sex and ten principal components (PCs) as covariates. The resulting summary statistics for variants with matching alleles and allele frequencies between the two datasets were then meta-analyzed using METAL  14  , weighting the effect size estimates using the inverse of the standard errors.
Variants showing heterogeneity of effect between the two datasets (P < 1 × 10 -5 ) were excluded. In total, 8,199,999 variants were meta-analyzed and four loci were identified as achieving genomewide significance (P < 5 × 10 -8 ) (Supplementary Figs. 1, 3a ). The genomic inflation (lambda) was 1. 01 (Supplementary Fig. 3c ), indicating no significant population stratification. To enhance the power for genetic discovery, we meta-analyzed the NIHR-RD/100KGP (UK) ICP GWAS meta-analysis summary statistics with GWAS data from FinnGen (Release 4) (https:// www. finngen. fi/en). FinnGen is a public-private partnership combining digital health record data from Finnish health registries with genotyping data from Finnish Biobanks. Release 4 includes association data at 16,962,023 variants for 2444 endpoints in 176,899 Finnish individuals.
2 ). We included adult liver tissue active chromatin features in the datasets used to prioritize causal variants, including accessible chromatin regions and those enriched in H3K27ac (see "Methods"). This analysis enabled the prioritization of nine variants with a high functional posterior probability (PP func > 0. 8) across eight ICP susceptibility loci (Supplementary Data 2). For the remaining three loci, which did not include any protein-altering variant, we assumed that variants residing within active liver CREs were more likely functional. We thus defined a high-confidence set of human liver accessible chromatin sites using ATAC-seq profiles from four whole liver samples (data from ENCODE  18  , see "Methods"), which allowed us to further prioritize one likely causal variant at each remaining locus (Table  2 ).
3b, c ), genes that are essential for hepatic metabolic homeostasis and when mutated may lead to monogenic diabetes  22. We examined the potential functional impact of these coding variants through the querying of the ClinVar variant interpretation database and Metadome analysis  23. At GCKR, we identified an ICP risk variant, rs1260326 (Leu446Pro), which has been previously associated with altered Fig. 1  Manhattan plot for the genome-wide association study meta-analysis of intrahepatic cholestasis of pregnancy (ICP). Data shown are from the combined meta-analysis including 1138 cases and 153,642 controls, all of European ancestry. The chromosomes are ordered on the x axis; the y axis shows the -log 10 (P) values for the association tests. Eleven loci achieved genome-wide significance (P < 5 × 10 -8 , indicated by the red line).
The frequencies of the risk alleles in control individuals in the UK (UK Freq) and Finnish (Finn Freq) datasets are shown. Association testing was performed using a generalized logistic mixed model to account for population stratification and saddle-point approximation to control for type 1 error rates due to unbalanced case-control ratios, followed by meta-analysis weighting the effect size estimates using the inverse of the standard errors. serum fasting plasma glucose and triglyceride concentrations  24, 25  and nonalcoholic fatty liver disease (NAFLD)  26. Previous functional studies of this variant have indicated that the risk allele in this locus is associated with increased glucokinase activity in the liver  27. At HNF4A, a single variant was associated with ICP susceptibility, rs1800961 (Thr139Ile), which Metadome analysis  23  predicted to be deleterious for protein function (Supplementary Fig.
7a ). Functional prioritization of noncoding variants at lipid and bile acid homeostasis loci. At the remaining loci identified in the GWAS meta-analysis for ICP, we observed that seven of the eleven loci contained noncoding and synonymous variants (examining the 95% genetic credible set, as well as the lead variant and those in linkage equilibrium at r 2 > 0. 8 in EUR individuals). This observation is in line with previous reports suggesting that common sequence variation predominantly contributes to human disease via disruption of noncoding cis-regulatory elements  28, 29. We therefore took advantage of the functional fine-mapping strategy described above, along with the integration of adult liver epigenomic datasets and pathogenicity measures, to further dissect the causal mechanisms at play in each ICP locus (Fig.
Combined analysis of adult liver epigenomic datasets and a high-throughput reporter assay dataset from liver cells (Survey of Regulatory Elements, SuRE  31  ), indicated that the lead variant rs4148211 (PP func > 0. 99) resides within a hepatic CRE and its risk allele is associated with allele-dependant transcriptional activity in liver cells (P = 9. 1 × 10 -4 ), where the risk allele is associated with transcriptional repression (Fig. 4b  and Supplementary Fig. 7c ). Analysis of the SuRE assay dataset from liver cells revealed seven additional ICP-associated variants in the ABCG8 locus for which the risk allele associates with Fig. 2  Flowchart describing the variant prioritization strategy followed for the eleven loci reaching genome-wide significance in the GWAS meta-analysis of ICP. Sequential steps used to prioritize the variants are shown with the different approaches for nonsynonymous, synonymous, and noncoding variants indicated.
94), which overlapped an active enhancer element (accessible chromatin enriched in H3K27ac) (Fig. 4c ). Transcription factor binding motif disruption analysis further revealed that the risk allele of rs55747905 (C) disrupts a recognition sequence for the Wnt signaling transcription factor TCF7L2 (Fig. 4d ). At the third ABC transporter locus, we prioritized ABCB11 as the gene mediating the ICP susceptibility process. In this case, functional fine-mapping with PAINTOR enabled the prioritization of a single variant residing in an intronic liver enhancer in the DHRS9 gene (Fig. 4e ). However, analysis of ENCODE  18  RNA-seq data revealed that DHRS9 has very weak expression in adult liver, in contrast with ABCB11, which encodes the bile salt export pump (BSEP). The ICP association signal on chromosome 8 is located between the UBXN2A and CYP7A1 genes (Fig.
18 × 10 -6 ). Even though this variant was not a statistically significant eQTL for CYP7A1 in the GTEx database (P = 0. 15), we observed a similar trend on the direction of effect of the risk allele (Fig. 5b ) and previous chromatin conformation capture and genome editing studies in human hepatocytes have linked this CRE with the modulation of CYP7A1 expression  35. These previous studies, combined with the functional significance of CYP7A1 in bile acid homeostasis, led us to prioritize CYP7A1 as the major effector transcript in this locus. Motif analysis further revealed that the risk allele disrupts a binding site for the transcription factor DMRTA1 (Fig. 5c ). In addition to CYP7A1, the ICP GWAS meta-analysis identified another important locus for bile acid homeostasis, SULT2A1, which encodes the enzyme Sulfotransferase 2A1, highly expressed in the liver and capable of phase II metabolism of bile acids  36  (Fig.
We next explored the remaining three ICP-associated signals where the closest genes were not directly involved in lipid and bile acid homeostasis. On chromosome 19, even though it was not possible to employ PAINTOR for functional finemapping due to the presence of ambiguous alleles (A/T or G/C), we prioritized the variant rs4806173 based on its overlap with a hepatic CRE (Fig. 6a ). This variant resides in an intron of the testis-specific gene GAPDHS  37. We thus queried eQTL GTEx  34  data to identify potential targets of this regulatory variant in liver tissue. This analysis led us to pinpoint TMEM147 (transmembrane protein 147) as the likely effector transcript of this association signal, with the risk allele associating with lower hepatic gene expression (1. 52 × 10 -40 ) (Fig. 6b ). Using transcription factor motif disruption analysis, we detected that the risk allele of rs4806173 creates a recognition sequence for the glucocorticoid receptor (Fig.
The final signal encompassed intronic variants in the SHROOM3 gene, including one variant in a hepatic CRE (rs4859682) (Fig. 7a ). However, analysis of liver eQTLs did not assist in the identification of potential effector transcripts in the locus. Topologically associating domains (TADs) correspond to self-interacting genomic domains that contain genes and regulatory elements that interact more frequently with each , respectively. The LocusZoom plots are as per Fig. 3  and association testing was performed as described in Fig. 1. At each locus, a single noncoding variant was prioritized as likely causal following the strategy outlined in Fig. 2. All epigenomic and transcriptomic datasets shown were retrieved from ENCODE  18. Adult liver ATAC-seq peaks track (green regions) corresponds to regions accessible in at least two out of four ENCODE adult liver ATAC-seq (see Methods).