dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgements This study was supported by the project grant from the  Canadian Institutes of Health Research  ( 153010 ). We would like to thank the participants of the Saguenay Youth Study cohort and their families for their participation in the project. This work received support from the following sources: the  European Union -funded  FP6  Integrated Project IMAGEN (Reinforcement-related behavior in normal brain function and psychopathology) ( LSHM-CT-2007-037286 ), the  Horizon 2020  funded  ERC Advanced Grant 'STRATIFY' (Brain network  based stratification of reinforcement-related disorders) (695313),  Human Brain Project  ( HBP SGA 2 ,  785907 , and  HBP SGA 3, 945539 ), the  Medical Research Council Grant 'c-VEDA' (Consortium on Vulnerability to Externalizing Disorders and Addictions)  ( MR/N000390/1 ), the  National Institute of Health (NIH)  ( R01DA049238 ), A decentralized macro and micro gene-byenvironment interaction analysis of substance use behavior and its brain biomarkers), the  National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London , the  Bundesministeriumfür Bildung und Forschung (BMBF  grants  01GS08152 ;  01EV0711 ;  Forschungsnetz AERIAL 01EE1406A ,  01EE1406B ;  Forschungsnetz IMAC-Mind   01GL1745B ), the  Deutsche Forschungsgemeinschaft (DFG  grants  SM 80/7-2 ,  SFB 940 ,  TRR 265 ,  NE 1383/14-1 ), the  Medical Research Foundation and Medical Research Council  (grants  MR/R00465X/1  and  MR/S020306/1 ), the  National Institutes of Health (NIH)  funded ENIGMA (grants  5U54EB020403-05  and  1R56AG058854-01 ),  NSFC  grant  82150710554  and  European Union  funded project ' environ-MENTAL ', grant no:  101057429.
Further support was provided by grants from: -the  ANR  ( ANR-12-SAMA-0004 ,  AAPG2019 -GeBra ), the  Eranet Neuron  ( AF12-NEUR0008-01 -WM2NA ; and  ANR-18-NEUR00002-01 -ADORe ), the  Fondation de France  (  00081242 ), the  Fondation pour la Recherche Médicale  ( DPA20140629802 ), the  Mission Interministérielle de Lutte -contre-les-Drogues-et-les-Conduites-Addictives (MILDECA), the  Assistance-Publique-Hôpitaux-de-Paris  and  INSERM  (interface grant),  Paris Sud University IDEX 2012 , the  Fondation de l ′ Avenir  (grant  AP-RM-17-013 ), the  Fédération pour la Recherche sur le Cerveau ; the  National Institutes of Health, Science Foundation Ireland  ( 16/ERCD/3797 ),  U. S. A. (Axon, Testosterone and Mental Health during Adolescence ;  RO1 MH085772-01A1 ) and by  NIH Consortium  grant  U54 EB020403 , supported by a cross- NIH  alliance that funds Big Data to Knowledge Centres of Excellence.
Serum cholesterol efflux capacity assay The human THP-1 monocytic cell line (RRID:-CVCL_0006) was obtained from the American Type Culture Collection (ATCC) and was maintained in culture in 5% CO2 at 37 • C in RPMI medium containing 10% heat-inactivated fetal bovine serum, 2 mM L-glutamine and 80 U/ml penicillin/80 μg/ml streptomycin. THP-1 monocytes were grown in the presence of 50 ng/ml phorbol 12-myristate 13-acetate (PMA) for 3 days in order to induce their differentiation into macrophages. Cholesterol efflux capacity (CEC) of whole serum from human THP-1 macrophages was evaluated as previously described. 27  Human THP-1 macrophages were cholesterol loaded for 24 h with 50 μg/ml [.
LC-MS/MS analysis Lipids were quantified by LC-ESI-MS/MS using a Prominence UFLC (Shimadzu, Tokyo, Japan) and QTrap 4000 mass spectrometer (AB Sciex, Framingham, MA, USA) equipped with a turbo spray ion source (450 • C) combined with an LC20AD HPLC system, a SIL-20AC autosampler (Shimadzu, Kyoto, Japan) and the Analyst 1. 5 data acquisition system (AB Sciex, Framingham, MA, USA). Quantification of phospholipids, sphingolipids and neutral lipids was performed in positive-ion mode. Sample (4 μl) was injected into a Kinetex HILIC 2. 6 μm 2. 1 x 150 mm column (Phenomenex, CA, USA). Mobile phases consisted of water and acetonitrile containing ammonium acetate and acetic acid. Lipid species were detected using scheduled multiple reaction monitoring (sMRM). Nitrogen was used as nebulization and collision gas.
Data processing An in-house R script adapted from Ejsing CS et al. 30  was used to correct for isotopic contribution to MRM signals. A quality assurance (QA) strategy was applied to assess the performance of the analytical measurements. Lipid species displaying CV >30% across double measurements were discarded. Lipidomic data have been deposited to Metabolomics Workbench under the ID#ST002732 (DatatrackID:4069).
Metabolomic analyses LC-MS measurements were performed using pFPP, Discovery HS F5-PFPP column (5 μm, 2. 1 × 150 mm, Sigma, Saint Quentin Fallavier, France). UPLC ® Waters Acquity (Waters Corp, Saint-Quentin-en-Yvelines, France) and Q-Exactive mass spectrometer (Thermo Scientific, San Jose, CA) were used for metabolomic analysis as detailed by Garali et al. 31  LC-MS raw data were first converted into mzXML format using MSconvert tool. 32  Peak detection, correction, alignment and integration were performed using XCMS R package with CentWave algorithm  33  and workflow4metabolomics platforms. 34  Datasets obtained were log-10 normalized, filtered and cleaned based on quality control (QC) samples. 35  The remaining features were annotated based on their m/z ratio and retention time using an in-house database as described previously,  36  and also putatively annotated based solely on their m/z ratio using public databases such as the human metabolome database HMDB  37  and the Kyoto Encyclopedia of Genes and Genomes database (KEGG).
Statistical analyses All statistical analyses were performed using the R statistical software, version 4. 1. 3 (R foundation for Statistical Computing). Data are shown as median (25p-75p) for continuous variables and frequency (%) for categorical variables. Kolmogorov-Smirnov test for normality was performed. Comparisons of two groups were performed by a Mann-Whitney U test, while for three groups, Kruskal-Wallis test were performed. Relative risk analysis using modified Poisson regression for binary data was performed to determine the association between cardiac infiltration and the continuous or categorical variables. The statistical analysis of the lipidomics and metabolomics data were performed using Multi Experiment Viewer (MeV) statistical software package (version 4. 9. 0). Comparisons of the two groups were performed by a 2-tailed independent t-test and three groups by one-way ANOVA.
Features were considered significant when the p value was below 0. 05 after false discovery rate (FDR) correction. 39  When the ttest/ANOVA returned an overall level of significance between the subjects, an agglomerative Hierarchical Clustering tree (HCL); Heatmap was produced. The features were clustered using complete linkage over the samples using Pearson correlation distance matrix. Affected metabolic pathways were identified using the MetExplore version 2. 30. 10 web-based tool  40  The metabolic network (source database: KEGG Map) of human species was based on 90 metabolic pathways including 1618 metabolites, 2067 reactions, 1456 enzymatic complexes, and 1456 genes. 41  Metabolic networks are directed graphs, so it is possible to calculate compound importance based on relative betweenness centrality and out-degree centrality of any given compound from a pathway.
The prognostic scores were plotted on an AUC-ROC curve to check for the sensitivity and specificity power of the models to predict the outcome. We deployed Random forest (RF) machine learning algorithm as a sensitivity analysis to validate the developed prognostic scores. RF uses an ensemble of classification trees, each of which is grown by random feature selection from a bootstrap sample at each branch, and made a 10-fold cross validation. RF analysis was performed using the randomForest R package. All the statistical analyses were performed using SPSS software, version 28.
At 190 °C, the fluid in the gold-capsule reached a maximum pressure of approximately 16 bar but remained as a single-phase fluid, as opposed to superheated vapor. To verify if a small residual chemical disequilibrium had an effect on the observed isotope exchange, we conducted experiments at the same temperature and the same experimental time frame but with different solution:calcite ratios. Changing the amount of fluid from about 20 to 300 mg had no effect on the observed isotope exchange (Supplementary Table  1 ), indicating that isotope exchange is not influenced by the small residual disequilibrium, which can result in small amounts of dissolution during the experiment run times. Speciation was calculated using the aqueous geochemistry software PHREEQC and the Davies activity equation  73.
The analysis of time series is becoming ever more popular due to the proliferation of sensors. A well-known similarity measure for time-series is Dynamic Time Warping (DTW), which does not respect the axioms of a metric. These, however, can be reintroduced through Learning DTW-Preserving Shapelets (LDPS). This article extends LDPS and presents constrained DTW-preserving shapelets (CDPS). CDPS directs the time-series representation to captures the user's interpretation of the data by considering a limited amount of user knowledge in the from of must-link-cannot link constraints. Subsequently, unconstrained algorithms can be used to generate a clustering that respects the constraints without explicit knowledge of them. Out-of-sample data can be transformed into this space, overcoming the limitations of traditional transductive constrained-clustering algorithms.
Introduction Time series data volume and complexity have grown as sensing technology has advanced. This makes obtaining accurate labels challenging, leading to a preference for unsupervised clustering over supervised classification. Unsupervised approaches, however, can be unreliable due to algorithmic bias. In contrast, semi-supervised approaches incorporate limited expert knowledge, in the form of constraints, to produce more reliable results. Constraints can have many forms and in this work must-link (ML) and cannot-link (CL) constraints are considered. These constraints do not define a class sample, instead they indicate that pairs are similar (must-link), hence belong to the same cluster, or not (cannot-link). This allows meaningful results to be obtained without exhaustive explicit labeling of all the samples.
Shapelets Shapelets are sub-sequences of time-series and were originally developed for to discrimination using a tree based classifier  [9, 18]. Originally shapelets were sub-sequences of the time-series being studied, however searching for them is expensive and much of the literature focuses on efficiency improvements. Lines et al. [19]  and Hills et al. [20]  first proposed to separate representation learning from classification, and introduced the shapelet transform, by which the raw data is mapped into a vectorial representation, the bases of which is defined by the shapelets. In order to overcome the exhaustive search for optimal shapelets, Grabocka et al. [21]  introduce supervised shapelet learning, in which shapelets are learnt by minimising an objective function. Shah et al. [22]  learn more pertinent and representative shapelets for classification by using DTW similarity.
Zakaria et al. [23]  introduced u-shapelets, the first shapelet-based approach to clustering time-series. U-shapelets are those that best partition a subset of time series from the dataset, however, they are not learnt and instead are searched for. Zhang et al. [24]  propose combining learning shapelets and unsupervised feature selection and, amongst others Paparrizos et al. [2] , learning DTW-preserving shapelets (LDPS)  [10]  broadens the learning paradigm further. LDPS models the distribution of DTW similarity between the time series, which allows for better discrimination. To learn the shapelets, the loss function L = 1 2 (DTW(T i , T j )-β⋅ ‖T i -T j ‖) 2 defined over two time series T i and T j is minimised, where T i (T j ) is the shapelet transform of the time series T i (T j ) and β scales the embedding time-series distance.
Definition 2. (Shapelets) An ordered set of real-valued variables whose length is less than or equal to that of the dataset's smallest time- series. Let S be a shapelet having length L k. A set of K shapelets is denoted as S = {S 1 ,⋯,S K }, such that S k = S j,1:L k. Although, S typically contains different length shapelets, for simplicity, uniform shapelets are assumed.
To bound this distance, 'well separated' is taken to be the maximum DTW distance between two points in the original dataset, and so this term is calculated such that w = max ∀i,∀j∕ =i ( DTW ( T i , T j )) -log ( DTW ( T i , T j ) max ∀i,∀j∕ =i (DTW ( T i , T j ) ). ( 7 ) Consequently, the overall loss function is defined to be L (T ) = 2 N(N -1) ∑ N i=1 ∑ N-1 j=i+1 L ( T i , T j ). ( 8 ) Algorithm 1 describes the CDPS approach to learning the representational embedding. ShapeletBlocks is a dictionary containing S max (maximum number of shapelets blocks) pairs, {shapelet length; shapelet number}, where shapelet length is L min ⋅b ind , L min is the minimum shapelet length and b ind ∈ {1, ⋯, S max } is the index of the shapelet block. The number of shapelets for each block is calculated using the same approach as LDPS  [10] : 10 × log(L i -L min ⋅b ind ), where L i is the length of the time series.
Definition 6. (Entropy) Measures the randomness of a given set of class labels: homogeneous cluster labels gives zero entropy, and a uniformly random sample gives the maximum value. Assuming a dataset T with C classes and N instances, each class c i contains n i samples, and the class probability is p(c i ). With ln the natural logarithm, T 's entropy is defined, such that  E(T ) = - ∑ C i=1 p(c i )ln(p(c i )), s. t. ⎧ ⎨ ⎩ ∑ n i = N, p(c i ) = n i N. ( 9 E(T )| ζ = N l N E(T l ) + N r N E(T r ), s. t. { N l = |T l |, N r = |T r |. (10).
Algorithm 1 Constrained Distance Preserving Shapelets. The CDPS learned shapelets are ranked using IG according to their ability to reproduce (i. e. explain) a clustering, termed 'Global-Wise' explanation (GE), or a specific cluster, termed 'Cluster-Wise' explanation (CE). The term 'reproduce' refers to the ability to recreate the clustering output (GE) or a specific cluster therein (CE), when the timeseries dataset is projected into the space defined by one or more shapelets (ranked according to their IG) by linear separation. More specifically, given a set of N time-series T , a set of K learned shapelets S , and a partitioning of T into M clusters C = {C 1 , ⋯, C M } (i. e. the result of an unsupervised clustering algorithm applied to the CDPS learnt representation), the goal is to rank the shapelets in importance for giving the best CE using C i (GE using C ).
Several approaches for ranking the shapelets, irrespective of the objective (either CE or GE), can be identified depending on the desired result. Independent. In which the IG of each shapelet is examined independently. Hence, S d is defined such that S d = {S j ⃒ ⃒ IG(T ,ζ j ) > IG(T , ζ i )}, where IG(T ,ζ i ) > IG(T ,ζ i-1 ) ∀S i ,S j ,S i-1 ∈ S , S i ∕ = S j , such that ζ i = ζ(T , S i ) partitions the dataset T in the space defined by shapelet S i into D i. Similarly ζ j = ζ(T , S j ) and ζ i-1 = ζ(T , S i-1 ) partition the dataset T in the space defined by shapelets S j and S i-1 into D j and D i-1 respectively. In this way, all shapelets are ranked according to their IG. Combined. Ranking the shapelets independently does not guarantee that the combination of the top-d shapelets maximises IG.
Line 1 the shapelet that best splits the dataset into M = |C | partitions is found based on the independent approach. Its index S l , corresponding IG * and partitions D j-1 are stored. Line 6 The for loop iterates through the shapelet indices in S but not in S d , it therefore iterates from the lowest to the highest embedding dimensions. Line 7 "embvector" is the embedding derived from the current dshapelets. Line 8 get_partitions_by_shapelets() returns the best possible partitioning by applying ζ to the data-points obtained via embvector. Line 9 Stores misidentified points and those whose partition differs. Line 10 The number of correct points in δ j is returned. Line 11 The information gain defined in the successive case is calculated. In order to achieve the Cluster-Wise explanation (CE), binary labels Algorithm 2 GS-SCE: Global Successive Shapelet Clustering Explanation.
Experimental setup Unsupervised (K-Means, and DCC without constraints) and semisupervised clustering algorithms (COP-KMeans  [13] , MIP-Kmeans  [25] , and DCC  [26] ) are used as baselines. Since CDPS takes constraints during training, none are given to the algorithms that uses CDPS' representation. The study is carried on the UCR repository  [34, 35]  and normalised Mutual Information (NMI) is used to measure clustering performance, where 0 signifies no mutual information and 1 a perfect correlation with the ground truth labels. Due to the computational complexity of COP-Kmeans and MIP-Kmeans, thirty-five univariate and fifteen multivariate datasets were chosen at random. Since CDPS and DCC are faster, their performance on all the datasets was evaluated and is presented in Appendix E. Details of the datasets are provided in Appendix G The number of clusters for each algorithm is set to the number of classes, and the datasets are standardised with respect to time.
Two use-cases are evaluated. In the first, termed Transductive Clustering, the training and test sets (as given by the repository) are combined, this reflects the real-world case in which a dataset is to be explored and knowledge extracted. In the second, termed Inductive Clustering, the embedding is learnt on the training set and its generalisation is evaluated on the test set. This inductive use-case is not normally possible with constrained clustering algorithms since clustering is a transductive operation. This highlights a key contribution of CDPS, its ability to generalise constraints to unseen data. The algorithms are evaluated with varying constraint levels, represented as a percentage of constrained samples: 5%, 15%, 25%. These represent a very small fraction of the total possible constraints, 1  2 N(N -1).
Transductive clustering Fig. 2  summarises the Transdictive results (see Table  E. 2 for details). It shows the NMI scores for CDPS (Euclidean K-means performed on the CDPS embeddings) compared to the other algorithms applied to the raw time-series (COP-Kmeans, MIP-Kmeans, and DCC), LDPS (CDPS with no constraints reduces to LDPS) and (unconstrained) K-means are presented as a basis for constrained algorithms to study how constraints benefit each. As expected LDPS and K-means offer similar performance, see Fig. 2a , although some datasets do favour the unconstrained K-means algorithms, outperforming LDPS. Nevertheless, CDPS efficiently uses the information given by constraints to outperform the other algorithms in almost all the different constraint fractions and datasets. Its performance increases with the number of constraints, whereas COP-KMeans and MIP-Kmeans tends to stagnate.
This can be seen as the cloud of points move upwards (CDPS' NMI score increases) as more constraints are given. DCC tends to work better for the datasets that have low NMI scores for COP-Kmeans and MIP-Kmeans. Indicating complex or similar structures with which DTW struggles to capture the detailed differences. In contrast, DCC is able to find more discriminative features beyond the structural similarity of the time series, resulting in improved performance. We can also observe that for some datasets the constrained algorithms behave similarly with 5% constraints, where COP-Kmeans and MIP-Kmeans have almost similar performance while DCC is a bit worse. Again CDPS benefits the most from increasing the number of constraints and significantly outperforms the comparison algorithms with larger constraint percentages.
Overall COP(MIP)-Kmeans perform similarly, however MIP-Kmeans was able to converge on datasets for which COP-Kmeans was not. Furthermore, CDPS exploits constraints better than other algorithms, and DCC seems to exploit them the least. For some datasets all algorithms perform equally (see top right and bottom left corners) which can be due to the fact that the datasets are either very difficult (hence low NMI) or very easy to cluster.
Inductive clustering Fig. 3  summarises the Inductive results (see Table  E. 3 for details), i. e. embedding learnt on the training set, generalised to test set. Note that for the same constraint percentage, there are significantly fewer constraints than in the transductive setting. It can therefore be concluded that even with fewer data and constraints, CDPS is still able to learn a generalisable representation and attain (within a certain margin) the same clustering performance than with the merged datasets. This is probably explained by the fact that having fewer samples and constraints means they are repeated in the mini-batches (see Section 3. 1). Allowing CDPS to focus on learning shapelets that are highly discriminative and preserve DTW rather than shapelets that model larger numbers of time series.
Thus, the resulting representational space better adheres to the constraints, allowing better clustering of unseen timeseries. It can also be observed that COP-Kmeans tends to struggles with a small number of constraints, since it increases the risk of constraint violations, while MIP-Kmeans is able to overcome this. In this setting, DCC outperforms both COP-KMeans and MIP-KMeans for some of the challenging datasets (low NMIlower left corner), and shows similar performance to the other algorithms on less complex data (top right corner).
Shapelet cluster explanation A case study of GS-SCE is presented herein (Appendix D presents the results for GI-SCE and GC-SCE). The Synthetic Control dataset is used, which contains 600 samples, representing synthetically generated control charts of length 60 and six categories: 1-Normal, 2-Cyclic, 3-Increasing trend, 4-Decreasing trend, 5-Upward shift, and 6-Downward shift. Fig. 4  shows two instances from each category. The Transductive approach is used, and the aim is to provide insight and explanation of the clustering result. The GS-SCE algorithm, described in Algorithm 2, takes clustering labels and transformed time series as input and returns a shapelet ranking based on the information added by each. Fig. 5  presents the normalised cumulative information gain (NCIG) (scaled between 0 and 1).
CDPS does not suffer from such limitations. Although it was included in this study for comparison, using constrained clustering algorithms in an inductive usecase is not usual practice. It was simulated by providing the constrained algorithms with the combined 'training' dataset, its constraints, and the test data to be clustered. The necessity to store and access this data can prevent such use-cases and also leads to very high computational cost. CDPS, on the other hand, offers a truly inductive approach in which new data can be projected into the resulting space, which inherently models user constraints. This demonstrates the new possibilities of 'constrained representations'. Moreover, CDPS' embedding space can be used for other tasks such as, classification, generation, etc. Overall, the CDPS algorithm leads to better clustering results since it is able to exploit the information brought to the learning process by the constraints.
In absolute terms, CDPS's performance tends to incease as cosntraints are introduced, while the performance of other algorithms either stagnate or decrease. These constraints bias CDPS to find shapelets that define a representation that respects both DTW and the constraints. Although the focus of this article is not to evaluate whether clustering on these datasets benefits from constraints, it can be observed that generally better performance is achieved when constraints are added. It is also conceivable to use a constrained clustering algorithm with CDPS' embedding. Although this was not studied, it would allow another mechanism to integrate constraints after the embedding has been learnt.