dataesr/scientific-paragraphs-categorization huggingface dataset:
e. N elements). To cope for this, it has recently been proposed in the compressive learning framework  [54]  to summarize the entire dataset by a relatively small vector of fixed size, referred to as data sketch, and to have the algorithm operating on his sketch only, irrespective of the original data. Concretely, we built the data sketch as an average of random Fourier features of the chromatographic profiles in the feature space (see Random Fourier feature sketching section). 3. Cluster and centroid definitions: Lloyd algorithm  [55]  (i. e. the most classical algorithm to cluster data according to the k-means objective function) cannot directly be applied on sketched data. Fortunately, it is possible to rely on the Compressive k-means (CKM) algorithm proposed in  [56]  (see Cluster computations section).
However, CKM only returns a set of cluster centroids and does not cluster the data per se. Therefore, traversing the entire (original) dataset to perform the assignment of each chromatogram to its closest centroid (according to the W1 distance) is necessary (see Cluster assignment section). CKM complexity does not depend on the original data size (as it operates on the data sketch) which makes it well-scalable. However, its complexity grows rapidly with the number of clusters, which is an issue as thousands of clusters can be sought in LC-MS data. To cope for this, we implemented a hierarchical clustering scheme, where each cluster is recursively divided into a small number of sub-clusters until the desired number of clusters is obtained (see Cluster assignment section). This procedure provides a set of clusters with centroids only defined in the feature space.
Performance metrics For experiments annotated with a ground truth (like UPS2GT dataset), clustering accuracy can be evaluated with the Rand index (RI). The Rand index measures the percentage of correctly clustered pairs of signals over the total number of pairs. Let us denote as U = {U 1 ,. , U k } the obtained clusters and as V = {V 1 ,. , V q } the ground truth clusters. A pair of signals is considered as correctly clustered: true positive (TP) or true negative (TN), if signals are assigned to the same cluster in U and V or on the contrary, to different clusters in U and V. A pair of signals is called false positive (FP) (resp. false negative (FN)), if signals are grouped in U (resp. V ) but not in V (resp. U ). Then, the Rand index is given by: RI = T P + T N T P + T N + F P + F N (16) The maximum value of the Rand index is 1 (perfect match with the ground truth).
Additionally, it is possible to evaluate how often different chromatograms are grouped in the same cluster; and how often similar chromatograms were assigned to different clusters. To do so, one classically relies on the Precision and Recall metrics, respectively: Precision = T P T P + F P Recall = T P T P + F N (17) For datasets without ground truth annotation (like both Ecoli datasets), it is possible to rely on the Davies -Bouldin (DB) index. Let us denote as J (c j ) the j th cluster with the cluster centroid c j , and as {J (c 1 ),. , J (c k )} the set of obtained clusters. The within cluster distance reads: S j = 1 |J (c j )| xi∈J (cj ) d W1 (x i , c j ) (18) The DB index is defined through the ratio of the within cluster distances to the between cluster distance d W1 (c i , c j ): DB = 1 k k i=1 max i =j S i + S j d W1 (c i , c j ) , (19) It should be noted that the distance metric in the DB index and in the clustering algorithm must be the same, in our case the W1 distance in the original space.
In contrast, highlighting the differences of our approach with respect to linkage-based agglomerative clustering and showing that despite noticeable differences, one obtains clusters which are meaningful, is of real practical interest to computational mass spectrometry experts. As reported in the Background section, comparisons with Xnet is mandatory. However, considering the reported specificities (trace extraction preprocessing, envelope assumption simplification, etc. ), comparing Xnet and CHICKN workflows may appear as somewhat arbitrary. To cope for this, we have made the following choices: First, we have focused on the core of each algorithm, as represented in Figure  2A. Second, we have adapted the UPS2GT and Ecoli datasets to be processed by each algorithm: The UPS2GT data are already formatted into a CSV file meeting Xnet requirements.
To construct a data matrix suitable to CHICKN from the UPS2GT data, we simply loaded the data points according to their retention time and trace labels in the matrix columns (similarly to Xnet, we excluded point with trace indices -1 and 0, as assumed to be noise). This led to a data matrix containing 57,140 columns and 6,616 rows. Conversely, to build the CSV files from Ecoli datasets, we stored any non-zero entry of the data matrix in a row, the column index being used in place of the trace labels. ∆RT captures the time translation effect. However, none of these classically used distances is able to capture both the shape and the translation simultaneously. On the contrary, W1 distance is efficient on both situations, making it a suitable distance to construct a similarity measure adapted to LC-MS data.
Parameter tuning Unlike Xnet, CHICKN is governed by eight parameters. Four of them are involved in the data compression: Nyström sample size (l), target rank (s), kernel parameter (γ) and sketch size (m). Three parameters are involved in the hierarchical clustering: number of clusters at each iteration of the hierarchical clustering (k), upper bound of the total number of expected clusters (k total ) and maximum number of levels in the hierarchy (T ). The remaining parameter is the neighbourhood size in the consensus chromatogram computation (q). However, all parameters except γ and q are interrelated (see the Data compression section as well as  [62, 53] ) and can be expressed through k, k total and N (the dataset size) as follows: l ≈ √ N , s ≈ √ k • N 1/4 , m ≈ k 3/2 • N 1/4 , T = log(k total , k).
( 20 ) These theoretical results can nonetheless be discussed. Notably, tuning the sketch size m to a larger value may be of interest if contrarily to our case, the computational efficiency is not the only targeted goal. Thus, we have performed Finally, four parameters remain (γ, q, k and k total ). Concretely, we tuned the kernel parameter γ as an average of the power of p distances to the ν nearest neighbors for all chromatographic profiles: γ = 1 N • ν N i=1 ν j=1 [d W1 (x i , x ij )] p , (21) where x i1 ,. , x iν are ν neighbors of x i (selected among the l points of the Nyström sample) and p ∈ {1, 2} depending on the kernel type. Practically, we observed that tuning ν to 32 guaranteed each data point to be sufficiently connected to the rest of the dataset, as advised in  [37]. Moreover, we observed that γ was rather stable with respect to ν, for both Laplacian W1 and Gaussian W1 kernels.
With this respect, we have evaluated different scenarios with k = 2, 4, 8 and 16. CHICKN execution times (excluding the data compression step, which remains constant whatever the various scenario) on the smallest (UPS2GT) and largest (Ecoli-FMS) datasets are depicted in Additional File 7. This experiment pointed out the importance of tuning k to a small enough value, which is coherent with the observation that the original CKM algorithm does not scale up well with the number of clusters. Practically, working with k = 2 or 4 appeared to be the most efficient. In the case of UPS2GT, the expected number of isotopic envelopes is known (i. e. 14,076). Thus, it is easy to tune k total accordingly (i. e. 2 14 = 4 7 = 16, 384). However, knowing that CHICKN does not rely on the envelope assumption simplification, it can be expected to find a much lower number of clusters: broadly, all the isotopic envelopes corresponding to different charge states of a same peptide can be expected to cluster together.
Therefore, it also makes sense to tune k total to 4 5 = 1, 024; i. e. close enough from the expected number of identifiable peptides in the sample (around 700, according to  [23] ). Tuning k total for any real life data (i. e. unlabeled) is much more complicated. However, the Escherichia Coli sample is well studied, and based on prior biological/analytical knowledge, 15,000 different peptides can be expected, broadly. Consequently, for both Ecoli datasets, k total = 16, 384 seems reasonable. Finally, even though it is not as sensible from a biological viewpoint, we have decided to also consider k total = 4 6 = 4, 096, which provides an even ground for computational load comparisons (see next section for details). To summarize, three different ways to tune k total are insightful: 1,024 for the UPS2GT dataset only (as it matches the number of expected peptides); 4,096 on all datasets (for computational benchmarcks); and 16,384 on all datasets (number of isotopic envelopes in UPS2GT and number of expected peptides in Ecoli datasets).
Computational load We have compared the execution times of CHICKN and Xnet cores (see Fig-  ure 2A ). Previously reported comparisons showed us that CHICKN execution time largely depends on k. However, it only has a sub-linear complexity with respect to k total. : As illustrated in Additional File 8, multiplying k total by 4 only results in a threefold (resp. twofold) increase in the CHICKN run-time for the Ecoli-FMS (resp. UPS2GT) dataset. As reducing k total to limit the execution time will therefore be of little interest, experiments hereafter reported only focused on the influence of k. Despite CHICKN being more efficient when run with k = 2 and 4 (see Parameter tuning section), we also included comparisons with k = 8 and 16 to investigate the consequences of sub-optimal parameter tuning.
The corresponding tests are referred to as CHICKN2, CHICKN4, CHICKN8 and CHICKN16. Therefore, to rely on an even basis for comparisons, we focused on k total = 4, 096: it is a power of 16, contrarily to 1,024 and 16,384 (which are even not a power of 8). Since CHICKN algorithm embeds a compressive k-means algorithm which may converge towards different local minima depending on the stochasticity of several steps, each scenario was repeated 10 times and the average execution time was reported. In contrast, Xnet being deterministic, it was executed once. In  [17] , Xnet exhibits impressive computational times on pre-processed and adequately formatted data. However, raw LC-MS data stored in a matrix format are more cumbersome. Thus, our first experiment was to compare the efficiency of Xnet and of CHICKN on the Ecoli-DIA dataset, using a laptop machine with the following characteristics: HP Pavilion g6 Notebook PC with Intel(R) Core(TM) i5-3230M CPU @ 2.
60GHz, 8 Gb of RAM, 4 cores, running under Ubuntu 18. 04. 4 LTS OS. Xnet produced an "out-of-memory" error when trying to cluster more than 10,000 columns (i. e. 5% of the Ecoli-DIA dataset) in a single batch. This is why Figure  2B  compares the computational time of CHICKN2, CHICKN4, CHICKN8 and of CHICKN16 on the entire Ecoli-DIA dataset to that of Xnet on only 5% of the same dataset. On this figure, different colors are used to discriminate between the clustering step per se and CHICKN preliminary data compression step. Let us note that the compression step is time consuming, however, it also includes the computations of all the W1 similarities. This as-a-matter-of-factly illustrates the computational cost of relying on more elaborated metrics to capture the semantics of data as complex as LC-MS ones.
Except for CHICKN16, which has already been pointed as suboptimal, CHICKN is always faster for a dataset 20 times larger. This first experiment clearly showed CHICKN could be used on a simple laptop, even with large datasets, in long but acceptable times (half an hour to two hours, broadly). Then, to reduce the execution times of our multiple experiments, but also to allow Xnet working on a larger dataset, we moved to a larger station using 10 cores of an Intel Xeon CPU E5-2470 v2 @ 2. 40GHz, 94 GB of RAM and running with CentOS Linux release 7. clearly means that CHICKN could not separate too many peptides with too similar elution profiles. However, this can be easily explained by the difference of complexity between the UPS2GT and the Ecoli samples: while the former is fairly simple (a handful of spiked proteins), the latter ones are complex real life samples for which the discriminative power of the liquid chromatography is clearly challenged (as illustrated in the next section).
Implementation and code availability CHICKN algorithm was implemented in R. The W1 distance computations and the gradient descent were accelerated using C and interfaced with R thanks to Rcpp. The data compression procedure and the hierarchical strategy were parallelized with RcppParallel, foreach and doParallel. To access and manipulate large data matrices, we relied on the File-backed Big Matrix class of the bigstatsr package  [67]. A File-backed matrix allows to overcome the memory limitation by storing the data on the disk, using a binary memory-mapped file. However, bigstatsr is only available under Linux OS, leading to a similar restriction for CHICKN. For practitioners, the proposed algorithm is available through an R package, available on Gitlab  [68] , as well as on the CRAN  [69].
Conclusion We have presented two complementary contributions to the cluster analysis of LC-MS data. We have demonstrated these contributions could help extracting other structures than isotopic envelopes, even on multiplexed data acquired with Data Independent Acquisition protocol. However, the experimental assessment of these contributions is difficult to interpret. On the one hand, when compared to the canonical application of isotopic envelope extraction, CHICKN does not outperform the state-of-the-art algorithm (better Recall and worse Precision, as it tends to under-cluster rather than over-cluster). However, it provides an important advantage: it can be run on raw data and does not require costly preprocessing. As for an application-independent evaluation, it clearly appears that CHICKN is able to extract patterns from the data which are not accessible to linkage-based algorithms.
Put together, we interpret this as following: Although cluster analysis has made important progresses in the theoretical front over the past 50 years, processing LC-MS data remains a challenge which requires research efforts. It is still necessary to propose complementary and differently principled algorithms that will help make LC-MS practitioners extract the best from their data. In this context, new kernels could be defined; and numerous state-of-the-art clustering algorithms recently developed in the machine learning community could advantageously be applied to LC-MS data.
Not applicable The Ecoli DIA and Ecoli FMS raw data (generated and analyzed for the current study) are not publicly available due to their too large size. However they are available from the corresponding author upon reasonable request. To reproduce all experiments described in the article, the preposessed Ecoli datasets in the file-backed matrix format can also be provided upon request. • ∆RT, peak Retention Time difference; • DIA, Data Independent Acquisition; • Ecoli-DIA, Dataset resulting from the analysis of an Escherichia coli sample in DIA mode; • Ecoli-FMS, Dataset resulting from the analysis of an Escherichia coli sample in FMS mode; • FMS, Full mass spectrum (only MS1s are recorded); • FN, False Negative; • FP, False Positive; • LC-MS, liquid chromatography and mass spectrometry; • MS1, peptide (or precursor) mass spectrum; • MS2, peptide fragment mass spectrum; • m/z, mass to charge ration; • PD, positive definite; • PSD, positive semi-definite; • RI, Rand index; • RT, Retention Rime; • SVD, Singular Value Decomposition; • TN, True Negative; • TP, True Positive; • UPS2GT, dataset resulting from the analysis of the Proteomics Dynamic Range Standard (UPS2) and manually annotated (ground truth); • W1, Wasserstein-1 distance;.
Ecoli datasets: wet-lab analysis Escherichia Coli bacteria were lysed with BugBuster reagent (Novagen, final protein concentration 1µg/µL). Around 560 µg of proteins were stacked in the top of a 4 -12% NuPAGE ZOOM gel (Life Technologies) and stained with R-250 Coomassie blue. Gel was manually cut in pieces before being washed by six alternative and successive incubations in 25 mM NH 4 HCO 3 for 15 min, followed by 25 mM NH 4 HCO 3 containing 50% (v/v) acetonitrile. Gel pieces were then dehydrated with 100% acetonitrile and incubated with 10 mM DTT in 25 mM NH 4 HCO 3 for 45 min at 56 • C and with 55 mM iodoacetamide in 25 mM NH 4 HCO 3 for 35 min in the dark. Alkylation was stopped by the addition of 10 mM DTT in 25 mM NH 4 HCO 3 (incubation for 10 min). Gel pieces were then washed again by incubation in 25 mM NH 4 HCO 3 followed by dehydration with 100% acetonitrile.
W1 distance was proposed to discriminate between signals that represent different elution profiles. To assess this choice, we compared it with two distances amongst the most widely used in mass spectrometry signal processing: The first one is the classical Euclidean distance. The second one is the peak retention time difference (or ∆RT): It corresponds to the difference between the time stamps at which each signal reaches its highest intensity value. Based on the Ecoli-FMS dataset (which provides the finest temporal sampling), we examined two situations presented in Figure3: In the first one, we selected 3 signals with different shapes, that we precisely aligned so that their pairwise ∆RT was zero; in other words, only the shape difference makes it possible to discriminate them. Conversely, in the second situation, an elution profile was translated to mimic a case where only the ∆RT was meaningful.
bilities of CHICKN, which captures similarities between slightly different but largely overlapping signals (third line); and (ii) the possibility to run CHICKN on raw data, which also contains many spurious signals that need be spread across various meaningful clusters. Similar conclusions regarding CHICKN behavior can be derived from the Ecoli datasets (let us focus on the Ecoli-FMS one, as it displays elution profile signals with higher sampling resolution, due to the Full-MS acquisition). The majority of clusters (Figure6for the Gaussian W1 kernel and AdditionalFile 12,      for the Laplacian W1 one) containing high intensity signals depicts meaningful consensus chromatograms, as well as similar profiles even though corresponding to different m/z values. However, we observed that some clusters could be separated into several sub-clusters to improve readability (see Additional File 13).
Funding This work was supported by grants from the  French National Research Agency :  ProFI  project ( ANR-10-INBS-08 ),  GRAL  project ( ANR-10-LABX-49-01 ), DATA@ UGA  and  SYMER  projects ( ANR-15-IDEX-02 ) and  MIAI @ Grenoble Alpes  ( ANR-19-P3IA-0003 ). Grants  ANR-10-INBS-08  and  ANR-10-LABX-49-01  contributed to wet-lab equipment, including mass spectrometer; Grant  ANR-15-IDEX-02  contributed to human resources; Grant  ANR-19-P3IA-0003  supported other expanses.
Getting land-sea mask information We need to read the global land-sea information for 2 purposes: get the land fraction (step 1) and find the number of land only subdomains in a given domain decomposition (step 3). By default, reading NetCDF configuration files in NEMO can be trivially done by the mean of a dedicated Fortran module ("iom"). However, in this specific context, things must be done more carefully to avoid a one-off but potentially extremely large memory allocation associated with a massive disk access. Until revision 3. 6 included, each MPI process was reading the whole global land-sea mask, which is clearly not the proper strategy when aiming at running very large domains on large numbers of MPI processes with limited memory. The difficulty here lies in the minimization of the memory used to get the needed information from the global land-sea mask.
Next, we define the element N + 1 as the couple (jpni, jpnj) that gives the smallest number of subdomains (jpni × jpnj). It happens rarely that several couples (jpni, jpnj) correspond to this definition. In this case, we decided to keep the couple with the smaller subdomain perimeter, so the volume of data exchanged during the communications is minimized. This choice is quite arbitrary as NEMO scalability is limited by the number of communications but not by their volume. Limiting the subdomain perimeter should therefore have a very limited effect. We stop the iteration process once there is no more couple with a subdomain size smaller than the one selected at rank N. Once this list of the best domain decomposition sorted from 1 to N sub max subdomains is established, we just have to follow the third step of our algorithm to get the best subdomain decomposition (see section 2.
The BENCH configuration Once we found the best domain decomposition, exploring the code numerical performance requires a proper benchmark that is easy to install and configure but, at the same time, keeps the code usage close to the production mode. The NEMO framework proposes various configurations based on a core ocean model: e. g. global (ORCA family) or regional grids, with different vertical and horizontal spatial resolution. Different components can moreover be added to the ocean dynamical core: sea-ice (i. e. SI3) and/or bio-geo-chemistry (i. e. TOP-PISCES). A comprehensive performance analysis must thus be able to scrutinise the subroutines of every module sets to deliver a relevant message about the computing performance of the NEMO model to its whole users community. On the other hand, as pointed out by the former RAPS consortium  (Mozdzynski, 2012) , performing such benchmarking exercise must be kept simple, since it is often done by people with basic knowledge of NEMO or physical oceanography (e.
BENCH flexibility Any NEMO numerical schemes and parameterizations can be used band be used in BENCH. To help the user in his namelist choices and to test diverse applications, a selection of namelist parameters is provided with BENCH to confer to the benchmarks the numerical properties corresponding to popular global configurations: ORCA1, ORCA025 and ORCA12. The SI3 sea-ice and TOP-PISCES modules can be activated or not by choosing the appropriate namelist parameters and CPP keys when compiling. The initial temperature/salinity definition was designed in such a was that if SI3 is activated in the simulation, the sea ice will cover about 1/5 of the domain, which corresponds approximately of the annual ratio of ORCA ocean grid points covered by sea-ice. Any closed or periodical conditions can be used in BENCH and specified through a namelist parameter (nn_perio in namusr_def ).
BENCH grid size, MPI domain decomposition and land only subdomains Like in any other NEMO configuration, BENCH computations can be distributed on a set of MPI processes. BENCH grid size is defined in the namelist with 2 different options. If nn_isize and nn_jsize are positive, they simply define the total grid size on which the domain decomposition will be applied. This is the usual case, the size of each MPI subdomains is comparable (but not necessary equal) for each MPI task and computed by the code according to chosen pattern of domain decomposition. If nn_isize and nn_jsize are negative the absolute value of these parameters will no more define the global domain size but the MPI subdomains size. In this case, the size of the global domain is computed by the code to fit the prescribed subdomains size.