dataesr/scientific-paragraphs-categorization huggingface dataset:
Investigating curation strategies The curation of these profiles exemplified a particular set of methodological choices adapted to the specific experimental dataset at hand. BioProfiling. jl does not enforce any single approach and other options could be considered at all steps, from feature selection to dimensionality reduction and quantification of profile distances. As a comparison, we also used one of the other implemented distance metrics, namely the Mahalanobis distance between the center of a perturbation's profiles and the refence profiles. When no dimensionality reduction was applied, we observed that all profiles were significantly distant from the DMSO. This reflects one consequence of the curse of dimensionality, namely that pairwise distances tend to be similar in high-dimensional spaces (Supplementary Fig.
Note that BioProfiling. jl is not only versatile in respect to the profiling approaches, but also supports multiple experimental setups and data types, as any tabular data compatible with the common DataFrame structure can be used as input to define an Experiment object. We provide one additional example from a publicly available dataset using the Cell Painting assay  (Bray et al. , 2016)  to characterize overexpression constructs. By curating and representing profiles for two plates, we observe a visible clustering for some target genes (Supplementary Fig. S4a  and b ). Overall, comparing the RHD of each target to non-targeting controls correlates well across plates despite processing both plates independently (Supplementary Fig. 4c ), which supports the robustness of the chosen approach.
Discussion HCS experiments offer a scalable and cost-efficient way to assess multiple conditions in a single experiment with a rich cellular readout. Assembling morphological profiles to describe these experimental conditions is thus essential and requires dedicated tools for data curation, feature selection, quality control, visualization and quantification of morphologically active perturbations. We implemented these tools in a single open-source software with intuitive and flexible data structures and syntax. We demonstrated by a concrete use case how BioProfiling. jl enables new research and allows the exploration of changes in cellular morphology by easing the analysis of large high-content imaging screens. As Julia is an efficient programming language and allows parallelization of the computations, BioProfiling.
jl can process large datasets in a performant manner. The biggest limitation for analyzing large experiments at the single cell level is currently the memory usage, as the full set of morphological measurements needs to be loaded, which can be an issue on personal computers. This may be improved in the future by using lazy loading and allowing the user to process the data by batches. In regard to profile interpretability, BioProfiling. jl can help identify which features are varying the most, rank features by absolute fold-change when comparing two conditions, highlight correlated measurements and format the data for other tools, for instance to represent the typical cell morphology in a particular condition  (Khawatmi et al. , 2021; Sailem et al. , 2015). Of note, BioProfiling. jl offers a systematic way to define filters for data curation and feature selection.
This simplifies the automated definition of these steps and could contribute toward the future development of data-driven feature engineering and machine-learningpowered artifact removal techniques to further streamline the process of morphological profiling. BioProfiling. jl expands the existing landscape of resources available for biological data analysis, as illustrated in our application where we processed morphological measurements so that they can be integrated with PPIs as well as chemical annotations. The library contributes to the growing package ecosystem for bioinformatics in Julia  (Roesch et al. , 2021)  which ensures that the morphological profiling analyses can be combined in larger projects together with other tasks ranging from sequencing to systems biology  (Greener et al.
, 2020; Heirendt et al. , 2017; Zea et al. , 2016) , and other libraries are conveniently available to integrate these different data types  (Zakeri et al. , 2018). Julia's interoperability with other programming languages also makes the onboarding easy for users with prior programming experience who, for instance, might prefer to perform certain tasks in R or Python. This is demonstrated in the provided Jupyter notebooks, with all plots being generated using R's ggplot2 library  (Wickham, 2016)  and with the computation of the MCD estimators for robust statistical distances which relies on R's robustbase package, which in turn calls efficient Fortran routines. Despite being initially designed and extensively tested for morphological profiling, the ability of BioProfiling. jl to handle large high-dimensional datasets and provide dedicated robust normalization and comparison methods could also be leveraged for other data analyses such as single cell transcriptomics or metabolomics experiments, which also require the curation and transformation of data in tabular format.
The red ellipses show the forecasted constraints on these quantities for PRISTINE and PIXIE (is hardly visible for PIXIE); more powerful missions in the Voyage 2050 program would provide even tighter constraints. state-of-the-art cosmological hydrodynamics simulations  [278, 279] , including precise predictions from different feedback implementations. The figure also shows forecasted constraints for PRIS-TINE and PIXIE, as illustrative spectral distortion missions. It is clear that such measurements will strongly distinguish between current sub-grid feedback models, yielding significant breakthroughs in our understanding of galaxy formation. A direct measurement of the average rSZ temperature would also shed new light on the 'missing baryon problem'  [273]  without the need to resolve the warm-hot-intergalactic medium, a unique opportunity that we should make use of in the future.
Technological challenges The seminal measurements of the CMB blackbody spectrum by COBE/FIRAS in the early '90s cemented the Hot Big Bang model by ruling out any energy release greater than ∆ρ γ /ρ γ 6 × 10 -5 (95% c. l. ) of the energy in CMB photons  [2, 3, 7]. Advances since then, in both detector technology and cryogenics, could improve this constraint by four orders of magnitude or more (e. g. , with experimental concepts like PIXIE  [40, 41] , PRISM  [32] , PRISTINE  [319]  or Super-PIXIE  [57] ), opening an enormous discovery space for both predicted distortion signals and those caused by new physics. On the timescales relevant to the Voyage 2050 program we expect to go beyond, surpassing the crucial threshold of detecting the dissipation µ-distortion at more than 3σ. COBE/FIRAS was not background limited; its sensitivity was set instead by phonon noise from the 1.
M-class mission within Voyage 2050 Program The second option would be to fly, preferably by 2030-2035 and thus in collaboration with international partners, an intermediate (e. g. , M-class) mission with a spectrometer design closely following Super-PIXIE  [57]. Depending on budgetary constraints, the number of FTS replicas can be adjusted, although the main cost driver stems from the cryogenic cooling chain. We expect small changes to the design to be compensated by further optimization of the instrument, so that a mission like this should be able to detect the dissipation µ-distortion at 3σ. Although the µ-detection significance could still suffer from foreground complexities, accurate measurements of the low-redshift y-distortion and SZ temperature correction are guaranteed. A vast discovery space would furthermore be explored, constraining many new physics examples with significantly improved bounds on µ (see Fig.
Instead, deconvolution methods can be used to infer cell-type composition in silico from bulk measurements, which enable the analysis of a large number of publicly available omic datasets. Bioinformatics tools that assess the different cell populations from bulk transcriptome  [2] [3] [4] [5]  and methylome  [6] [7] [8] [9]  samples have been recently developed, including reference-based and reference-free methods. Recent efforts have been made to objectively compare existing tools in order to guide the users. In particular, two recent benchmark studies proposed a comprehensive comparison of transcriptome-based deconvolution methods using various parameters and simulation settings  [10, 11]. In the same vein, the DREAM challenge proposed in 2019  [12]  a data challenge dedicated to the prediction of immune cell types, showing the emerging spirit towards reproducibility and benchmarking.
Recent collective initiatives provided formal guidelines and unified frameworks to improve unbiased performance evaluation  [16]. For instance, the Global Alliance for Genomic and Health (GA4GH) published an open access benchmarking tool to assess germline small variant calls in human genomes  [17]. More recently, BEELINE, a uniform interface to evaluate Gene Regulatory Network inference from single-cell data, was published and made freely accessible in the form of a docker image  [18]. In this project, we built on a previous HADACA (Health Data Challenge consortium) benchmarking study  [7]  to develop a standardized benchmark framework for accurately evaluating quantification of tumor intra-heterogeneity from a multi-omic dataset. First, we built in silico 10 paired methylome and transcriptome benchmark datasets, using pancreatic cancer (PDAC, pancreatic adenocarcinoma) as a case study.
These benchmark datasets were made realistic by the integration of the latest knowledge on PDAC biology  [19] [20] [21]  in the simulation models and can be used as 'truth' to evaluate computational methods quantifying tumor heterogeneity. Second, we defined Mean Absolute Error (MAE) on estimated cell-type proportions and computational time as standard performance metrics. Third, we embedded the benchmark dataset and the scoring algorithm into a web platform called DECONbench. This web platform enables continuous and crowdsourced benchmarking, by asking participants to submit source code of their algorithm. Each submission is therefore run by the platform on the benchmark dataset and results generated in a reproducible way. Fourth, we implemented on the platform baseline methods based on some previously published deconvolution algorithms and tools.
Therefore, DECONbench is an open resource to evaluate novel computational methods in an unbiased way. It provides a private general report on the overall performances of the method submitted by any participant and offers the possibility to share all source code of the contributing methods, as well as performance evaluation on a public leaderboard. Here we present DECONbench, an innovative public benchmarking platform, open source and freely available, aiming at comparing integrative deconvolution methods for tumor heterogeneity quantification. This framework supports both crowdsourcing benchmarking (collaborative and competitive assessment of the methods) and continuous benchmarking (possibility to continuously integrate novel methods), two features that should contribute to the widespread community adoption of benchmarking good practices  [15, 22].
Usage DECONbench is optimized to execute methods developed in R statistical programming language, using a docker image provided on our website. The benchmark is structured around an ingestion program used as a wrapper object to execute an R program. Should anyone wish to benchmark a method coded in another language, R could then be used as a script language to execute the given program by invoking a System Command. A list of R packages installed on the docker image is as well provided. Users need: (i) to register to DECONbench on the participate tab and to download the starting kit and the public datasets; (ii) to develop an algorithm according to DECONbench guidelines; (iii) to submit their code (as a zip file) in the participate tab. Submitted algorithms are evaluated on DECONbench datasets and benchmarked with the other baseline methods.
Provided benchmark datasets We have generated paired transcriptome and methylome benchmarking datasets from primary cells from pancreatic tumors and sorted cells from public datasets (Fig. 2 ). Gold standard heterogeneous samples were simulated using mixtures of individual cell populations (fibroblast, immune cells, normal epithelial cells and cancer cells, see "Methods" section). Exact sample compositions are not accessible to the users. Participants are facing a deconvolution problem to solve the following model: D = TA , with D the complex matrix of molecular profiles measured on het- erogeneous samples; T , a reference matrix of cell-type specific molecular profiles; and A , a proportion matrix of cell-type abundance in each sample. The aim of the competition is to find the best estimate of the proportion matrix A.
Selection of baseline methods from a data challenge We used these unreleased benchmark datasets in a data challenge aiming at inferring cell-type proportions from a cancer dataset including both transcriptome and methylome profiles (https:// tinyu rl. com/ hadac a2019). Baseline methods provided on DECONbench were collectively designed, tested and implemented during the challenge. They are composed of two steps: first, we operate a feature selection process to reduce the dimensions of the dataset, second, we apply a deconvolution algorithm. These algorithms consist of various statistical tools already published, based on unsupervised source separation approaches: ICA-based (Independent Component Analysis)  [23] [24] [25]  or NMF-based (Non-negative Matrix Factorization)  [8, 9, 26]. Each baseline method was designed to be applied either on single-omic (see Table  1 , Data type "RNA" or "DNAm") or in an integrated fashion on both the transcriptome and the methylome dataset (see Table  1 , Data type "both" and Multiomic integration strategy).
Performance of the baseline single-omic methods We run all the baseline methods on 10 different simulated datasets and computed the corresponding MAEs (Fig. 3 ). The best algorithms based on single-omic datasets were the r_WNM method for RNA-based data (mean MAE of 0. 024) and the m_MDC method for DNAm-based data (mean MAE of 0. 038). Both are NMF-based algorithms, details on the methods can be found in the "Methods" section. DECONbench provides also the computing time for each method, as an indicator of algorithms optimization. It is worth underlying that the computation time of m_MDC algorithm is significantly higher than the other DNAm-based methods we explored, suggesting that even high performance single-omic algorithms might be further optimized.
Toward crowdsourced and continuous benchmarking As an example of continuous benchmarking, we used DECONbench to assess the performances of two recently evaluated single-omic algorithms in a comprehensive benchmark of reference-based deconvolution pipelines. We selected the Ordinary Least Square (OLS) and Robust Linear Regression (RLR) approaches, which have been shown to be effective in estimating cellular composition of simulated bulk healthy pancreatic transcriptomes  [10]. We implemented the methods as recommended by Avila Cobos et al. , including the generation of cell-type reference profiles from a pancreatic singlecell dataset  [27]  (see supplemental information for source code: Additional file 1: Source code). Interestingly, the performance of these methods is not better than the baseline methods, possibly due to the use of healthy pancreatic cells as a reference to estimate the composition of a simulated pancreatic adenocarcinoma (Additional file 1: Figure  S1 ).
Conclusion The DECONBench platform is a unique opportunity to compare the performance of deconvolution methods on different omics data. It can be used to assess the performance of newly developed methods by applying them on high quality benchmark datasets in a user-friendly fashion. Currently, the accuracy of new methods can be compared with the eight baseline methods that have been included in the benchmarking platform. As compared with previous time-bound comprehensive benchmarks of deconvolution methods (see Avila Cobos et al. [10] ), our platform provides the possibility to continuously test and integrate newly developed methods, rather than focusing on an exhaustive comparison of existing tools. The baseline methods and user's methods performances are reported on the leaderboard and on the graphical output of DECONBench (Fig.
4 ). The source code of the baseline methods can be downloaded directly on the DECONbench platform. The structure of DECONbench is open to evolution. Work is ongoing to generate new benchmark datasets including other omic types that will be added to the platform. In the near future, we plan to expand the usability of DECONbench by offering the possibility for owners of benchmark datasets to directly upload them on the platform. DECONbench evaluation framework presents standard benchmark limitations  [15, 16] , such as the use of artificial in silico simulated data that do not capture the real experimental complexity, or the ranking of the methods based on a single performance metric. We would like to emphasize that MAE as scoring metric is only an imperfect proxy to evaluate quantification of tumor heterogeneity, as it does neither reflect the accuracy of cell-type specific molecular profile prediction (i.
Data collection and preprocessing For both transcriptome and methylome in silico mixtures, the same five cell types present in pancreatic tumors were considered (Fig. 2a, b ): tumor cells A, tumor cells B, normal pancreatic cells, immune cells and fibroblasts. Pure cell type transcriptome profiles were retrieved from the GTEX RNA-seq dataset for the immune and normal pancreatic cell types (https:// gtexp ortal. org/) and a previously published pancreatic tumor patient derived xenograft (PDX) RNA-seq dataset (E-MTAB-5039) for the two tumor cell types (total of 96 pure transcriptome profiles with 3 to 33 replicates per cell type) (Fig. 2c ). Pure cell type methylation profiles were retrieved from the same samples of the PDX dataset for the two tumor cell types and tissue or isolated cell profiles were used for the microenvironment cell types (total of 32 methylome pure profiles with 3 to 15 replicates per cell type).
Transcriptome dataset was restricted to protein coding genes and subjected to TMM normalization using the edgeR R package and log2 transformation. For the methylation data, we used the beta-value DNA methylation scores and removed probes with low-quality, that contained SNPs or located on sex chromosomes. Data were then adjusted for color balance bias and normalized between samples using the SSN (shift and scaling normalization) method using the lumi package functions (Fig. 2c ). For both omics, the median of the replicate profiles for each cell type was calculated to compute the T RNA and T MET matrices, representing the cell type specific profiles for each omic. The median calculation may prevent underlying germline differences. These matrices were used for the in-silico mixtures, as detailed in the next sections (Fig.
Description of the baseline methods The baseline methods we propose here are wrappers of already published unsupervised deconvolution algorithms (ICA-based or NMF-based). We assume here that A, T MET and T RNA are unknown and need to be estimated, either independently (single-omic pipelines) or integratively (double-omic pipelines). Before deconvolution, we systematically apply a pre-treatment step of dimensionality reduction based on feature selection. All baseline methods source code is downloadable on the DECONbench platform. All baselines relies on unsupervised deconvolution algorithms, which consists in solving D = TA , either by ICA-based (i) or NMF-based (ii) approaches. (i) ICA-based approaches (r_WIC, m_WIC and b_WIC) consist of minimizing mutual information of sources by defining independent components.
RNA_wICA (r_WIC, ICA-based deconvolution on RNA) The method RNA_wICA (r_WIC) uses transcriptomic data as input and is based on the ICA algorithm for both feature selection and deconvolution. It relies on the use of the functions "runICA" and "getGenesICA" developed by P. Nazarov (sablab. net/ scripts/LibICA. r) and the deconica R package  [23]. STEP1: feature selection For the ICA-based feature selection, the function "runICA" is run at first with the parameters ncomp = 10 and ntry = 50. Then, the function "getGenesICA" selects top-contributing genes with a FDR of 0. 2, the feature selection is done on these contributing genes belonging to a component having an average stability greater than 0. 8. Finally, duplicated genes are removed. STEP2: deconvolution First, we perform FastICA unsupervised deconvolution (deconica::run_fastica is run with the parameters overdecompose = FALSE and n.
DNAm_MeDeCom (m_MDC, NMF-based deconvolution on DNAm) STEP1: feature selection The method DNAm_MeDeCom (m_MDC), uses DNA methylation data as input and is based on the pipeline of the R package medepir. The feature selection is performed as for DNAm_EDec above to select the 5000 most variable probes. STEP2: deconvolution The deconvolution step, however, uses the MeDeCom R package  [8]. It is run with the function MeDeCom::runMeDeCom, with the lambda parameter set to 0. 01. As EDec implementation of NMF algorithm, MeDeCom algorithm consists in minimizing the error term ||D -TA|| 2 with constraints on methyla- tion values: 0 ≤ A ≤ 1 and 0 ≤ T ≤ 1 ; and constraints on proportions K k-1 A kn = 1 where A kn is the proportion of the n th sample for the k th cell type. It also uses a regularization function that favors methylation values close to 0 or 1.
Acknowledgements We thank all members of the  HADACA consortium  for helpful discussion and contributions during the HADACA data challenge 2nd edition (November 2019, Aussois, France). We also thank  Daniel Jost  and the members of the  BCM  team for inspiring discussions during regular joint group meetings. We are grateful to the Codalab data challenge open source platform. The authors gratefully acknowledge the  EpiMed core facility  for their support and assistance in this work. This work is part of the national program  Cartes d'Identité des Tumeurs  supported by the  Ligue Nationale Contre le Cancer. Where authors are identified as personnel of the  International Agency for Research on Cancer/World Health Organization , the authors alone are responsible for the views expressed in this article and they do not necessarily represent the decisions, policy or views of the  International Agency for Research on Cancer/World Health Organization.
Funding The research leading to these results was supported by  Univ. Grenoble-Alpes  via the  Grenoble Alpes Data Institute [MR, AA]  ( ANR-15-IDEX-02 ),  EIT Health Campus HADACA  and COMETH programs [MR, YB], activities 19359 and 20377 and the  Ligue Nationale Contre le Cancer. Other fundings:  South-Eastern Norway Regional Health Authority  (project number  2019030  [MJ]),  European IMI   IMMUCAN  project [NS],  European Union 's  Horizon 2020 program  (Grant  826121 , iPC project, [JM, FAC]). This article did not receive specific sponsorship in the design of the study, analysis, interpretation of data and in writing the manuscript.