dataesr/scientific-paragraphs-categorization huggingface dataset:
Performances can reach a plateau with sufficient cell numbers While computational parameters can have an important role in the quality of the representation  [13] , experimental ones also have a strong influence. In this section, we look at the role of the number of cells on such representations, in order to help practitioners design their experiments. For that purpose, we systematically downsample each dataset by randomly selecting a subset of cells of various size, and assess the quality of the representation obtained from the downsampled datasets. We show on Fig. 6A  the best performance reached across matrix construction for each method on each mark, as a function of the size of the downsampled dataset, for the mouse brain dataset. We further add a finer grained sweep over dataset size for ChromSCape_LSI, by increasing the size of the datset by 500 cells per step as can be seen in Fig.
6B. We see that all methods, on all datasets, benefit from an increase in the number of cells. However, it is interesting to note that the benefits resulting from a larger number of cells diminish as the number of cells increases. Indeed, we can observe that the performances increase quickly up to ∼6000 cells and then only keep increasing at a much smaller rate. PeakVI is an exception to that observation, and we can see that its performances have not yet reached this plateau; see Additional file 1: Table  S11. This is consistent with the intuition that deep learning based models require a large amount of data to achieve their full performances, and in the datasets used in this paper, this full performance does not seem to have been achieved. The gains in performances are also quite important, with an average increase of 34% by increasing the number of cells by 150% and 18% by increasing the number of cells by 66%.
On the other hand, the more standard methods, such as LSI or kernel PCA, reach their peak performances around 6000 cells, and only gain an average of 5% in performances by going from 6000 to 10,000 cells. Since these methods are the best performing Fig. 5  Role of feature selection, using the Highly Variable Gene (HVG) method used for scRNA-seq on the mouse brain dataset. A Each point corresponds to the best performance across matrix construction of a given method and a given percentage of features kept, for the 7 methods, 5 marks, and 7 feature selection conditions. B Performances of Signac and ChromSCape_LSI as a function of matrix construction on H3K4me1 and H3K27me3 for different feature selection thresholds ones in regime tested in this paper (less than 12,000 cells), it means that practitioners can sequence less cells while keeping relatively good performances.
The case of Chrom-SCape_LSI is shown in more details in Fig. 6B , where the diminishing return effect of adding more cells is very pronounced. It also allows us to confirm that the difference in performances between the enhancing and repressive marks is not due to the number of cells present in the datasets, as we see a clear separation of 3 groups: H3K27ac and H3K4me1 having the best performances, H3K9me3 and H3K27me3 following them, and finally H3K4me3 having the worst performances. The plateauing effect can also be seen for all of these marks, leading us to believe that it is not specific to just some marks. It is however very possible that given more cells (> 12,000), PeakVI and SCALE could outperform these methods and lead to better representations. This would be consistent with the behavior of deep learning-based methods on other modalities such as text of images, even though we can only conjecture that this would happen.
Discussion In this paper, we studied the role of experimental parameters, cell and feature selection, matrix construction, and dimension reduction on the quality of the representation from scHPTM datasets. We decided to focus on the quality of the dimension reduction as it is Fig. 6  Effect of downsampling uniformly at random the number of cells in the experiment. Each point corresponds to the best performance across matrix construction. A Performances the 7 methods, on the 5 marks of the mouse brain dataset and on 5 sizes of dataset (by increase of 20% of the dataset size). B Performances of ChromSCape_LSI on the 5 marks, using an increase of 500 cells per step generally the input of most downstream tasks such as clustering, cell type identification, differential enrichment, or trajectory inference.
A good representation is thus beneficial to all these tasks. Unlike other benchmarks  [12] [13] [14] , we decided not to measure the quality of the representation based only on the ability of clustering algorithms to retrieve known cell types, because there are limited number of scHPTM datasets with high-quality labels. We explored the possibility of using the labels derived from co-assays, such as RNA or protein measurements. This approach allows us to be independent of labels, as well as working with potentially continuous cell states. Yet, when using this score, we make the assumption that cells that have similar epigenomic landscapes, measured by scHPTM, should at least locally display similar RNA or protein expression patterns. While we know that this assumption holds well for enhancer marks (e.
g. , H3K4me1 or H3K27ac), it might suffer some exceptions for repressive marks (H3K27me3, H3K9me3). We can note that this approach has already been successfully used in evaluating scATAC-seq pipelines in  [18, 19]. In addition to this neighbor score, we have also included in our study measurements of ARI and AMI based solely on labels from authors, obtained computationally from the epigenomic datasets. In this case, the main limitation is that the method evaluation will tend to favor the method initially used by authors. While we expected the choice of matrix construction algorithm to have an impact, that impact is larger than what we expected a priori. Indeed, as is shown in Additional file 1: Table  S4 , the performances using the best bin size can be up to 80% better than using the worst one.
Surprisingly, the ranges of bin size are larger than what could be expected, and we were also surprised to find that enhancer marks such as H3K4me1 benefited from large bin sizes (up to 200 kbp) despite being known to accumulate into small peaks (in the order of a few kbps  [17] ). Yet, at a bin size of several 100 kbp, embeddings will not rely on local epigenomic enrichments, such as the ones observed for enhancers. The coverage of current scHPTM datasets might not be sufficient to produce reliable embeddings from small bins and may thereby be unable to distinguish cell states that differ by only few local enrichments. Identifying differences in enrichment for smaller regions than the bin size used for the embeddings can however be done when running differential enrichment analysis with a more appropriate bin size, while using the clusters obtained from the embedding for annotation.
One could imagine that these models could show better performances with larger datasets, such as cell atlases, but they do not seem appropriate for experiments as they are currently designed. On the other hand, we found that the performances of all methods largely benefited from being run on high coverage cells and that these performances did not stagnate on the available data, suggesting that future improvements of protocols-increasing coverage, such as in  [36] -will surely provide additional information and granularity to refine embeddings. We were also surprised to observe that feature selection, using either a variance or a coverage criteria, almost always had a negative impact on the performances. This may be due to the excessively low coverage per cells compared to other protocols where this procedure can be beneficial (see  [13]  for scRNA-seq).
Matrix construction We downloaded the mouse brain dataset from GSE15 2020. The data come in count matrix format, with 5 kbp bins for all marks except for H3K4me3 which is in 1 kbp bins. The larger bin sizes were obtained by merging the original bins together to form new bins using a custom script, available at https:// github. com/ vallo tlab/ bench mark_ scepi genom ics. The GeneTSS annotation comes from the ChromSCape package, and the matrix was done by merging the bins containing the regions in that annotation using the same custom script. We keep all the cells present in that matrix, as the original authors already applied QC steps on the cells. The human PBMC dataset data was downloaded from https:// zenodo. org/ record/ 55040 61; the data was processed from the fragment files. We used ChromSCape for generating 5 kbp matrices and then used our custom script to generate the other matrices similarly to the mouse brain dataset.
The MacsPseudoBulk annotation was obtained by turning the fragment files into bams, calling the peaks using MACS2, and then merging them using bedtools. The various SICER2 annotations were also generated by turning the fragment filed into beds, calling the peaks with SICER2 with island width of 200, 500, 2000, 5000, 20000, and a gap size thrice the size of the island size. We then select only the cells used in the original paper analysis, by keeping only the barcodes present in the rds objects on Zenodo. The scChIP data was downloaded from GEO under the SuperSeries GSE16 4716, the matrices were constructed using ChromScape.
scRNA-seq and CITE-seq processing The scRNA-seq matrix for the mouse brain dataset was processed using the scanpy  [21]  package and following their best practice notebook (https:// scanpy-tutor ials. readt hedocs. io/ en/ latest/ pbmc3k. html). We have previously shown in  [13]  that the algorithms used in that package are robust and perform well. The CITE-seq matrix for the human PBMC dataset was extracted from the rds objects and processed with standard PCA.
Representations for scHPTM For computing the representations using the different methods, we used the implementation from the original packages, except for SnapATAC for which we used the reimplementation of  [12]  as it allowed a nicer API for running a large number of jobs; their implementation can be found on their github https:// github. com/ pinel lolab/ scATACbench marki ng. For cisTopic, we ran the runWarpLDAModels method from the cis-Topic Bioconductor package (version 0. 3. 0) and followed the steps from  [12]. For Signac, we followed the scATAC-seq best practices vignette (https:// satij alab. org/ signac/ artic les/ pbmc_ vigne tte. html) and used the Signac CRAN package (version 1. 7. 0). For Chrom-SCape_LSI and ChromSCape_PCA, we processed the matrix with the tpm_norm and TFIDF methods respectively, then called the pca method, and removed the first principal component; all the methods were callled from the ChromSCape Bioconductor package (version 1.
6. 0). For PeakVI, we followed the tutorial on the package website https:// docs. scvi-tools. org/ en/0. 15. 1/ tutor ials/ noteb ooks/ PeakVI. html using the scvi-tools (version 0. 15. 1) [37]  pip package. Since SCALE did not have an API for calling their model, we modified the main. py script from the scale python package (version 1. 1. 0), so that it does not remove cells. The scripts for processing used for all R methods are in the R_analysis. R script, PeakVI, and SCALE are respectively peakVI_process. py and scale_process. py scripts. The R methods were run on CPUs with 16 cores and 32 GB of RAM, the deep learning ones (PeakVI and SCALE) on V100 GPUs with an additional 2 cores CPU.
Neighbor score computation To compute the neighbor score of an scHPTM representation, we first compute the k nearest neighbor graphs (kNNG) for values of k ranging from 0. 1% up to 10% of the cells present in the dataset. We then compute the representation for the second modality using scanpy  [21] , whose algorithm (PCA) has been identified in  [13]  as being the most reliable for achieving good representations for scRNA-seq. We then compute kNNG on this second representation, count the number of common neighbors in the kNNG for each cell, divide by k, and average over the cells. This gives a score between 0 and 1, where 1 means that the two representations perfectly agree on which cells are similar, and a score of 0 means complete disagreement. We further average that score across the various values of k, which were selected to be 0.
Compute time The runtime for each method has been measured on the H3K4me1 Mouse brain data, with a matrix built with 100 kbps. We measure the runtime as a function of the number of cells in Fig. 7  and as a function of the number of features in Fig. 8. All measures were run on a computer with CPU with 32 GB of DDR4 RAM and an NVidia 2080Ti GPU. SCALE and PeakVI were trained using GPU acceleration. The time measured only accounts for training the models and generating the cells representation, reading the data and transforming it into the appropriate format was left out in order to avoid unfairly advantaging some methods over others. The fact that SCALE appears faster as the number of cells increases is due to the fact that it is trained by default by seeing a fixed number of cells (30000) instead of seeing all the cells multiple times.
Availability of data and materials The code used to perform the experiments and the analysis can be found on the github || have been deposited on Zenodo  [38, 39]. The mouse brain data was obtained from the NCBI Gene Expression Omnibus (GEO) (http:// www. ncbi. nlm. nih. gov/ geo/) under accession number GSE15 2020. The human PBMC data was obtained from Zenodo  [40]. The MDA-MB468 data was obtained from GEO, under accession number GSE16 4716.
, 2019). In parallel, other additional tools relevant to medicinal chemistry, like NP-likeness scores  (Ertl et al. , 2008)  or chemical similarity search filters offered by software such as DataWarrior (DW)  (DataWarrior User Manual, 2015; Sander et al. , 2015)  open the possibility to query the metabolome information from a new perspective. In this context, effective queries can only be performed if bioactive hits were previously characterized and if valuable bioactivity data on reference chemical libraries is available. Analysis of bioassay results on standard compound libraries can therefore pave the way for the search of chemically related compounds in the metabolome data of natural extracts and enable their isolation and characterization as new hits of interest. In this context and in recent years, several phenotypic screens were performed in the context of infected host models to identify antimycobacterial compounds  (Brodin et al.
Consequently, we can easily evaluate the anti-infective properties and selective activity of NP samples in one combined assay. Additionally, samples can also be tested on M. marinum growing in broth to evaluate their antibiotic potential. Taken together, testing samples both "in infection" and "in broth" can allow us to confidently identify samples which are active on mycobacteria residing inside a host cell. Herein, we leveraged our previously performed screen of druglike compounds on the D. discoideum-M. marinum infection model system  (Tobin and Ramakrishnan, 2008). A Highly Diverse Pathway-Based Library (HD-PBL) consisting of small ligandbased chemicals was established. It comprised 1,255 different compounds chosen from the ZINC lead-like database  (Sterling and Irwin, 2015) , with an incentive to maximize the chemical diversity of the library and the drug likeness of candidates chosen.
This was achieved by querying the ZINC database for 18 host-pathogen interaction relevant biological pathways and collecting chemo diverse hit pharmacophores, determined by the LINGO method  (Vidal et al. , 2005). The derived set of compounds was screened for anti-infective activity in a combination of three different assays including the D. discoideum-M. marinum infection model. From this study, 37 compounds stood out as hit structures, with 29 specifically inhibiting the intracellular growth of M. marinum by more than 20%  (Hanna et al. , 2020). In the present study, we took advantage of the screening of drugrelated compounds on the above-mentioned D. discoideum-M. marinum infection model system by specifically searching for potential hits through their predicted structures in natural extracts (metabolomics).
d. , 15 μm, Montluçon, France). Semi-preparative HPLC-UV was conducted on a Shimadzu system equipped with an LC-20 A module pumps, an SPD-20 A UV/VIS, a 7725I Rheodyne ® valve, and an FRC-40 fraction collector (Shimadzu, Kyoto, Japan) using an XBridge BEH C18 OBD Prep column (250 × 19 mm i. d. , 5 μm, Waters ® ). NMR data were recorded on a Bruker Avance Neo 600 MHz NMR spectrometer equipped with a QCI 5 mm cryoprobe and a SampleJet automated sample changer (Bruker BioSpin, Rheinstetten, Germany). Technical grade hexane (Hex), ethyl acetate (EtOAc) and methanol (MeOH) were purchased from Thommen-Furler AG, Rüti b. Büren, Switzerland. HPLC grade methanol and ethyl acetate were purchased from Fisher Chemicals, Reinach, Switzerland. LC-MS grade water, acetonitrile (ACN), and formic acid were purchased from Fisher Chemicals, Reinach, Switzerland.
Plant material The plant containing the compounds of interest was Cananga brandisiana (Pierre) Saff. (Annonaceae). This plant belongs to the Pierre Fabre Laboratories (PFL) collection with over 17,000 unique samples collected worldwide. The PFL collection was registered at the European Commission under the accession number 03-FR-2020. This registration certifies that the collection meets the criteria set out in the EU ABS Regulation which implements at EU level the requirements of the Nagoya Protocol regarding access to genetic resources and the fair and equitable sharing of benefits arising from their utilization  (Sharing nature, 2022). PFL supplied all the vegetal material (ground dry material). The collected samples have photographs, herbarium vouchers, and leaf extracts preserved in dry silica gel.
Precise localization of the initial collection, unique ID and barcode, and GPS data are stored in the dedicated data management system. The plant material was dried for 3 days at 55 °C in an oven; then the material was ground and stored in plastic pots at controlled temperature and humidity in PFL facilities. Two plant parts of C. brandisiana were used in this study, with their following unique ID within the PFL collection: V114260 (roots) and V114261 (trunk bark).
HRMS data analysis and generation of a molecular networking Proteowizard's msConvert software (v. 3. 0) was used to convert raw MS files into mzXML formats  (Kessner et al. , 2008; Chambers et al. , 2012). The converted files were treated by MZMine software v. 2. 53. In positive mode, MS1 and MS2 for each scan were detected at threshold of 1E6 and 0, respectively. The ADAP module was used to build chromatograms by connecting data points from mass lists. The chromatograms were deconvoluted into individual peaks by application of the "wavelet" algorithm. In the presence of isotopic patterns, isotopes were grouped to the lowest m/z ion. The extracted ions were aligned in a table and represented as features that demonstrate m/z, RT and peak area. The parameters for each mentioned treatment were adjusted according to  Rutz et al.
(2022). MNs were built online through the GNPS platform (GNPS, 2016) and visualized by Cytoscape, 2001 (Cytoscape: an open-source platform for complex network analysis and visualization, 2001). The GNPS parameters were adjusted according to  Houriet et al. (2020). For the annotation of the features generated, each of their recorded MS/MS spectra was compared against an In-Silico MS/ MS fragmentation database (ISDB)  (Allard et al. , 2016)  corresponding to structures reported in the Dictionary of Natural Products (DNP)  (Chapman, 2020)  using taxonomical informed scoring  (Rutz et al. , 2019).
DataWarrior structure similarity search DataWarrior is open and interactive software for data analysis and visualization that integrates well-established and novel chemoinformatics algorithms in a single environment and notably offers the possibility of exploring chemical space  (López-López et al. , 2019). In DW, the FragFp descriptor (value ranging from 0 to 1) was used to establish chemical similarity to a reference compound as a filter. A threshold of 0. 80 on the latter metric was chosen as a minimum value for the descriptor to display chemically similar structures (www. openmolecules. org, 2015).
Semi-preparative HPLC-UV Fractions from the previous Flash chromatography purification step with MS signals potentially corresponding to alkaloids of interest (fractions number F49, F51 + F52, F56, F57, F59), were subjected to further purification using a Shimadzu system equipped with an LC-20 A module pumps, an SPD-20 A UV/VIS, a 7725I Rheodyne ® valve, and an FRC-40 fraction collector (Shimadzu, Kyoto, Japan). The system was controlled by the LabSolutions software, also from Shimadzu. Each fraction was dissolved in 200 µL of MeOH and was mixed with the stationary phase (C18 Zeoprep ® 40-63 μm, one spatula), to form uniform slurries. MeOH was evaporated to obtain fine powders which were introduced in a dry-load cell according to our previously published method  (Queiroz et al. , 2019). The separation was performed with an XBridge BEH C18 OBD Prep column (250 × 19 mm i.
, 2020). Among hits obtained on our intracellular M. marinum infection model, a first step of determination of their possible structural similarity with Natural Products was carried out. This was performed by calculation of a "Natural-Product likeness" score (cf. 3. 1)  (Ertl et al. , 2008)  virtual chemical space. The re-ranked list of structures according to this score was used for a systematic search of analogs in the virtual chemical space consisting of all structural annotations obtained on the massive metabolome dataset (1,600 extracts) of our Natural Extracts (NE) library  (Allard et al. , 2023). To efficiently search for hits in the virtual chemical library of all extracts, a software enabling molecular similarity measures (Datawarrior) was used. This procedure allowed the ranking of the annotated structures that were the most similar to each of the identified hits of the ZINC library.
Determination of the NP-likeness-score for hits of the ZINC library From our previous study  (Hanna et al. , 2020) , the 37 hit compounds active on M. marinum were considered. We excluded hits from anti-virulence assays on M. marinum to focus exclusively on hits that reduced its growth in infection, resulting in 29 hit compounds (Supplementary Data table for DW). A first round of filtering was applied to the 29 hit compounds from the ZINC Library, using a score of "Natural Product likeness" described by  Ertl et al. (2008)  for each of those hits. This score translates into numerical values that vary between -5 (least NP-like) and +5 (most NP-like). Application of this NP-likeness score gives values ranging from -2. 667 to 1. 718 for these 29 hit compounds. Based on the distribution of the NP-likeness scores for a collection of synthetic molecules (mostly negative scores) compared to that of Natural Products libraries (mostly positive scores), it was decided that a cutoff at 0 would be suitable for the choice of prioritized NP-like hits.
Search for analogs of the NP-like hits in a virtual library of annotated compounds In order to search for these four compounds (A, B, C, D) and/or analogs in NEs, we exploited a set of structural annotations obtained from a library of 1,600 chemo-diverse plant extracts previously profiled by LC-HRMS/MS analysis  (Allard et al. , 2023). For this we compared spectra of recorded MS/MS spectra against an In-Silico MS/MS fragmentation database (ISDB)  (Allard et al. , 2016)  corresponding to structures reported in the Dictionary of Natural Products (DNP) ( ® CRC Press, 2020) using taxonomical informed scoring (Rutz   et al. , 2019). This yielded a set of 36,128 annotations (which can be found in Supplementary Tables  S1, S2 ). In this library of putative annotations, all compounds annotated are linked to the extract they originated from and this can be efficiently visualized at the level of each detected feature in MNs.