dataesr/scientific-paragraphs-categorization huggingface dataset:
We define I m ðvÞ, the input to hidden unit m coming from the observed sequence v, as the sum of the weights entering that particular hidden unit:  4 ) I m ðvÞ = X i w im ðv i Þ (Equation During the modeling step called ''training'', the weights w im , the local potentials g i ðv i Þ and the parameters specifying U m ðh m Þ are inferred from data by maximizing the average log-likelihood L RBM = ClogPðvÞD data of sequence data v, as previously described  (Tubiana and Monasson, 2017; Tubiana et al. , 2019). This leads to inferring the RBM probability distribution that optimally reproduces the statistics of the training dataset (see Figure  S3H  for comparisons of data and model single-site frequency and pairwise connected correlations). During training, the contribution of sequences to L RBM can be re-weighted following a re-weighting scheme we have conceived (see STAR Methods section ''Sequence re-weighting scheme'') to correct for amino-acid frequency biases in the training dataset, as the ones introduced by mass spectrometry (MS) detection.
The package was ported to Python3 and the optimization algorithm was changed from SGD to RMSprop (i. e. , to ADAM without momentum) with learning rate 5,10 À3 , b 1 = 0, b 2 = 0:99, ε = 10 À3 ) (see  Kingma and Ba, 2015  for definition of the parameters). The parameter values were chosen for their robust learning performance across a variety of datasets studied in previous works such as MNIST, Ising models, and MSAs of protein domains of various sizes. Overall, the adaptive learning rate of RMSprop/ADAM result in larger updates for the fields and weights attached to rare amino acids, and hence speeds up convergence. as is a common choice in the ''soft'' version of K-means. We define RBM-km as the application of K-means to sequence representations in the space of inputs to hidden units (i. e.
Generation of HLA-Specific Artificial Peptides A RBM is a generative model. As it is based on fitting an entire probability distribution to a given dataset, sampling from this distribution allows one to generate new candidate antigens. The binding specificity to HLA-I of such generated sequences can be controlled by conditioning (fixing) the RBM probability on the values of inputs to hidden units coding for the desired specificity. The search for these values is guided by the HLA-I classifier. This procedure directly builds on the idea of sampling while conditioning on structural, functional, phylogenetic features emerging in RBM representations of protein families  (Tubiana et al. , 2019). Schematically, the steps of this HLA-specific sampling are as follows: 1. we select a HLA-I class c (e.
Every arbitrary amount of moves, we generate new configurations by conditional sampling as in step 2. Figure  S2C  shows examples of I n values (2 of its components for simplicity) covered in this search with b = 50 and s = 0:01 (dark red points). By this sampling procedure, we generate samples of artificial peptides that would be predicted to be presentable and specifically presented by the selected HLA-I protein complex. Such samples explore a broad diversity of peptides, that are typically 3-4 mutations away from the closest natural ones (Figure  S2D ) while preserving the profile of amino acid abundances constrained by a given binding specificity (Figure  S2E ). Sequence Re-weighting Scheme RBM presentation models trained on datasets obtained by mass spectrometry (MS) might underestimate the probability of presentation of ligands that are less frequently detected by MS.
The most evident case are ligands containing cysteine, an amino acid that accuracy across alleles, emerging especially in multi-allelic models, due to the different abundances of peptides with different HLA-I preferences available as training data. (The use of multi-allelic models is intended as tool to characterize unannotated samples through motif reconstruction). For each model, we randomly selected in the corresponding mono-allelic dataset 100 peptides to use for model evaluation. Since datasets from  (Sarkizova et al. , 2020)  feature in the latest version of IEDB, we carefully excluded the 100 peptides per model from the IEDB-derived training datasets. We produced, as described above, n-fold excess of generic peptides, choosing n in such a way as to consider a proportion between presented and generic peptides that resembles natural conditions of epitope selection.
Large-scale experimental and bioinformatic studies on viral peptidomes  (Yewdell, 2006; Paul et al. , 2013; Vitiello and Zanetti, 2017)  found that ~1% of peptides bind to given MHCs (suggesting the value n=99). Comparing the amount of all 9-long peptides from human proteome (~10 7 ) and the average, allele-specific antigen repertoire size (~10 4 sequences) derived from existing databases as IEDB, Refs. (Abelin et al. , 2017; Sarkizova et al. , 2020)  concluded rather that ~0. 1% of peptides is ''presentable'' (corresponding to n=999). These approximate estimates allowed us to fix a threshold for positive prediction at, respectively, the top-scoring 1% and 0. 1% of peptides in order to evaluate model performance by the PPV metric. (The random expectation for the PPV in these conditions is 10 À2 and 10 À3 for the PPV estimated respectively at the top scoring 1% and 0.
1%). AUCs do not vary, apart from noise, with n; the AUC values shown in Figures  1D  and S5  are the ones obtained with n=99 (i. e. , generic and presented peptides mixed in proportion 100:1). To better rationalize differences between RBM and NetMHCpan4. 0 in connection to different HLAs, we monitored performance as a function of the distance between the query HLA and its nearest neighbor in the NetMHCpan4. 0 training dataset, where such distance is determined from the similarity between the two HLA-I sequences (more precisely, the 34 residues in contact with the peptide). The distance between sequences X and Y is defined as 1 À SðX;YÞ= ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi SðX; XÞSðY; YÞ p , where e. g. SðX; YÞ is a global alignment score between X and Y based on the BLOSUM50 substitution matrix, see  (Nielsen et al.
MS-Based Model Validation for Neoantigen Discovery To evaluate the predictive power of the HLA-A*02:01-specific RBM model for neoantigen discovery, we acquired the list of missense mutations for ovarian and melanoma cancer cell lines SKOV3, A2780, OV90, HeLa and A375 from the Cosmic Cell Lines Project  (Forbes et al. , 2016). We further filtered them largely following the steps described in Ref. (Marty et al. , 2017) , i. e. we retained only mutations in genes with non-zero expression levels in the corresponding cell line (based on expression data from the Genomics of Drug Sensitivity Center;  Yang et al. , 2013)  and we excluded common germline variants documented in the Single Nucleotide Polymorphism Database  (Sherry et al. , 2001). As a result, the number of missense mutations considered is as follows: 444 (A375), 521 (SKOV3), 602 (A2780), 280 (OV90), 511 (HeLa).
For each cell line, the list of mutated peptides to score results from adding up all the up to 38 peptides of length 8-11 containing each mutation. Since the MS database for the 5 cancer lines produced by Ref. (Marty et al. , 2017)  was annotated by comparison to the consensus human proteome, it contained only wildtype peptides whose mutated version was in the list of putative neoantigens. As long as a mutation does not affect an anchor site, the mutated version should preserve a high probability of presentation. We followed the authors' choice of not considering the two peptides mutated at anchor positions as presented and we excluded them from the analysis of Figure  1E. When we consider all neoantigens (including the two neoantigens arising from anchor-site mutations of MS-derived peptides) the mean score percentiles are: RBM-MHC 3.
As the 11 neoantigens are 9-10 residues long, following Ref. (Bassani-Sternberg et al. , 2017)  we considered 9-mer and 10-mer peptides overlapping the missense mutations observed in the patient's WES, which resulted in a list of thousands of putative neoantigens (see Table  S3 ). The RBM-MHC training set per se can thus take into account only 9-10 mers from patients' samples; patients' neoantigens to validate were not included in the training dataset. In this multi-allelic case where antigens may be presented by several HLA types (as opposed to the allele-specific case of the previous paragraph) we used the global score accounting for the probability of presentation (by RBM) as well as the confidence of the HLA assignment (either by the HLA-I classifier in RBM-MHC or by K-means clustering in RBM-km) of Equation  9.
Model Predictions for SARS-CoV-2 Epitopes We downloaded the protein-coding regions of SARS-CoV-2 genome from GenBank: NC_045512. 2 (Benson et al. , 2013). We extracted all the 9-mers contained in the SARS-CoV-2 proteome, giving a list of 9656 candidate cytotoxic T-cell epitopes (HLA-I antigens). Ref. (Immunitrack and Intavis, 2020 ) (available at https://www. immunitrack. com/free-coronavirus-report-for-download/) tested in vitro the 94 top scoring epitopes according to NetMHC4. 0  (Andreatta and Nielsen, 2016)   (Jurtz et al. , 2017)  for C*01:02. 159 peptides were identified as high-stability binders (i. e. , with stability above the threshold of 60% of the reference peptide value for the corresponding HLA-I allele). Ref. (Immunitrack and Intavis, 2020 ) also estimated the trend of binding stability (expressed as % of the reference value) vs predicted score (binding affinity in [nM] for NetMHC4.
Inspection of the inputs to the 10th hidden unit (h 10 ), found by projecting peptides predicted by RBM-MHC as HLA-B*51:01-specific onto the corresponding weights (STAR Methods, Equation4), reveals a bimodal distribution, enabling the discrimination of the ''hydrophilic/charged'' pattern (i) from the ''hydrophobic'' pattern (ii), as recapitulated by the sequence logos. (C) Performance (AUC) of 6 methods for HLA-I prediction on 9-mer in 10 synthetic-individual samples, each carrying 6 HLA-I covering A, B, and C alleles (see also FigureS8; TableS2). Bars are standard deviations over the 10 datasets. (D) Sequence logos of clusters found with RBM-MHC trained on melanoma-associated sample 12T (Kalaora et al. , 2016 (Kalaora et al. , , 2018) )  and IEDB mono-allelic data; r gives the Pearson correlation between respective amino acid frequencies.
ACKNOWLEDGMENTS We thank  Benjamin Greenbaum  for suggesting to look at SARS-CoV-2 and  Clement Roussel ,  Andrea Di Gioacchino ,  Carlos Olivarez ,  Hannah Carter ,  David Gfeller ,  Shelly Kalaora ,  Yardena Samuels ,  David Hoyos ,  Jayon Lihm , and  Anna Paola Muntoni  for helpful exchanges. This work was partially supported by the  European Research Council  consolidator grant number  724208 , the  European Research Council Marie Curie-Sklodowska ITN QuanTI , the  ANR-17 RBMPro  and  ANR-19  Decrypted  CE30-0021-01 , the  ANR (Agence Nationale de la Recherche )  Flash Covid 19 -FRM, PROJET ' ' SARS-Cov-2immunRNAs '' grants. J. T. was supported by a postdoctoral fellowship from the  Human Frontier Science Program Organization  (reference number:  LT001058/2019 ). This material is based upon work supported under a collaboration by  Stand Up to Cancer , a program of the  Entertainment Industry Foundation , the  Society for Immunotherapy of Cancer , and the  Lustgarten Foundation.
Data and Code Availability This paper analyzes existing, publicly available data. These datasets' sources are provided in the Key Resources Table. RBM-MHC original code and trained models are publicly available at https://github. com/bravib/rbm-mhc. The scripts used to generate the figures reported in this paper are available at https://github. com/bravib/rbm-mhc. Any additional information required to reproduce this work is available from the Lead Contact. REAGENT or RESOURCE SOURCE IDENTIFIER Deposited Data IEDB MHC ligand data Vita et al. (2019) https://www. iedb. org/database_export_v3. php IEDB-derived multi-allelic benchmark samples This paper Table S2; https://github. com/bravib/rbm-mhc/tree/ master/data/Multiallelic_Synthetic_individuals Software and Algorithms RBM-MHC This paper https://github.
Delay-and-sum (DAS)   is the most widespread digital beamformer in high-frame-rate ultrasound imaging. Its implementation is simple and compatible with real-time applications. In this viewpoint article, we describe the fundamentals of DAS beamforming. The underlying theory and numerical approach are detailed so that users can be aware of its functioning and limitations. In particular, we discuss the importance of the -number and speed of sound on image quality, and propose one solution to set their values from a physical viewpoint. We suggest determining the -number from the directivity of the transducer elements and the speed of sound from the phase dispersion of the delayed signals. Simplified Matlab codes are provided for the sake of clarity and openness. The effect of the -number and speed of sound on the lateral resolution and contrast-to-noise ratio was investigated in vitro and in vivo.
Although the term is generally reserved for cardiac application, most transducers used in ultrasound medical imaging are phased arrays since the delays can be modified electronically in transmission or reception. Multidisciplinary DAS -In telecommunication, in contrast to ultrasound imaging, the involved travel distances are very large This work was supported in part by the LABEX CELYA (ANR-10-LABX-0060) and LABEX PRIMES (ANR-10-LABX-0063) of Université de Lyon, program "Investissements d'Avenir" (ANR-16-IDEX-0005). compared to the size of the receiver. A telecommunication antenna then receives quasi-planar waves only. An array of equally spaced unit antennas can be steered by phase-shifting to collect a radio wavefront from one direction. The radio waves of the different unit antennas are time-shifted with predefined delays (the greater the delays, the greater the receive angle) so that they can be summed to increase the signals in the desired receive direction while attenuating those from undesired directions  [3].
The denominator is the sample variance of the unwrapped phase along the  th hyperbola ℋ  associated with the scatterer located at   (). Note that   can also be calculated from compound data to return better estimates. In this case, the variances are estimated on compound hyperbolas obtained by coherent summation. We define the expected speed of sound ̂ as the one that uniformizes the phases along the hyperbolas, i. e. ̂ maximizes the   metric (Fig. 6 ): 𝑐𝑐̂= arg max 𝑐𝑐 �𝒬𝒬 𝑝𝑝 (𝑐𝑐)�. (17) In  [30] , only the sum of the phase variances -the denominator in (  16 ) -was minimized, without taking into account the respective intensities. The   metric considers these intensitiesthe numerator in (  16 ) -to prioritize the contribution of the bright speckles. The appendix 6. D provides a simplified Matlab function (called ezsos) to estimate the speed of sound by maximizing  (16).
7 ). Let  denote its number of nonzero elements. If a -point interpolation is used to estimate the signals at   (  ), we have  ≤ (  ). The right-hand term is an upper bound since limited apertures (i. e. # > 0) can be used during receive beamforming, which reduces. The sparsity  [33]  of the DAS matrix verifies: 𝑠𝑠𝑝𝑝𝑠𝑠𝑠𝑠𝑠𝑠𝑖𝑖𝑡𝑡𝑠𝑠(𝐅𝐅 DAS ) = 1 - 𝑛𝑛𝑛𝑛𝑧𝑧 𝑀𝑀𝑛𝑛 s 𝑁𝑁 e ≥ 1 - 𝑞𝑞 𝑛𝑛 𝑠𝑠. ( 19 ) To give an example, if each array element acquires 1000 I/Q samples and if the signals   are interpolated linearly (i. e. = 2), then the DAS matrix has a sparsity greater than 1-2/1000 = 99. 8%, i. e. a density smaller than 0. 2%. Sparse matrix-vector multiplication (SpMV) can be computed on GPUs  [34]. Its computational complexity is () ≤ (  )  [33]. A construction of the complex DAS matrix (for I/Q beamforming) is given in the Matlab function (called ezdas) included in the appendix.
In vitro and in vivo examples The respective effects of the speed of sound and -number on image quality were evaluated using the PICMUS dataset available online  [35]. We analyzed RF data that were acquired in an in vitro CIRS phantom (040GSE) and sampled at four times the center frequency. These data were obtained by transmitting steered plane waves with a 128-element 5-MHz linear array. The RF data were IQ-demodulated then beamformed using the DAS matrix of Eq. (  18 ) on the Cartesian image grid specified in PICMUS. A series of real envelopes were constructed from one unsteered plane wave (no transmission delay) for a large range of speed of sound (1400 to 1700 m/s) or -number (0 to 4). For reference, high-quality real envelopes were also generated by coherent compounding with the whole dataset (75 planes waves with steering ranging from -16 to +16 degrees).
The optimal -number was estimated by using Eq. (  12 ) and (  13 ) with  ℎℎ = 0. 71 (-3dB threshold). The optimal speed of sound was determined by maximizing the phase-based metric using Eq. (  16 ). Contrast and lateral resolution were quantified using the contrast-to-noise ratio (CNR) and full width at half maximum (FWHM), respectively, as detailed in  [35]. The CNR and FWHM reported in our work (Fig. 9 ) were an average from two cysts and seven nylon wires, respectively (see Figures  2b  and 2d  in  [35] ). Fig. 8. In vitro results obtained in a CIRS phantom (experimental data from PICMUS). The top and bottom rows show the effect of the speed of sound and -number on CNR (contrast-to-noise ratio) and lateral resolution, respectively. The thick vertical bars represent the optimal speed of sound and -number estimated by the equations introduced in the manuscript.
To avoid biased conclusions when comparing alternative beamformers against DAS through these metrics, it is therefore essential to ensure that the speed of sound and/or the -number have been correctly set for DAS beamforming. Another increasingly growing direction for ultrasound image formation is deep learning. Deep learning (DL) can intervene at different stages, for example at the beamforming  [48]  or compounding  [49]  level. If properly trained, DL systems are expected to provide high-quality images with fewer input data. Data-driven DL algorithms, however, are highly dependent on accurate and clean training datasets to learn from: for DL systems to learn to produce high-quality images, they must be heavily trained with high-quality images. The question then arises as to how to provide such images of optimal quality for a wide variety of organs.
These all pose hazard to cities along the mountain range front such as Almaty, Kazakhstan. Here, we explore the seismic hazard and risk for Almaty from specific earthquake scenarios. We run three historical-based earthquake scenarios (1887 Verny 8. 0) on the current population and four hypothetical scenarios for nearfield faulting. By making high-resolution Digital Elevation Models (DEMs) from SPOT and Pleiades stereo optical satellite imagery, we identify fault splays near and under Almaty. We assess the feasibility of using DEMs to estimate city building heights, aiming to better constrain future exposure datasets. Both Pleiades and SPOT-derived DEMs find accurate building heights of the majority of sampled buildings within error; Pleiades tri-stereo estimates 80% of 15 building heights within one sigma and has the smallest average percentage difference to field-measured heights (14%).
We use Agisoft Metashape version 1. 6. 2 to build dense clouds and create DEMs for the Pleiades images, after initially using ERDAS IMAGINE Photogrammetry software (ERDAS IMAGINE 2016), but found tall buildings were not resolved using ERDAS. As part of this workflow, tie-points are automatically identified between images, from which triangulation is calculated. From this, a point-cloud of points with latitude, longitude, and elevation is produced. We process this to Ultra High resolution (which uses the native resolution of the input images) and use "mild" depth filtering, as we found that increased filtering removes tall buildings as anomalous noise. We used ERDAS software standard photogrammetry workflow to process the AMEY ET AL.
10. 1029/2021EA001664 4 of 25 SPOT images; we found ERDAS produced better results than Agisoft for SPOT, discussed in Section 4. We note that AMES stereo pipeline is a freely available software not used here, as it was not designed for urban environments. Due to potential geolocation errors between the satellite-derived DEMs, we coregistered each DEM to the 12 m Tandem-X DEM following the geolocation procedure of  (Nuth & Kääb, 2011). The coregistration shifts are reported in Table  S1  and all DEMs are referenced to the ellipsoid. Pointclouds of the DEMs derived in this study are available to download on OpenTopography  (Amey, 2021a; Amey, 2021b; Amey, 2021c). We remove buildings from the pointcloud to investigate whether there are any subtle splays within the city, by filtering the buildings using LAStools  (Isenburg, 2020) , script availability listed in acknowledgments.
This is done by first removing high and low outliers in 3. 5 m cells, then classifying the lowest non-noise points, and classify the ground in steps of 25 m (as is recommended for a city  [Isenburg, 2020] ). We output the digital surface model (DSM, including buildings) and digital terrain model (DTM, ground level) at 1. 5 m resolution for Pleiades and 4 m for SPOT. (Reuter et al. , 2007) , SRTM at 30 m resolution  (Farr et al. , 2007) , ASTER Global DEM at 30 m  (Tachikawa et al. , 2011) , the AW3D (ALOS World 3D) at 30 m  (Takaku et al. , 2016, original  data are provided by JAXA). Right column are commercial DEMs including TanDEM-X at 12 m  (Borla Tridon et al. , 2013) , and DEMs that we have derived from SPOT and Pleiades stereo optical imagery at 4 and 1. 5 m respectively. The lower resolution DEMs are able to pick out large, mountain-range scale fault structures but are unable to measure houses, whereas the higher resolution clearly picks out the outline of buildings.
This means in profile P1 we find points greater than 3. 5 km distance along the profile, as this is where the road went through an underpass, which caused the GNSS to lose satellite connection leading to a lack of reliable data. We also removed points with uncertainties greater than 4 m in profile P2 (at more than 3. 8 km along the profile), and those with uncertainties greater than 2. 5 m in profile P3 (at less than  E 1. 1 km along profile). The mean error of the four GNSS profiles are 1. 2, 1. 7, 2. 2, and 1. 5 m respectively. We also took profiles along these roads in the satellite-derived DTMs, taken with bins of 5 m and swaths of 2 m for Pleiades, and bins of 10 m and swaths of 4 m for SPOT. Profiles P1 to P4 show subtle but measurable offsets which we attribute to the surface trace of a fault splay under Almaty (Figure  3 ), the magnitude of which we have calculated separately for each data set (Table  1 ).
We note in some profiles there is an offset between the GNSS and DTM elevations-both datasets are referenced to the ellipsoid and we have not done any post-processing/co-registration of the GNSS here. We also estimate the offset of the fault for six profiles for which we do not have GNSS ground-truthing (Figure  S1 ). We attribute the differences in magnitude of the offset between satellite DTMs for the same profile, to how well the buildings have been removed from the DEMs in the lastools processing as well as errors in the satellite-derived DEM. Using the location of these offsets, we map out the fault originally identified by  Grützner et al. (2017)  through Almaty for hazard modeling. In addition,  (Grützner et al. , 2017)  identified folding in the north of Almaty city, partially in the city's Alatau district.