dataesr/scientific-paragraphs-categorization huggingface dataset:
Parameter Fixing, Linking, and Optimization Parameters After an MC-model is instantiated, the last step before the potential model fitting is the imposition of linked or fixed parameters or the addition of optimization parameters: • Parameter linking is used to impose certain mathematical relationships between otherwise independent parameters. Examples are setting parameters equal to each other (e. g. , telling two models to have the same orientation), imposing a tortuosity constraint (e. g. ,  Szafer et al. , 1995) , or imposing that a parameter must be smaller or equal to another parameter (e. g. , λ ⊥ ≤ λ for a Zeppelin); • Parameter fixing is used to enable fixing of known global diffusion properties (e. g. , CSF diffusivity), or to do parameter cascading, which is to fix voxel-wise varying parameters for a certain dataset, which have been optimized from a previous model; • Optimization parameters are used to rescale a model parameter to a rescaled version that is more appropriate to the optimization problem, like using the Orientation Dispersion Index (ODI) in lieu of concentration parameter for a Watson distribution  (Zhang et al.
Fitting MC and MC-SM Models From the perspective of optimization, MC and MC-SM models, given in Equations (  3 ) and (  10 ), can be treated equally as nonconvex optimization problems. Currently, we implement two optimization algorithms: Brute2Fine: The standard "brute2fine" optimizer first uses brute force optimization by sampling all optimized parameters p on an equispaced grid of Ns samples between each parameter's minimum and maximum optimization bounds. Obtaining the best fitting starting position based on Equations (  3 ) or (10), we then use the constrained optimization algorithms L-BFGS-B  (Byrd et al. , 1995)  of the open-source scipy package  (Jones et al. , 2001)  to obtain the locally best fitting parameters p *. The unity constraint on f i is enforced by nesting volume fraction following  Zhang et al.
IVIM Data For our first demonstration, we will use a freely available dataset with an IVIM acquisition, which is downloadable from the dipy examples page  (Garyfallidis et al. , 2014). It is a small dataset with only 21 DWIs at b-values spread between 0 -1, 000 s/mm 2 , including a single b 0 measurement. Information about diffusion times and TE are unknown for this data, but these will not be needed to fit the 2-compartment Gaussian IVIM model.
Spatio-Temporal Cat Spinal Cord Data To illustrate MC-models that are aimed at estimating axon diameter, we use a recent dataset where both AxCaliber and multi-shell diffusion MRI acquisitions have been registered to one axial slice of cat spinal cord  (Duval et al. , 2016). For each voxel, the mean axon diameter, restricted volume fraction and myelin volume fraction are known from histological measurements. The dMRI data is 64 × 64 voxels with resolution 0. 16 × 0. 16 × 0. 16 mm 3. One AxCaliber acquisition was acquired (perpendicular to the axon axis) with parameters δ=3/8/8/8 ms, = 7/12/25/40 ms, G = [0. 849] mT/m (199 increments) and TE minimized (36-62 ms). The data was TE-normalized by dividing the data for every TE by the G = 0 signal. The multi-shell acquisition was acquired with parameters δ = 3 ms, = 30 ms, 4 shells with b-value={40, 189, 1,680, 6,720} s/mm 2 , TE = 47 ms, with a total of 796 diffusionweighted images.
Multi-Shell WU-Minn Human Connectome Project Data To illustrate MC-models that require only multi-shell data, we use the WU-Minn Human Connectome Project data  (Feinberg et al. , 2010; Moeller et al. , 2010; Setsompop et al. , 2012; Xu et al. , 2012; Glasser et al. , 2013; Van Essen et al. , 2013). In this dataset, the diffusion directions were obtained such that every subset of the first M directions is still isotropic  (Caruyer et al. , 2013). The data was sampled on 3 shells with b-values {0, 1,000, 2,000, 3,000} s/mm 2 , with {14, 90, 90, 90} directions, respectively. The diffusion time and pulse separation time in this data are δ/ = 10. 6/43. 1 ms with 2 × 2 × 2 mm resolution and TE/TR = 89. 5/5, 520 ms.
RESULTS: DMIPY MC-MODEL IMPLEMENTATIONS In this section, we demonstrate the power of Dmipy's modular MC-modeling design. We start by implementing, fitting and showing the results of many well-known MC-models known from the literature. The models we will illustrate are listed in Table  3 , where we also indicate each model's composition in terms of biophysical models, the number of lines in Dmipy code it takes to implement, and the primary reference. Note that it is not the point here to deeply study each model's behavior, but to demonstrate how simple it is to create and fit any of them. Deeper, more detailed studies of each model are readily available as examples at the Dmipy Github page at https://github. com/AthenaEPI/dmipy. For each model, we will also denote its schematic, mathematical representation in terms of parameter dependencies, where we will use the "|" sign to separate estimated from fixed parameters during the optimization.
Generalized, On-the-Fly Multi-Compartment Modeling and Signal Generation Dmipy's key contribution is the way it generalizes dMRI signal modeling regardless of the MC-model composition or acquisition scheme design. In this way, Dmipy can be better seen as a "model generator, " i. e. , it does not hard-code any model combinations, but can generate any one of them on the fly. By standardizing the interaction between the PGSE acquisition scheme representation and the mathematical representations of any biophysical model, all signal representations become the same from a user point of view. An MC-model can then be seen as nothing more than an aggregated set of the input model parameters, including volume fractions, which is neatly represented in Equation  (3). In this way, Dmipy enables the user to have only highlevel interaction in choosing which model is appropriate the chosen application and never has to worry about the underlying mathematical implementation.
Throughout this work we have provided the basic commands in Snippets that allows easy interaction with all aspects of MCmodeling. In particular, we can load the acquisition parameters for any PGSE sequence (Snippet 1), generate the dMRI signal for any individual compartment model (Snippet 2), design any MC-model variant and set S 0 responses (Snippet 5), define parameter links (Snippet 6), and fit an MC-model to data (Snippet 7), from which the fitted model parameters and fitting error can be obtained. Using these elementary commands, we can implement and fit many well-known MC-models from literature in around 10 lines of code (see Table  3 ). In section 3, we explicitly provided the Dmipy code and described the estimated model parameters of IVIM  (Le Bihan et al. , 1988) , AxCaliber  (Assaf et al.
Modular, Generalized Optimization Algorithms By unifying the theory of MC, MC-SM, and MC-SH modeling, by extension Dmipy allows generalized implementations of standard or dMRI-specific data fitting algorithms, based on Equations  (3, 10, 12). Calling different optimization algorithms can be done using a single line of code (see Snippet 7). For MC and MC-SM models, Dmipy uses Scipy  (Jones et al. , 2001)  to implement the "brute2fine" optimizer, using standard bruteforce to find a good starting point, and then using L-BFGS-B  (Byrd et al. , 1995)  to refine the solution to a local minimum. Enforcing the unity constraint on the volume fraction is done by nesting the volume fractions based on works by  Zhang et al. (2012). We also implement the recent MIX algorithm by  Farooq et al. (2016) , taking advantage of Scipy's differential evolution algorithm  (Storn and Price, 1997).
et al. Dmipy: Diffusion Microstructure Imaging in Python Snippet 3 | One-line instantiation of parameter and spherical distributions and DistributedModels. Notice DistributedModels can be seen as sub-MC-models that apply a distribution to the parameters of base CompartmentModels. 1 from dmipy. distributions import parameter_distributions, spherical_distributions 2 # instantiation of the distributions themselves 3 gamma = parameter_distributions. Gamma() 4 watson = spherical_distributions. Watson() 5 bingham = spherical_distributions. Bingham() 6 7 # one-line instantiation of distributed models given input models 8 from dmipy. signal_models import distributed_models 9 gamma_bundle = distributed_models. PD1GammaDistributed( 10 models=[model1], parameter_name=parameter_name) 11 watson_bundle = distributed_models.
DATA AVAILABILITY STATEMENT code and all results in this work can be completely reproduced using the tools available at the Dmipy repository at https://github. com/AthenaEPI/dmipy. The Human Connectome Project (HCP) data used to illustrate several of the algorithms is available at https://db. humanconnectome. org/. The spinal cord data is available at the White Matter Microscopy Database at https://dipy. org. The IVIM dataset was downloaded through the Dipy example at http://nipy. org/dipy/examples_built/reconst_ ivim. html.
Fick et al. Dmipy: Diffusion Microstructure Imaging in Python Snippet 3 | One-line instantiation of parameter and spherical distributions and DistributedModels. Notice DistributedModels can be seen as sub-MC-models that apply a distribution to the parameters of base CompartmentModels. 1 from dmipy. distributions import parameter_distributions, spherical_distributions 2 # instantiation of the distributions themselves 3 gamma = parameter_distributions. Gamma() 4 watson = spherical_distributions. Watson() 5 bingham = spherical_distributions. Bingham() 6 7 # one-line instantiation of distributed models given input models 8 from dmipy. signal_models import distributed_models 9 gamma_bundle = distributed_models. PD1GammaDistributed( 10 models=[model1], parameter_name=parameter_name) 11 watson_bundle = distributed_models.
Typical DmipyAcquisitionScheme creation using b-values, gradient orientations as 3D unit vectors, and optional single-value or arrays of pulse durations δ, pulse separations and TEs. The DmipyAcquisitionScheme handles all interactions with the fixed acquisition parameters, meaning we only need to focus on CompartmentModel parameters afterwards. Notice that the printed acquisition information is not in SI units to avoid too small or large numbers, but the input must always be given in SI units. Fick et al. Dmipy: Diffusion Microstructure Imaging in Python Snippet 1 | 1 from dmipy. core. acquisition_scheme import acquisition_scheme_from_bvalues 2 dmipy_scheme = acquisition_scheme_from_bvalues( 3 bvalues, # of shape (N_DWI,) in s/m^2 4 gradient_directions, # of shape (N_DWI, 3) 5 pulse_duration, # number or of shape (N_DWI,) in seconds 6 pulse_separation, # number or of shape (N_DWI,) in seconds 7 TE) # number or of shape (N_DWI,) in seconds 8 9 dmipy_scheme.
One-liners to setting fixed or linked parameters to instantiated distributed model or MC-models. Fick et al. Dmipy: Diffusion Microstructure Imaging in Python Snippet 6 | 1 # tortuous parameters can be set at distributed model level 2 distributed_model. set_tortuous_parameter( 3 tortuous_diffusivity_parameter_name, 4 volume_fraction_name, 5 parallel_diffusivity_parameter_name) 6 7 # fixing parameters for optimization 8 mc_model. set_fixed_parameter(parameter_name, value) 9 mc_model. set_initial_guess_parameter(parameter_name, value) 10 11 # other linked parameters set at MC level 12 mc_model. set_equal_parameter(parameter_name_1, parameter_name_2) 13 mc_model. set_fractional_parameter( 14 fractional_parameter_name, 15 target_parameter_name).
Dmipy implementation of Ball and Stick in 6 lines of code. Snippet 11 | Dmipy implementation of NODDI or NODDIx with 2 bundles in 12 lines of code. The implementation of Bingham-NODDI is equivalent to Watson-NODDI, only replacing SD1WatsonDistributed with SD2BinghamDistributed. Fick et al. Dmipy: Diffusion Microstructure Imaging in Python Snippet 10 | 1 from dmipy. core. modeling_framework import MultiCompartmentModel 2 from dmipy. signal_models import gaussian_models, cylinder_models 3 ball = gaussian_models. G1Ball() 4 stick = cylinder_models. C1Stick() 5 ball_and_stick = MultiCompartmentModel(models=[ball, stick]) 6 ball_and_stick. set_fixed_parameter('C1Stick_1_lambda_par', 1. 7e-9) 1 from dmipy. core. modeling_framework import MultiCompartmentModel 2 from dmipy. signal_models import gaussian_models, cylinder_models, distributed_models 3 ball = gaussian_models.
Dmipy implementation of Single-Shell Three-Tissue (SS3T) Meta-Algorithm in 19 lines (ignoring commenting lines). Fick et al. Dmipy: Diffusion Microstructure Imaging in Python Snippet 14 | 1 from dmipy. tissue_response. three_tissue_response import three_tissue_response_dhollander16 2 from dmipy. core. modeling_framework import MultiCompartmentSphericalHarmonicsModel 3 S0_tissue_responses, tissue_response_models, selection_map = three_tissue_response_dhollander16( 4 scheme_hcp, data_hcp, wm_algorithm='tournier13') 5 N_iterations = 4 # following Dhollander16 6 fit_args = {'acquisition_scheme': acquisition_scheme, 7 'data': data, 'mask': data[. , 0] > 0} # where b0 has positive value 8 for it in range(N_iterations): 9 # step one: fix WM and fit GM + CSF 10 mt_csd_mod = MultiCompartmentSphericalHarmonicsModel(tissue_response_models, S0_tissue_responses) 11 if it == 0: 12 mt_csd_mod.
The number of human genomes being genotyped or sequenced increases exponentially and efficient haplotype estimation methods able to handle this amount of data are now required. Here we present a method, SHAPEIT4, which substantially improves upon other methods to process large genotype and high coverage sequencing datasets. It notably exhibits sub-linear running times with sample size, provides highly accurate haplotypes and allows integrating external phasing information such as large reference panels of haplotypes, collections of prephased variants and long sequencing reads. We provide SHAPEIT4 in an open source format and demonstrate its performance in terms of accuracy and running times on two gold standard datasets: the UK Biobank data and the Genome In A Bottle.
Specifically, it allows processing either SNP array or sequencing data accurately with running times that are sub-linear with sample size, therefore making it well suited for very large-scale data sets. In addition, it also facilitates the integration of additional phasing information such as reference haplotypes, long sequencing reads, and sets of pre-phased variants altogether to boost the quality of the resulting haplotypes. To achieve this, the method builds on three main components: (i) the Li and Stephens model (LSM)  18  to capture long-range haplotype sharing between individuals, (ii) the Positional Burrows-Wheeler Transform (PBWT)  19  to speed up the computations involved in the LSM and (iii) the compact representation of the solution space built in previous versions of SHAPEIT  20, 21  , which allows efficient haplotype sampling and easy integration of additional phasing information  17, 22.
Results Overview of SHAPEIT4. SHAPEIT4 improves upon previous SHAPEIT versions at two main levels. As a first major improvement, it now uses an approach based on the PBWT to quickly assemble small sets of informative haplotypes to condition on when estimating haplotypes. This provides a computationally efficient alternative to the previous approach based on Hamming distance  20, 24. A PBWT of haplotypes is a data structure in which any two haplotypes sharing a long prefix (i. e. , match) at a given position are sorted next to each other at that position. SHAPEIT4 takes advantage of this by maintaining a PBWT of all the haplotype estimates so that long matches between haplotypes can be identified in constant time. In practice, SHAPEIT4 works within overlapping genomic regions (of 2 Mb by default) and proceeds as follows to update the phase of an individual in a given region: (i) it interrogates the PBWT arrays every eight variants to get the P haplotypes that share the longest prefixes with the current haplotype estimates at that position, (ii) it collapses the haplotypes identified across the entire region into a list of K distinct haplotypes, and (iii) it runs the LSM conditioning on the K haplotypes (Fig.
1a, b ). In this approach, P is the main parameter controlling the trade-off between speed and accuracy and gives a model in which K varies and adapts to the data and region being processed as opposed to previous methods in which K is usually fixed  20, 24. Indeed, K varies depending on the length of the matches found in the PBWT: longer matches involve smaller K, which typically occurs as the algorithm converges, as the level of relatedness between individuals increases and more importantly as the number of samples in the data set increases. The latter implies, and we later show, that SHAPEIT4 scales sub-linearly with sample size. All other methods proposed so far exhibit at best linear or close-to-linear scaling. In other words, the time spent per genome decreases as the total number of genomes being processed increases.
We predict that these two functionalities will help improving phasing servers dedicated to whole-genome sequenced samples  29. It is also important to mention here that SHAPEIT4 is not designed to perform genotype calling from low coverage sequencing and adapting the model for such data would need some additional work. Overall, we believe SHAPEIT4 to be particularly well suited to fully exploit the potential of the large data sets being generated using either SNP arrays or new sequencing technologies.
Fig. 4  Phasing performance on high-coverage sequence data (GIAB). a Switch error rates with 95% binomial confidence intervals for each combination tested of sequencing reads and haplotype scaffold. b Same information than in a measured only at variants with minor allele frequency (MAF) above 1%. c Switch error rates measured only at variants belonging to phase sets (i. e. , haplotype assembly) before (squares) and after (triangles) refinement by SHAPEIT4. Results are shown here assuming 0. 01% error rate in the phase sets. Source data are provided as a Source Data file. -We defined more compact data structures to store genotype graphs in memory, which resulted in a >3× decrease in memory usage. -We optimized the code for HMM computations to make it cache friendly and vectorised. This speeded up the HMM computations by ~3×.
In SHAPEIT4, we implemented three new features to improve the mixing of the MCMC: -Initialization: We use a quick-and-dirty initialization of the haplotypes so that the MCMC does not start from a random resolution. To do so, we use a simplified version of the recursive phasing approach implemented in the PBWT software package (see https://github. com/richarddurbin/pbwt), which provides initial haplotype estimates very quickly in O(NL) with SERs in the range of 8-10% in our data. In practice, we first build a vector B l. + 1 that contains the 2N alleles carried by the N individuals at variant l + 1 in which reference alleles are encoded as -1, alternative ones as 1 and those occurring at heterozygous genotypes as 0. Then, we phase genotypes by progressively imputing missing data in B l + 1 (i.
e. , zeros) given the PBWT prefix array A l derived from the previous position l. Specifically, we make inference on the alleles carried by the two haplotypes of a given individual by copying those carried by neighbors as defined by the PBWT prefix array A l (i. e. , located just before and after in A l ). Of note, for efficiency reasons, this procedure also requires to keep track of indexes of the haplotypes in A l : we store those in a vector I l (see Fig. 5 ). Details on the exact procedure can be found in Fig. 5. Once all missing data filled, we build the prefix array A l + 1 for position l + 1 and move to the next, l + 2, until completion. -IBD2 protection: We designed an approach to prevent SHAPEIT4 to copy haplotypes across individuals sharing both of their haplotypes IBD (i. e. IBD2).
Haplotype scaffold. SHAPEIT4 also allows for some heterozygous genotypes to be phased a priori. This approach has been previously introduced in SHAPEIT2 in the context of the 1000 Genomes project to perform genotype calling from low coverage sequencing data  20. This scaffold of pre-phased heterozygous genotypes can originate for instance from pedigrees in which many of the children' genotypes can be accurately phased using Mendel inheritance logic. In practice, we implemented this functionality by simply pruning out all haplotype configurations in the genotype graphs that are inconsistent with the available scaffold of haplotypes (Supplementary Fig. 9 ). Of note, SHAPEIT4 does not have any requirements in terms of variant overlap between the main genotype data and the scaffold haplotype data and therefore allows the latter to be derived from SNP array data while the former from high-coverage sequencing data.
Using the BCF format significantly speeds up I/O operations. VCF/BCF formats natively define phase sets, which facilitates the integration of phase information contained in sequencing reads in SHAPEIT4. HTSlib allows reading simultaneously multiple VCF/BCF files, which facilitates complex SHAPEIT4 runs combining, for instance, genotype data with phase sets, a reference panel of haplotypes and a scaffold of haplotypes. The UKB data sets. We generated genotype data sets with large sample sizes from the full release of the UKB data set containing 488,377 individuals genotyped by SNP arrays at 805,426 genetic variants. To do so, we proceeded according to the five following steps. First, we filtered out all genetic variants with more than 5% missing genotypes. Second, we extracted data only for chromosome 20, resulting in 18,477 variants in total.
Third, we build trio candidates (two parents and one child) from the pairwise kinship and IBS0 estimates (identity-by-state equals 0) between individuals that have been measured as part of the UKB study 2 and took the first 500 trios minimizing Mendel inconsistencies between parents and children genotypes. The 500 children constitute high-quality validation data since their haplotypes can be almost entirely resolved using Mendel inheritance logic. Fourth, we merged these 500 individuals with multiple random subsets of other UKB individuals in order to build 11 genotype data sets comprising between 500 and 400,000 individuals in total. Finally, we remove all individuals containing more than 5% missing genotypes in each data set. This gave us 11 data sets comprising 499, 995, 1997, 4973, 9939, 19,894, 49,752, 99,452, 198,894, 298,383, and 397,839 individuals for which genotype data is available at 18,477 variants.
Of note, all data management was done using bcftools v. 1. 4  31. Phasing runs on the UKB data sets. We phased the UKB data sets using SHA-PEIT4 and three other widely used phasing methods: SHAPET3 (https://jmarchini. org/shapeit3/), Eagle v2. 4 (https://github. com/poruloh/Eagle), and Beagle5 (beagle. 14May18. ff7. jar; https://faculty. washington. edu/browning/beagle/beagle. html). All runs were done on a RedHat server with Intel(R) Xeon(R) CPU E7-8870 v4 @ 2. 10 GHz (80 physical cores and 160 logical cores) and 3 Tb of RAM. Eagle v2. 4 and Beagle5 have been run with default parameters. SHAPEIT3 has been run using the options-states 200,-cluster-size 4000 and-early-stopping. All methods were run using 10 threads to speed up computations. The reported running times and memory usages were obtained using the GNU time v1.