dataesr/scientific-paragraphs-categorization huggingface dataset:
Experimental setup Data and pre-processing. We experiment with the IWSLT 2014 bilingual dataset  (Cettolo et al. , 2014) , which contains transcripts of TED talks aligned at sentence level, and translate between German (De) and English (En) in both directions. Following the setup of  (Edunov et al. , 2018) , sentences longer than 175 tokens and pairs with length ratio exceeding 1. 5 were removed from the original data. There are 160+7K training sentence pairs, 7K of which are separated and used for validation/development. We report results on a test set of 6,578 pairs obtained by concatenating dev2010 and tst2010-2013. We tokenized and lowercased all data using the standard scripts from the Moses toolkit  (Koehn et al. , 2007). For open-vocabulary translation, we segment sequences using joint byte pair encoding  (Sennrich et al.
(128 in each direction ). The decoder is a single layer LSTM with similar input size and a hidden size of 256, the target input embeddings are also used in the pre-softmax projection. For regularization, we apply a dropout of rate 0. 2 to the inputs of both encoder and decoder and to the output of the decoder prior to softmax. As in  (Bahdanau et al. , 2015) , we refer to this model as RNNsearch. The ConvS2S model we trained has embeddings of dimension 256, a 16-layers encoder and 12-layers decoder. Each convolution uses 3×1 filters and is followed by a gated linear unit with a total of 2 × 256 channels. Residual connections link the input of a convolutional block to its output. We first trained the default architecture for this dataset as suggested in FairSeq  (Gehring et al. , 2017b) , which has only 4 layers in the encoder and 3 in the decoder, but achieved better results with the deeper version described above.
Nevertheless, the max-pooling operator in our model generates implicit sentence alignments that are qualitatively similar to the ones generated by attention mechanisms. We evaluate our model on the IWSLT'14 dataset, translation German to English and vice-versa. We obtain excellent BLEU scores that compare favorably with the state of the art, while using a conceptually simpler model with fewer parameters. We hope that our alternative joint source-target encoding sparks interest in other alternatives to the encoder-decoder model. In the future, we plan to explore hybrid approaches in which the input to our joint encoding model is not provided by tokenembedding vectors, but the output of 1D source and target embedding networks, e. g. (bi-)LSTM or 1D convolutional. We also want to explore how our model can be used to translate across multiple language pairs.
I. INTRODUCTION AutoDock is a protein-ligand docking tool which has now been distributed to more than 29000 users around the world  [1]. The application for this software arises from problems in computer-aided drug design. The ideal procedure would optimize the interaction energy between the substrate and the target protein. It employs a set of meta-heuristic and local search optimization methods to meet this demand. The Autodock puts protein-ligand docking at the disposal of nonexpert users how often do not know how to choose among the dozens of configurations for such algorithms. This motivated us to bring the benefits of hyperparameter tuning to users of AutoDock. The global optimization problem arises in many real-world applications as well as molecular docking. A standard continuous optimization problem seeks a parameter vector x * to minimize an objective function f (x) : R D → R, i.
, k} The MO-SRS divides the objective space into hypercubes and assigns a fitness value to each hypercube based on the number of Pareto-optimal solutions that lie in it. Then, it employs the roulette-wheel to find the superior hypercube. A randomly selected solution from that hypercube is determined to be the best solution x best. All the N candidates are evaluated by the previously trained surrogates f1 and f2. Again, we select the best solution according to the above mentioned strategy. This best solution is then used to update our surrogate models. All the aforementioned steps continue until some stopping criteria are met. To sum up, MO-SRS approach is illustrated in Fig. 3. The MO-SRS will be available soon on https://github. com/ML-MHs/Auto-Autodock. Algorithm 1 MO-SRS Use LHS to initialize a population X ← {x 1 , • • • , x n0 } n ← n 0 y (1) i = f (1) (x i ) : i ← 1 • • • n y (2) i = f (2) (x i ) : i ← 1 • • • n repeat Find Pareto-optimal P using y (1) i and y (2) i Set x best using P and described hypercube technique Fit surrogate f (1) by B n ← x i , y : i ← 1 • • • n Fit surrogate f (2) by C n ← x i , y (1) i : i ← 1 • • • n Generate new solutions σ j : j ← 1 • • • N as in (8) (2) i Apply the bound constraints on each solution σ j ŷ(1) j = f (1) (σ j ) : j ← 1 • • • N ŷ(2) j = f (2) (σ j ) : j ← 1 • • • N Find Pareto-optimal P using ŷ(1) and ŷ(2) Set x n+1 using P and described hypercube technique y (1) n+1 = f (1) (x n+1 ) y (2) n+1 = f (2) (x n+1 ) B n+1 ← B n ∪ (x n+1 , y (1) n+1 ) C n+1 ← C n ∪ (x n+1 , y (2) n+1 ) n ← n + 1 until stopping criteria are met Fig.
V. EXPERIMENTAL RESULTS In this section, the performance of MO-SRS hyperparameter tuning method on Autodock task is investigated. We considered two scenarios from Autodock documentations which should be minimized: 1)1DWD and 2) HSG1. All simulations are performed using Matlab. We adopted the same implementation and parameter configuration for the SRS as suggested in  [20]. The Cubic function is used as the kernel in the RBF. We applied the MO-SRS to tune the hyperparameters of Autodock for the genetic algorithm (GA)  [21] , simulated annealing (SA)  [22] , a local search (LS) called as pseudo-Solis-Wet  [1]  and a hybrid method (HB) which combines the GA and LS. The GA is a well-known evolutionary optimization algorithm which is highly sensitive ot the initial value of its parameters. The performance of the GA is dependent on the structure of the problem at hand and do not scale well with complexity.
The linkage disequilibrium (LD) blocks delineating these cancer risk loci are distinct from the 8q24. 21 BCP-ALL association signal suggesting this risk locus is unique to BCP-ALL (pairwise LD metrics r 2 < 0. 2; Supplementary Table  5 ). rs17481869 maps to an intergenic region at 2q22. 3 with no candidate gene nearby (Fig. 2 ). Relationship between SNP genotype and patient outcome. We examined the relationship between SNP genotype and patient outcome using data from UK GWAS II and German GWAS. Neither rs28665337 or rs17481869 showed a consistent association with either event-free survival (EFS) or risk of relapse, even when stratified by ETV6-RUNX1 status (Supplementary Table  6 ). Functional annotation of risk loci. To gain insight into the biological basis underlying the association signals at these as well as previously identified risk loci, we examined the epigenetic landscape of BCP-ALL risk loci genome wide.
For each risk locus we evaluated profiles of three histone marks of active chromatin (H3K27ac, H3K4me1, and H3k4me3) using ChIP-seq data of 14 cell types from ENCODE, including lymphoblastoid cell line (GM12878), and multiple ALL and acute myeloid leukemia (AML) samples from the Blue-Print Epigenome database (Supplementary Fig. 4 , Supplementary Table  7 )  15, 16. Since the strongest associated GWAS SNP may not represent the causal variant, we examined signals across an interval spanning all variants in LD with the most strongly associated SNP at each risk loci (r 2 > 0. 8 and D′ > 0. 8 based on the 1000 Genomes EUR reference panel). The analysis across all risk loci combined revealed that risk SNPs are enriched for markers of open chromatin and that enrichment is highest in ALL cells (Supplementary Fig.
Discussion The evidence for the two risk loci we report has been based on a meta-analysis of three independent GWAS data sets. While the combined association P-values for each risk locus is genome-wide significant with each series providing support for association we acknowledge that we did not provide additional replication. For rare cancers such as childhood ALL, ascertaining case series which are appropriately ethnically matched and are sufficiently powered to provide independent replication is inherently problematic. Moreover as exemplified by the 10q21 and 10p14 risk loci, associations can be highly subtype-specific which adds to the difficulty in obtaining appropriate replication series. Accepting such caveats our analysis provides evidence for the existence of two additional risk loci for childhood BCP-ALL at 2q22.
illumina. com); control genotypes were obtained from 2699 individuals from the 1958 British Birth Cohort (Hap1. 2M-Duo Custom array data) and 2501 from the UK Blood Service produced by the Wellcome Trust Case Control Consortium 2 (http://www. wtccc. org. uk/; 51% male)  42. The German GWAS comprised 1155 cases (620 male; mean age at diagnosis 6 years) from the Berlin-Frankfurt-Münster (BFM) trials (1993-2004) genotyped using Illumina Human OmniExpress-12v1. 0 arrays (834 samples post QC). Control data was generated on 2132 (50% male) healthy individuals from the Heinz Nixdorf Recall study; 704 individuals genotyped using Illumina-HumanOmni1-Quad_v1 and 1428 individuals genotyped on Illumina-HumanOmniExpress-12v1. 0 platform. In total 2024 controls remained post QC in the German cohort. New GWAS samples.
Post QC we obtained genotype data for 784 cases (365 female; mean age at diagnosis 5. 3 years). Controls consisted of: (1) 2976 cancer-free, men ascertained by the PRACTICAL Consortium; (2) 4446 cancer-free women from the UK through the Breast Cancer Association Consortium. All controls were genotyped on Infinium OncoArray-500K BeadChip arrays. Statistic and bioinformatics analysis of GWAS data sets. Analyses and/or data management were undertaken using R v3. 2. 3 (R Core Team 2013; http://www. Rproject. org/)  72 , PLINK v1. 9  43  , and SNPTEST v2. 5. 2 software  44. GenomeStudio software (Illumina, San Diego; Available at: http://www. illumina. com) was used to extract genotypes from raw data. QC of all GWAS data sets was performed as suggested by Anderson et al  45. PLINK v1. 9  43  was used for conducting the sample and SNP QC steps.
The imputation was based on a reference panel from the T1DGC consisting of genotype data from 5225 individuals of European descent with genotyping data of 8961 common SNPs and indel polymorphisms across the HLA region, and four digit genotyping data of the HLA class I and II molecules. This reference panel has been used previously and showed high imputation quality for the HLA regions in other studies  [27] [28] [29]. Individual GWAS studies were imputed at the 6p21 region and meta-analyzed to identify significant HLA risk alleles. A significance threshold of 5. 7 × 10 -6 was set after Bonferroni correction as the number of SNPs tested was 8654. Sanger sequencing. To assess the accuracy of imputed genotypes, a random series of samples was Sanger sequenced using BigDye ® Terminator v3. 1 Cycle Sequencing Kit (Life Technologies, CA, USA) and analyzed using a ABI 3700xl sequencer (Applied Biosystems, CA, USA).
Oligonucleotide primer sequences are provided in Supplementary Table  12. Chromatin mark enrichment analysis. To assess for an over-representation of markers for open chromatin the variant set enrichment method of Cowper-Sal Lari et al. was adapted  56. For each risk locus, SNPs in LD were defined (i. e. , R 2 > 0. 8 and D′ > 0. 8), and termed associated variant set (AVS). Transcription factor ChIP-Seq broad peak data were obtained from the ENCODE project for 14 cell lines for H3K27ac, H3k4me1, and H3K4me3 chromatin signatures. ChIP-Seq broad peak data for three AML and six childhood ALL cell types were obtained from the Blue-Print Epigenome database (www. blueprint-epigenome. eu)  15. For each mark, overlap of SNPs in the AVS and the ChIP peak were derived, generating a mapping score. The null hypothesis was tested by scoring randomly chosen SNPs with the same LD structure at the risk-associated SNPs.
After 10,000 iterations, approximate P-values were calculated as the proportion of permutations where null mapping score was at least equal to the AVS mapping score. Enrichment was calculated normalizing scores to the median of the null model. Hi-C analysis. Hi-C analysis was conducted using the HUGIn browser  57  , which is based on the analysis by Schmitt et al  58. Specifically we analyzed Hi-C data generated on the H1 ES Cells and GM12878 lymphoblastoid cell lines originally described in Dixon et al. 59  and Schmitt et al. 58  , respectively. Plotted topologically associating domains boundaries were obtained from the insulating score method at 40 kb bin resolution  57. We searched for significant interactions (P-values generated using "Fit-Hi-C"  18  ) between bins overlapping the currently identified ALL risk loci with target genes (e.
g. , "virtual 4C"). Functional annotation. SNPs in LD (r 2 > 0. 8) with the top SNPs from each risk loci were assessed for histone marks in relevant tissue, proteins bound and location were annotated using HaploReg  17  (Supplementary Data 1). eQTL analysis was performed by testing each sentinel SNP with genes 1MB upstream and downstream using the whole blood tissue data available from GTEx portal v6p  60  and Blood eQTL browser  61  (Supplementary Data 1). Methylation quantitative trait loci (mQTL) for all known BCP-ALL risk loci where assessed using the mQTL Database (www. mqtldb. org), which shows the presence of significant methylated CpG sites at various stages of life as described by Gaunt et al  62. SMR analysis. SMR analysis was conducted as per Zhu et al. (at http:// cnsgenomics. com/software/smr/index.
html)  63. Publicly available eQTL data was extracted from the whole blood eQTL, Muther consortia, and GTEx16 v6p release portals  60, 61, 64. GWAS summary statistics files were generated from the metaanalysis of UK GWAS I, UK GWAS II, and German GWAS data sets. Reference files were generated by merging 1000 genomes phase 3 and UK10K (ALSPAC and TwinsUK) vcfs. Summary eQTL files for the GTEx samples were generated from downloaded v6p "all_SNPgene_pairs" files. BESD files were generated from downloaded SNP-gene eQTL data, which were converted into a query flat file format as mentioned in the SMR online guide (http://cnsgenomics. com/software/ smr) and then using the -make-besd command to make binary versions of the files. Only probes with eQTL P < 5. 0 × 10 -8 were considered in the SMR analysis.
2  4. The discriminatory value of risk SNPs was examined by determining the AUC for the ROC curve. GCTA to estimate heritability. Since artefactual differences in allele frequencies between cases and controls have the potential to bias estimation genetic variation, additional QC measures were imposed on the GWAS data sets which have been advocated by Lee et al  73. Typed SNPs were excluded if they had a MAF < 0. 01 or a HWE test with P < 0. 05. SNPs were also excluded if a differential missingness test between cases and controls was P < 0. 05. In addition, individuals were excluded if having a relatedness score of >0. 05. Filtering resulted in the 260,127 SNPs in the UK GWAS I and 355,899 SNPs in UK GWAS II data sets, respectively. GCTA (http://cnsgenomics. com/software/gcta/) was employed to estimate the fraction of the phenotypic variance attributed by SNPs given a prevalence of 0.
0005 for ALL  30. Data availability. The UK GWAS I control set comprised 2699 individuals in the 1958 British Birth Cohort (Hap1. 2M-Duo Custom array data) and 2501 individuals from the UK Blood Service obtained from the publicly accessible data generated by the Wellcome Trust Case Control Consortium 2 (http://www. wtccc. org. uk/; WTCCC2:EGAD00000000022, EGAD00000000024). The reference panels used in the imputation can be obtained from the 1000 genomes phased haplotypes (n = 1092) from the Phase I integrated variant set release (ftp://ftp. 1000genomes. ebi. ac. uk/vol1/ftp/release/20110521/) and the UK10K (n = 3781; EGAS00001000090, EGAD00001000195, EGAS00001000108; www. uk10k. org) sequenced data sets. eQTL data for various functional analyses were obtained from the MuTHer studies (genome-wide expression profiled samples with genotype array data and methylation data; E-TABM-1140), Blood eQTL (whole-genome gene exression array data sets with RNA sequencing and genotyping data: E-TABM-1036, E-MTAB-945, E-MTAB-1708; http://www.
Genome Sequencing and Assembly We sequenced four individuals of the unknown species using the Illumina platform at the CNAG (Barcelona). To make a de novo genome assembly of this species one of the individuals was deeply sequenced to ~112X. The other three individuals were sequenced at an average coverage of ~29X. All reads were paired-end 126 bp long (Table  S2 ). The genome assembly of the more deeply sequenced An. fontenillei individual was performed at the Bioinformatics Unit, CRG (Barcelona) (Table  S2 , S3A). Reads were trimmed and filtered using Skewer version 0. 2. 2 [73] to remove the adapter sequence and trimming the low quality part. A FastQC analysis was performed to check the quality of the trimmed reads. We looked at the presence of contaminants in a Kraken database, which includes complete bacterial, archaeal, and viral genomes in RefSeq [74].
We only found an enterobacteria phage phiX as contaminant (Table  S3B ). Then, we assembled the trimmed reads by using Platanus software version 1. 2. 4 [75]  producing contigs and scaffolds using the paired-end information. To join the contigs within the same scaffolds, stretches of N need to be added. To fill those gaps we used Platanus gap_close function using the original reads (Table  S3A ). At this point, improve the scaffolding of the assembled genome by using the proteins described for AgamP4 reference in VectorBase (www. vectorbase. org). We used Blat [76] to map the proteins to the assembled scaffolds and reorder and join scaffolds accordingly with PEP_scaffolder [77]. We made another round of gap filling, and due to format incompatibilities, this time we used GapCloser tool from the SOAPdenovo package [78] (Table  S3A ).
To evaluate the quality of our assembly, we scanned for the presence of conserved genes among the diptera order by using BUSCO software [79]. We used 2,799 gene models conserved among diptera, and classify those genes as: i) completely found in a single sequence, ii) fragmented in different sequences, or iii) completely missing. Most of the BUSCO genes, 96%, were completely found in a single sequence (Table  S3C ). We finally performed a polishing step by removing the scaffolds mapping to previously found contaminants (Table  S3A ).
Phylogenetic analysis To make the genome-wide phylogenetic tree by window analysis, we took advantage of the available multiple alignment file (MAF) for six species of the An. gambiae complex including two outgroup species: An. christyi and An. epiroticus  [10]. Briefly, we used the alignment formed by whole genome sequences from population samples of multiple individuals of An. gambiae, An. coluzzii, An. merus, An. melas, An. quadriannulatus and An. arabiensis. The An. gambiae PEST v3 (AgamP3) reference genome obtained from VectorBase (www. vectorbase. org) was also included. Fontaine et al,. [10]  made a whole genome alignment using ROAST [80] that represents approximately 40% of the euchromatic genome. We download the resulting MAF based on field-collected samples from http://datadryad.
org/resource/doi:10. 5061/dryad. f4114 [10]. We then added to this MAF our An. fontenillei assembly, and the highest coverage An. bwambae genome sequences available (see below). Anopheles fontenillei. We first generated a database with the scaffolds of the An. fontenillei assembly. Then we run blastn for each region in the MAF using AgamP3 as query sequence against the An. fontenillei scaffold database. We then repeat this blastn analysis using other species of the MAF regions as query; An. arabiensis, An. quadriannulatus, An. melas and An. merus (Text S2. 1. 1 -Text S2. 1. 4). We did not repeat the analysis using An. coluzzii and An. gambiae as queries, due to its similarity to the reference genome AgamP3, or the two outgroup species, that are too divergent. We then selected the MAF regions for which we get one unique hit in any of the species, which represents the 63.
6. 2)  [10]. We used the closer related species, An. christyi, as an outgroup because Fontaine et al. [10]  already showed that the choice of the outgroup did not substantially alter the results. We excluded the windows with less than 10% of informative base pairs (e. g. < 5,000 bp) (following  [10] ). The different topologies obtained were sorted, counted, and analyzed using ad-hoc perl scripts (Text S2. 6. 3).
Acknowledgements We are grateful to  P. Nosil  and  G. Lanzaro  for comments on the manuscript. We thank the  "Agence Nationale de la Preservation de la Nature" (ANPN) , the  "Station d'Etudes des Gorilles et Chimpanzes" (SEGC)  and the  "Centre National de la Recherche Scientifique et Technologique of Gabon" (CENAREST)  that authorized this study and facilitated the access to the national parks of  La Lopé. We specially thank  Vincenzo Petrarca  for its help interpreting chromosome polymorphism. Funding was provided by the  "Institut de Recherche pour le Developpément" , the  "Agence Universitaire de la Francophonie " (grant:  OKANDA ), the  "Centre National de la Recherche Scientifique" (CNRS)  and the  "Consejo Superior de Investigaciones.
6 1. 3 4. 9 GO:0019531, GO:0015106, GO:0015116, GO:0008271, GO:0051453, GO:0005254, GO:1902476, GO:0005887 GO:0004252 --GO:0004499, GO:0050661, GO:0055114, GO:0050660, GO:0004497 GO:0050911, GO:0004984, GO:0005549, GO:0005886 GO:0035069, GO:0097200, GO:0097194, GO:0005737 IPR001902, IPR011547, IPR002645 IPR001254, IPR018114, IPR009003 IPR018181, IPR013126 -IPR000960, IPR020946, IPR023753 IPR004117 IPR002138, IPR001309, IPR015917. CC-BY-The copyright holder for this preprint (which was this version posted November 3, 2018.
General procedures All stimuli were presented at 65 dBA in sound field in an anechoic chamber via single loudspeaker (Tannoy Precision 8D; Tannoy Ltd. , North Lanarkshire, United Kingdom) connected to a computer soundcard (Asus Virtuoso; ASUSTeK Computer Inc. , Fremont, CA). Participants were seated directly facing the loudspeaker, positioned 1 m away. All stimuli were processed and presented via custom software (iStar: www. tigerspeech. com/istar; 21 AngelSound: angelsound. tigerspeech. com  22 ). To familiarize participants with the stimuli and procedures for each listening task, participants were given a brief training session with auditory and visual feedback using stimuli different from those used for testing; during testing, no feedback was provided.
5  Figure  2  shows boxplots of MCI and vocal emotion identification scores for adolescent and adult musicians and non-musicians. Mean performance was generally better for adults than adolescents both with unprocessed or degraded signals. Because the distribution of data was sometimes not normal, non-parametric Mann-Whitney tests were used to compare age effects in musicians and non-musicians; the results are shown in Table  2. Significant age effects were observed for MCI performance in musicians listening to unprocessed or degraded signals; there were no significant age effects for MCI performance for non-musicians. Significant age effects were observed for vocal emotion identification in musicians listening to unprocessed signals, and in non-musicians listening to unprocessed or degraded signals.