dataesr/scientific-paragraphs-categorization huggingface dataset:
This makes it possible to evaluate the improvement brought by training on simulated data in addition to real data and to compare the performance on simulated vs. real test data for various techniques. In this article, we study the above mismatches in the context of the CHiME-3 dataset. Our analysis differs from the one of  Barker et al. (2016) , which focuses on the speaker characteristics and the noise characteristics of each environment and compares the achieved ASR performance with the intelligibility predicted using perceptual models. Instead, we focus on mismatched noise environments, different microphones, and simulated vs. real data. We provide a critical analysis of the CHiME-3 results in that light and perform a number of new experiments in order to separately assess the impact of these mismatches on speech enhancement and ASR performance.
Based on these results, we conclude that, except for a few techniques, these mismatches generally have little impact on the ASR performance compared to, e. g. , reducing the number of microphones. We introduce the CHiME-4 Speech Separation and Recognition Challenge, which revisits the CHiME-3 dataset and makes it more challenging by reducing the number of microphones. The structure of the paper is as follows. In Section 2, we briefly recall how the CHiME-3 dataset was recorded and simulated and we attempt to characterize these mismatches objectively from data. We measure the impact of data simulation mismatch in Section 3 and that of environment and microphone mismatch in Section 4. We introduce the CHiME-4 Challenge in Section 5. We conclude in Section 6.
Characterization of the mismatches The CHiME-3 dataset consists of real and simulated recordings of speech from the Wall Street Journal (WSJ0) corpus  (Garofalo et al. , 2007)  in everyday environments. Four environments are considered: bus (BUS), café (CAF), pedestrian area (PED), and street (STR). The real data consists of utterances spoken live by 12 US English talkers in these environments and recorded by a tablet equipped with an array of six sample-synchronized microphones: two microphones numbered 1 and 3 facing forward on the top left and right, one microphone numbered 2 facing backward on the top center, and three microphones numbered 4, 5, and 6 facing forward on the bottom left, center, and right. See  Barker et al. (2016, Fig. 1 ) for a diagram. The distances between microphones are indicated in Table  1.
In order to help estimate the ground truth, speech was also captured by a close-talking microphone approximately synchronized with the array. Note that this close-talking signal is not clean and it is not used as the ground truth directly: see Section 2. 3. 1 for the ground truth estimation procedure for real data. The simulated data is generated from clean speech utterances and continuous background noise recordings, as described in more detail in Section 2. 3. 2 below. The overall dataset involves a training set of 1600 real and 7138 simulated utterances, a development set of 1640 real and 1640 simulated utterances, and a test set of 1320 real and 1320 simulated utterances. The speakers in the training, development, and test sets are disjoint and they were recorded in different instances of each environment (e.
Simulation and ground truth estimation mismatches One last important mismatch between data concerns real vs. simulated data. As mentioned earlier, the CHiME-3 dataset contains real data, which were spoken live by 12 talkers in noisy environments, and simulated data, which were constructed by mixing clean speech recordings with noise backgrounds in a way that tries to match the properties of real data. The ground truth speech and noise signals underlying real data are not readily available and must be estimated by means of signal processing. Indeed, the close-talking speech signal is not clean enough for this purpose: as can be seen in Fig. 3 , top, it includes background noise (e. g. , between 0 and 0. 5 s), breathing noises (e. g. , between 9. 4 and 10. 1 s), "pop" noises due to plosives (e.
Ground truth estimation for real data The speech and noise signals underlying every real recording r are estimated as follows. Let us denote by x ri (t) and c r (t) the signals recorded by the i-th array microphone and the close-talking microphone, respectively. The signals are represented in the complex-valued short-time Fourier transform (STFT) domain by their coefficients X ri (n, f ) and C r (n, f ) in time frame n and frequency bin f. The STFT is computed using half-overlapping sine windows of 256 samples (16 ms). The time frames are partitioned into K r variable-length, half-overlapping, sine-windowed blocks indexed by k 2{ 1,. ,K r } such that the amount of speech is similar in each block. To do so, the number of significant STFT bins (above the median STFT magnitude) in the close-talking signal is accumulated over time and the center frame n k of the k-th block is chosen as the k-1/2 Kr -th quantile of this distribution.
Impact of data simulation mismatch After having characterized and quantified the various mismatches, we now analyze their impact on ASR performance. This section concerns the impact of data simulation mismatch on the main processing blocks typically involved in a robust ASR system, namely speech enhancement, feature extraction, and ASR backend. We report several pieces of evidence stemming both from a critical analysis of the results of the systems submitted to CHiME-3 and from a new experiment. Our analysis can differ depending whether the considered processing techniques rely on training or not. In the former case, results are reported both on development and test data since these techniques may overfit the development data (which is typically used for validation during training). In the latter case, similar behavior is typically observed on development and test data and we report the results on development data only, when those on test data are unavailable.
, 2001; Araki et al. , 2003). These beamformers aim to minimize the noise power under the constraint of a unit response in the direction of the speaker. This constraint is valid for the CHiME-3 simulated data, which are simulated using a pure delay filter, but it does not hold anymore on real data. Indeed, early reflections (and to a lesser extent reverberation) modify the apparent speaker direction at each frequency, which results Table  4 : WER (%) achieved by beamforming and spatial post-filtering applied on all channels except ch2 using the GMM backend retrained on enhanced real and simulated data  (Prudnikov et al. , 2015). in undesired cancellation of the target. Fixed beamformers such as DS and its variant known as BeamformIt  (Anguera et al. , 2007)  which was used in many challenge submissions do not suffer from this issue due to the fact that their spatial response decays slowly in the neighborhood of the estimated speaker direction.
More work on ground truth estimation is required to close this gap and benefit from simulated training data. In addition, training on real data now leads to a performance decrease on simulated data, while  Sivasankaran et al. (2015)  found it to consistently improve performance on both real and simulated data. Along with the recent results of  Nugraha et al. (2016b)  on another dataset, this suggests that, although weighted EM made little difference for spectral models other than DNN  (Liutkus et al. , 2015) , weighted EM outperforms exact EM for the estimation of multichannel statistics from DNN outputs. More work on the estimation of multichannel statistics from DNN outputs is therefore also required. 7  illustrates the performance of two robust features, namely damped oscillator coefficients (DOC)  (Mitra et al.
Multichannel DNN-based separation Our first experiment deals with the impact of environment mismatch on training-based enhancement techniques, as measured by the resulting ASR performance. We considered the DNN-based multichannel source separation technique of  Nugraha et al. (2016a). As explained in Section 3. 2. 4, this technique is trained on real data only. We study three different training setups: • single-condition: train on 1 environment (e. g. , BUS), test on the same one or another one, • few-condition: train on 3 environments (e. g. , all but BUS), test on one of them or on the remaining one, • many-condition: train on all 4 environments. The first setup is a classical single-condition matched/mismatched setup. The second setup aims to assess the number of environments required for multicondition training, as well as its performance in a test environment which was not seen during training.
The main challenge in this track is to mitigate the performance degradation from 6 to 2 microphones. • The 1ch Track only uses a single channel. Again, the microphone is selected for each utterance in such a way that microphone failures do not arise. This track is similar to conventional ASR tasks based on singlechannel processing. Important techniques used in this context could be single-channel enhancement, data simulation, and acoustic modeling, and we expect that participants in the ASR community mainly deal with this track. For the training data, we do not restrict microphone usage unlike for the test data. We use the same regulations as in CHiME-3 6  for all three tracks. In short, participants are allowed to use the speaker labels in all datasets and the environment labels in the training set.
They can also exploit the embedded test data in the limit of the 5 s preceding each utterance. They are not allowed, however, to use external datasets or to augment the provided dataset unless each speech signal is mixed with the same noise signal as in the original simulated training set (i. e. , only the impulse responses can be modified, not the noise instance). We provide an official baseline using Kaldi  (Povey et al. , 2011)  for each track from the official challenge package and the Kaldi repository. We consider three baseline models: a GMM-HMM with a 3-gram language model, a DNN-HMM with a 3-gram language model and sMBR, and a DNN-HMM with a 5-gram language model and RNN-LM based rescoring. The unprocessed data of channel 5 (ch5) is used to train all systems.
Perspectives In this paper, we provided an exhaustive assessment of the impact of acoustic mismatches between training and test data on the performance of robust ASR systems. We studied this issue in the context of the CHiME-3 dataset and considered three possible sources of mismatch: data simulation, different noise environments, and different microphone setups. We showed that, with the notable exception of MVDR beamforming, most of the methods implemented in robust ASR systems result in comparable relative WER improvements on real and simulated data and benefit from training on large amounts of simulated data. Also, we found that training on different noise environments and different microphones barely affects the ASR performance, especially when several environments are present in the training data.
Second, we also found the procedure used to estimate the ground truth clean speech signal for real and simulated data to have an impact on the performance of training-based enhancement techniques beyond the mismatches between the noisy signals themselves. Improved signal processing techniques are therefore required for this task too. Third, the estimation of multichannel statistics from DNN-based time-frequency masks (see Section 3. 2. 4) and the use of bottleneck features (see Section 3. 3. 2) or discriminative system fusion (see Section 3. 4. 3) yielded unexplained differences between real and simulated data that call for additional investigation. Finally, although the CHiME-4 challenge will push research further in the direction of mismatched microphone setups, the extent to which this can be evaluated is limited by the fact that the microphones used in the experimental setup are all from the same brand and type and are limited in number.
• In 1991, Harold B. Munster produced a formal specification of the MAA in LOTOS  [15]. The MAA was described using only the data part of LOTOS: the behavioural part of LOTOS, which serves to describe concurrent processes, was not used. The LOTOS specification, which made intensive use of the predefined LOTOS data-type libraries, was mainly declarative but not executable, as all facilities of LOTOS abstract data types were used in an unconstrained way. For instance, some equations could be rephrased as: "given x, the result is y such that x = f (y)", which required to invert function f in order to compute y. • In 1992, Hubert Garavel and Philippe Turlier, taking the aforementioned LOTOS specification as a starting point, gradually transformed it by successive modifications. Their goal was to obtain an executable specification that could be processed by the CAESAR.
g. , 32-bit vs 64-bit words or little-vs big-endian ordering. • Our model is executable. From a theoretical point of view, this was enabled by the aforementioned shift from general LOTOS abstract data types to term rewrite systems, which are less declarative and more operational. From a practical point of view, this shift was not sufficient, as the MAA intensively manipulates block values (i. e. , 32-bit numbers), which cannot be reasonably implemented in the Peano style (the execution stack quickly overflows when these numbers are represented using the zero and succ constructors). To overcome this issue, we chose to represent blocks in binary form, as words of four octets. So doing, the logical operations on blocks (AND, OR, XOR, and CYC) are easy to define using bitwise and octetwise manipulations.
For staining of total protein, arrays were incubated 15 min in 7% acetic acid and 10% methanol, rinsed twice in water, incubated 10 min in Sypro Ruby (Invitrogen) and rinsed again. The processed slides were dried by centrifugation and scanned using a GenePix 4000B microarray scanner (Molecular Devices). Spot intensity was determined with MicroVigene software (VigeneTech Inc). All primary antibodies used in RPPA have been previously tested by Western Blotting to assess their specificity for the protein of interest. Samples with low overall signal or with aberrant dilution curves, which are often indicative of protein degradation, were discarded during quality control. For each antibody, the median signal intensity was at least three times higher than the background array without primary antibody.
Statistical Analysis Response to anti-EGFR treatment was determined using RECIST criteria  (Therasse et al. 2000). Continuous variables are described as mean and standard deviations and qualitative data are presented as a number and percentage of sample size. The association between clinical variables and proteins was determined by Wilcoxon or Kruskal-Wallis tests. Association between discrete variables was tested using Chi-square or Fisher's exact test. Hierarchical clustering was performed using Ward metrics and Pearson correlation and represented as a heatmap. Protein expression was divided in high expression and low expression with the cut-off at the median expression level. The RECIST Criteria were restricted to a dichotomous output, where complete or partial response were considered as a response and stabilization or progression as a non-response.
Patient characteristics and response to anti-EGFR therapy according to primary tumor location Final RPPA data was obtained for 34 CRCs and 86 antibodies. The antibodies were selected according to signaling pathways that have been put forward as being involved in EGFR signaling and anti-EGFR resistance (various RTKs, PI3K/Akt pathway, MAPK pathways) or more generally in CRC carcinogenesis (Wnt/Notch) and chemotherapy response (apoptosis, cell proliferation, DNA repair, angiogenesis). Clinical characteristics are summarized in table 1 and antibodies are listed in Supplementary Table  1. Median follow-up of these 34 patients was 15. 5 months from the beginning of anti-EGFR treatment (range 3 -45. 6 months). All patients received anti-EGFR as second or third line treatment, except for four patients who received anti-EGFR in combination with chemotherapy as a first line treatment but did not respond.
E545K, one patient p. E542K and one patient p. N1044K) and one other patient a BRAF V600E mutation. Biopsies were obtained before initiation of the anti-EGFR therapy. We analyzed if the response to treatment, measured according to RECIST criteria, was associated with any of the clinical parameters. The RECIST score was not significantly different according to sex (men versus female), age (<50 versus ≥50) or the molecule of anti-EGFR treatment (cetuximab versus panitumumab) that was administered. As expected, the absence of objective response was associated to worse overall survival (data not shown). Interestingly, and in accordance with recent findings  (Moretto et al. 2016 , Tejpar et al. 2016 , Holch et al. 2017 ) the response to anti-EGFR treatment was better in leftsided CRC (descending colon and rectum), where 46% of patients show partial response, than in rightsided CRC (ascending colon), where none of the patients shows partial response (p= 0.
However, the sample was small (11 in the at-risk group). Consequently, we intended to extend these findings in another community sample of young adolescents with psychoticlike experiences. Of note, considering that psychotic-like experiences are, for most individuals, transient and not persistent  (1) , it is crucial to understand to what extent these early neural abnormalities relate to a subsequent psychosis vulnerability in terms of clinically validated symptoms. The primary aim of the present exploratory study was to identify brain correlates of psychotic-like experiences in youth prior to exposure to regular substance use on fMRI measures of emotion processing, inhibitory control and reward anticipation using data from the IMAGEN study, in which two sites, London and Dublin, assessed these preclinical experiences to participants when they were 14.
METHODS Participants 2257 14-year old adolescents were recruited through high schools in the large European multicenter IMAGEN study from 8 sites across the United Kingdom, Ireland, France and Germany. Parents and adolescents gave written informed consent to the study procedures. All procedures were approved by each local institutional ethics committee. A detailed description of the study recruitment and assessment procedure, exclusion criteria, data storage and safety, as well as imaging acquisition protocol may be found elsewhere  (24).
Acknowledgments Dr. Banaschewski  has served as an advisor or consultant to  Bristol-Myers Squibb, Desitin Arzneimittel ,  Eli Lilly ,  Medice ,  Novartis ,  Pfizer ,  Shire ,  UCB , and  Vifor Pharma ; he has received conference attendance support, conference support, or speaking fees from  Eli Lilly ,  Janssen McNeil ,  Medice ,  Novartis ,  Shire , and  UCB ; and he is involved in clinical trials conducted by  Eli Lilly ,  Novartis , and  Shire ; the present work is unrelated to these relationships. Dr. Gallinat  has received research funding from the  German Federal Ministry of Education and Research ,  AstraZeneca ,  Eli Lilly ,  Janssen-Cilag , and  Bristol-Myers Squibb ; he has received speaking fees from  AstraZeneca ,  Janssen-Cilag , and  Bristol-Myers Squibb. Dr Barker  has received honoraria from  General Electric  for teaching on scanner programming courses.
Further support was provided by grants from:  ANR  (project  AF12-NEUR0008-01 -WM2NA , and  ANR-12-SAMA-0004 ), the  Fondation de France , the  Fondation pour la Recherche Médicale , the  Mission Interministérielle de Lutte-contre-les-Drogues-et-les-Conduites-Addictives (MILDECA) , the  Assistance-Publique-Hôpitaux-de-Paris  and  INSERM  (interface grant),  Paris Sud University  IDEX 2012; the  National Institutes of Health, U. S. A. (Axon, Testosterone and Mental Health during Adolescence ;  RO1 MH085772-01A1 ), and by  NIH Consortium  grant  U54 EB020403 , supported by a cross- NIH  alliance that funds Big Data to Knowledge Centres of Excellence.
375 U of Taq DNA polymerase (Promega), and 40 ng of DNA. The amplification of the DNA was performed in a 2720 thermal cycler (Applied Biosystems, Thermo Fisher Scientific, Singapore). For the microsatellite loci ABRII-MP, pg and Pom, the cycling conditions of the two-step PCR were as follows: first step: 94 °C for 5 min, five cycles of 45 s at 94 °C, 30 s at 57 °C, lowered by 1 °C in each cycle, and 30 s at 72 °C; and second step: 30 cycles of 30 s at 94 °C, 30 s at 52 °C, and 90 s at 72 °C; and a 10-minute extension step at 72 °C. The amplification protocol for the microsatellite locus PGCT was performed according to Schuelke  (21). Separation of the amplified microsatellites was performed on a 3130 Genetic Analyzer (Applied Biosystems, Hitachi High-Technologies Corporation, Tokyo, Japan), using Ge-neScanTM-500 LIZ ® (Applied Biosystems, Woolston, UK) for the size standards, and the data were analysed using the Gene Mapper v.
Genetic data analysis For a total of 35 pomegranate samples, the following genetic parameters were calculated from the data for the 12 microsatellite loci: the observed and expected heterozygosity, the Hardy-Weinberg equilibrium test, the effective number of alleles, the probability of identity, the frequency of null alleles per locus, and the polymorphic information content (PIC). The genetic parameters of observed and expected heterozygosity and PIC were calculated using the CER-VUS v. 3. 0. 7 program  (22) , with the IDENTITY v. 1. 0 program (23) used to calculate the probabilities of identity. The frequencies of null alleles per locus were estimated using the FreeNA v. 1. 2 software  (24). GENEPOP v. 4. 2  (25, 26)  was used to test the genotypic frequencies for conformance to the Hardy-Weinberg equilibrium test expectations, to test the loci for linkage disequilibrium, and to estimate the significance of the genotypic differentiation between the population pairs.
Probability tests for loci with less than five alleles were based on the complete enumeration method, as described by Louis and Dempster  (27) , while for loci with five or more alleles, the Markov chain algorithm was used  (25, 28) , with 1000 dememorisation steps, 100 batches, and 1000 iterations per batch. The genetic distances between these 35 pomegranate samples were calculated using Jaccard's coefficient of similarity, and the dendrogram was constructed using the unweighted pair-group method. The correlation coefficient between the similarity matrix and the cophenetic value matrix was computed to test the goodness-of-fit of the cluster analysis. All calculations were performed using the NTSYS v. 2. 02 package  (29).
Jaccard's similarity coefficient was used for the calculation of the genetic distances among pairwise combinations in the set of 35 pomegranate accessions. The dendrogram shown in Fig. 1  was constructed from the genetic similarity data, and the clusters were tested for associations. The cophenetic correlation coefficient was 0. 844, which indicated a good fit of the original data to the clustering. The accessions were clustered into three main groups. The first cluster consisted of eight local accessions of unknown identity plus the varieties 'Rumenjaš slatki', 'Glavaš', 'Slatki rani', 'Dente di Cavallo' and 'Nar'. The second cluster comprised varieties 'Medun', 'Irački ljutunac', 'Barski sladun', 'Barski', 'Domača' and 'Wonderful'. The third cluster included the Spanish varieties 'Valenciana' and 'Mollar de Elche', as well as three local accessions and the wild pomegranate Pg21.