dataesr/scientific-paragraphs-categorization huggingface dataset:
The validation results presented in this paper have shown that mean TCO3 fields from CAMSRA and CIRA are similar and agree to within 1 % when averaged over the period  2003-2016. are larger but are still within 5 %. All reanalyses have small positive biases with respect to Dobson TCO3 observations, with MACCRA having the largest global mean bias and CAMSRA the smallest (see Table  3 ). Agreement with ozonesondes is within 10 % in the long-term global mean. The reanalyses agree well in the stratosphere and have larger differences in the troposphere. CAMSRA agrees better with ozone-sondes above 15 hPa than MACCRA, which overestimates O 3 there. This makes CAMSRA a better dataset to be used as climatology in, for example, radiation schemes or radiance observation operators. CAMSRA and CIRA agree better with ozone-sondes in the tropical mid-troposphere to upper troposphere than MACCRA, which shows a large underestimation here (-30 %).
The choice of fire emissions has a large impact on the TCCO field, and the largest differences between CAMSRA and MACCRA are seen in biomass burning areas because different fire emission datasets were used in these reanalyses. CO from CAMSRA is more consistent in time than MACCRA, which showed some changes in the CO field because the assimilation of IASI CO was started in MACCRA in 2008  (Inness et al. , 2013) , while in CAMSRA and CIRA only TCCO from MOPITT was assimilated. For NO 2 , data from more instruments were assimilated in CAMSRA (SCIAMACHMY, OMI and GOME-2) than in CIRA (none) and MACCRA (SCIAMACHY only). This led to differences between the reanalyses, but the limited amount of independent validation observations for NO 2 made it difficult to assess these differences. The seasonal cycle of the tropospheric NO 2 columns is underestimated over East Asia, and to a smaller extent, over Europe by all three reanalyses compared with a tropospheric column NO 2 dataset based on SCIAMACHY and GOME-2A data.
e. the model-resolution orography not resolving the height of the volcanoes and therefore not being representative of the measurement sites with respect to the volcanic plumes. When calculating global mean statistics, it is advisable to exclude these two stations as unrepresentative. In addition to being a dataset of better quality and better temporal consistency, CAMSRA has the advantage that it provides more chemical species than CIRA (where only a limited subset was archived) and that data are available at a higher temporal and spatial resolution. In total 56 tropospheric chemical species of the CB05 chemical mechanism, 12 aerosol components and many additional diagnostics such as total columns and extinction coefficients can be obtained from CAMSRA. An inventory of the available model fields can be found at http://apps.
More time should be spent on acquiring and assessing new observations so that problems like the OMI row anomaly are addressed properly and the quality of the reanalysis is not degraded at times when lower quality data are assimilated (e. g. degraded NO 2 analysis during 2003 because of worse-quality SCIAMACHY data). It would also be advisable to explore the use of reprocessed datasets thoroughly, e. g. datasets processed by ESA's Climate Change Initiative (CCI) and the Seventh Framework Programme (FP7) Quality Assurance for Essential Climate Variables (QA4ECV) project. It could also be investigated if enough atmospheric composition datasets are available prior to 2003 to start a future reanalysis before 2003. Furthermore, work has started to look at emission inversion with the CAMS system, and we hope the next reanalysis will include some inversion capability to update the emissions during the assimilation according to the satellite observations.
Acknowledgements. Thanks to the data providers of the data assimilated in the CAMS reanalysis and the data used for the validation studies in this paper. The Copernicus Atmosphere Monitoring Service is operated by the  European Centre for Medium-Range Weather Forecasts  on behalf of the  European Commission  as part of the Copernicus Programme (http://copernicus. eu,last access:  15 March 2019 ). Review statement. This paper was edited by  Kostas Tsigaridis  and reviewed by two anonymous referees.
However, since it is not realistic to manually annotate hundreds or even thousands of hours of audio data from such daylong recordings, automated speech processing solutions are needed. This is where automatic WCE systems can come to the rescue, as they can provide an invaluable automated tool for measuring the number of words children have heard in a period of time. The existing state-of-the-art solution for the daylong recording and analysis task is the LENA TM system  ( Xu et al. , 2008; Gilkerson and Richards, 2009 )  developed by the LENA Research Foundation ( http://www. lena. org ). The LENA setup includes a compact recorder that can be placed inside the pocket of a vest worn by the child, and software that analyzes various aspects of the child's daily language experience from the audio, including measures such as conversational turns, adult word counts, and counts of child vocalizations.
Despite its tremendous value for the language research community, LENA as a software solution is not without problems. First, the software is proprietary and expensive. Second, only audio captured with the LENA recorder can be analyzed with the software, i. e. other audio files cannot be run through the same software. In addition, the included algorithms for speech processing are potentially outdated due to aging of the system, the basic building blocks having been introduced nearly 10 years ago (e. g. ,  Xu et al. , 2008 ). Since the algorithms are not open-source, it is also not possible to improve the software. Finally, LENA speech processing algorithms, including the WCE module, have been optimized for American English. While the system can be used with recordings in any language, its accuracy is not necessarily consistent across different populations, complicating any attempt at cross-linguistic comparison.
Given this background, there is an increasing demand from the research community to develop an alternative to LENA that would be open source, free of charge, compatible with audio data obtained using a variety of recorders, and robustly applicable to a variety of languages and language environments. In order to address this challenge, our ongoing collaborative project called Analyzing Child Language Experiences Around the World (ACLEW; https://sites. google. com/view/aclewdid/home ) aims to develop an open-source software package that would address the mentioned shortcomings of LENA (see  Le Franc et al. , 2018 , for initial work). The developed tools will be distributed as a Linux virtual machine that can be operated on a variety of computing platforms without special technical expertise in installing or operating speech processing algorithms (see  Metze et al.
, 2013; Plummer et al. , 2014 ). The system also aims to be scalable to large data sets with modest computational resources, as the aim is to make the tools usable by a broad population of researchers using a variety of computing environments. WCE is one among several tools under development that we hope to integrate into the software package. In the present paper, we describe our recent developments for the WCE component of the daylong analysis toolkit. After describing the WCE problem in more detail (the next subsection), we present our basic WCE pipeline. In a nutshell, our solution is based on language-independent syllabification of speech, followed by a languagedependent mapping from syllable counts (and a number of other acoustic features) to the corresponding word count estimates. Our work extends the earlier WCE system by  Ziaei et al.
(2016)  and also earlier syllable-based speech rate estimators such as those by  Wang and Narayanan (2007)  and  Morgan and Fosler-Lussier (1998). However, we go beyond the existing studies by (1) investigating applicability of syllable-based WCE to daylong child-centered recordings in several languages and in participant samples with varied socioeconomic status, and (2) comparing the impact of several speech activity detectors (SADs) and syllabifiers on the WCE performance. In addition, we (3) explore cross-language generalization of a language-independent supervised syllabification algorithm, thereby potentially replacing the unsupervised syllabification algorithms  ( Ziaei et al. , 2016 )  or acoustic phone models  ( Xu et al. , 2008 )  used in the earlier WCE systems. The ultimate aim of this study is to identify the best performing WCE system configuration that generalizes well to new languages and domains, and to see how it compares against LENA performance.
In addition, signal artefacts from clothing scratching against the microphone during child movement are also common. A large proportion of the collected data is also mono (e. g. , all LENA output), removing any useful directional information that could help in source separation. Finally, researchers are increasingly collecting daylong recordings with a variety of non-LENA recorders, which means the technical characteristics of the devices can also differ from one dataset to the next. This means that the overall properties of the audio data are largely uncontrolled, calling for robust signal processing methods. Another central challenge comes from the cross-domain applicability of the WCE system: performance of the system should ideally be similar in high-resource languages such as English (across all its dialects and social environments of the talkers) and in low-resource languages, such as Tseltal, a Mayan language included in our experiments.
State-of-the-art, open issues, and the present contributions So far, there are essentially two systems for WCE that have been proposed in the earlier literature, LENA and the system by  Ziaei et al. (2016)  , both already mentioned above. While LENA is specifically designed for analyzing child-centered daylong recordings (including a WCE module for measuring speech heard by the infant), the system by  Ziaei et al. was designed to only count the words of the person wearing the microphone. Their best performing system variant uses TO-Combo-SAD  ( Sadjadi and Hansen, 2013; Ziaei et al. , 2014 )  for speech detection, spectral subtraction for speech enhancement, and an automatic syllabifier from  Wang and Narayanan (2007)  for syllable count estimation before mapping the counts to word counts.
Ziaei et al. evaluated their system on Prof-Life-Log database consisting of 13 recordings from one adult participant wearing the LENA microphone during typical working days, with the data manually transcribed for word counts  ( Ziaei et al. , 2016  ; see also  Ziaei et al. , 2013 Ziaei et al. , , 2014 ) ). According to our knowledge, applicability of their system to child-centered daylong data has not been tested to date. However, our experiments will partially address this issue by having one of our WCE system configurations being highly similar to theirs (i. e. , using TO-Combo-SAD, spectral subtraction, the same syllabifier, and a linear model between signal features and words). Also, our system generally builds on that work, but as we show in the experiments, we also introduce more robust techniques for automatic syllabification of speech in daylong recording conditions.
As for LENA, the key components on the adult WCE pathway include detection of adult speech segments with a hidden-Markov model that uses so-called Minimum Duration Gaussian Mixture Models, application of a phone recognizer to the segments, and a linear mapping of the resulting vowel and consonant counts and speech duration measures to word counts as described in the previous subsection  ( Xu et al. , 2008 ). Since the introduction of LENA, several studies have evaluated LENA WCE performance across a number of languages and participant populations (see Table  1  ). The major technical drawback of LENA is its reliance on the structure of American English phonology and lexicon in the WCE process. As LENA uses an acoustic phone model trained on En-glish and a linear mapping from vowel and consonant counts to words, also optimized on English, the estimated word counts can be expected to be accurate only for languages that have the same ratio of vowels and consonants to words as the American English used in the training.
In the reported literature, this problem is often masked by the use of Pearson's linear correlation between estimated and hand-annotated words as the primary performance metric to measure LENA reliability (e. g. ,  Weisleder and Fernald, 2013; Soderstrom and Wittebolle, 2013; Canault et al. , 2016; Gilkerson et al. , 2015; Elo, 2016; Schwartz et al. , 2017 ). Longer stretches of speech in any language also mean more words and subword units, so relatively high correlations between LENA output and manually annotated reference word counts have been reported in the literature for a variety of languages, as summarized in Table  1. However, the picture is very different when comparing the estimated counts N hypo and true counts N true with a measure that also considers the absolute counts, such as the mean absolute relative error used by  Ziaei et al.
(2016)  in Eq. (  1 ) or median absolute relative error in Eq. (  2 ) used in the present study (more on evaluation in Section 4. 1 ). 𝐸𝑅 𝑅 mean ( % ) = mean ⎛ ⎜ ⎜ ⎝ | | | 𝑁 hypo -𝑁 true | | | 𝑁 true ⎞ ⎟ ⎟ ⎠ * 100 (1) 𝐸𝑅 𝑅 median ( % ) = median ⎛ ⎜ ⎜ ⎝ | | | 𝑁 hypo -𝑁 true | | | 𝑁 true ⎞ ⎟ ⎟ ⎠ * 100 (2) As an example, Elo (2016) reports a correlation of r = 0. 99 between LENA and hand-coded word counts for several 1-hour segments from two Finnish children. At the same time, the estimated and actual word counts differ by 75. 2%, most likely due to the highly different phonological and morphological structure of Finnish compared to English. This is not to say that LENA would not be applicable to languages different from English: a high correlation means that the relative word counts within the study population are still accurately measured, allowing word counts to be linked to other factors (e.
g. , developmental outcomes) with high validity. However, comparison of word counts across different participant populations is more problematic since LENA adult word counts are not guaranteed to correspond to actual words in a new language or in a substantially different dialect. 1 n the present paper, we aim to remedy the problem of languagespecificity by proposing a system that is always adapted to the target language using a small amount of orthographically transcribed speech. Since nearly all behavioral studies using LENA have checked the validity of the automated analyses in a given domain by comparing automated system outputs to manual annotations on a subset of the data (e. g. ,  Soderstrom and Wittebolle, 2013; Weisleder and Fernald, 2013; Canault et al. , 2016; Elo, 2016 ) , the data transcribed for validity could also be used to adapt the WCE system to the domain in question (and one can still estimate validity by a cross-fold validation procedure on the same data).
By implementing this type of low-resource adaptability, we aim for our system to be applicable to any language or use domain so long as the user is able to provide orthographic transcripts for roughly 30 min of audible adult speech (not to be confused with the total duration of annotated audio), ideally consisting of multiple different talkers and families. To validate our approach, we conduct experiments on six different corpora and use performance metrics that take into account the absolute accuracy of the estimator instead of measuring linear correlations. The overall purpose is to understand whether the adaptation approach is feasible with the amount of orthographically transcribed data that are manageable for language researchers to produce, and which technical components (SADs, syllabifiers) provide the best performance in WCE when overall accuracy and consistency across datasets are used as the primary criteria.
Our system's adaptability to new languages is achieved by having only a small number of free parameters that depend on the language L in question: (a) a syllable detection threshold  L , (b) feature-specific coefficients  L used in the linear mapping (highlighted in red in Fig 1 ), and (c) a correction coefficient  L for limited recall of the used SAD. Given that a dataset with ∼30 or more minutes of orthographically transcribed adult speech is available as training data, these parameters can be adapted to the language in question. A simple linear model is sufficient if we can assume that the word and syllable counts increase linearly with the duration of speech input at the time-scales of interest for WCE analysis (see also  Xu et al. , 2008; Ziaei et al. , 2016 ). Parsimony in such a model also reduces the risk of having substantially different estimator behavior in different languages and conditions, as it mitigates the risks of overfitting the system to limited adaptation data or creating far more accurate models for high-source languages.
, 2016 ) , and the Rosemberg corpus ( "ROS ") of Argentinian Spanish families from Buenos Aires metropolitan area  ( Rosemberg et al. , 2015 ). Some recordings in BER, and all recordings in CAS, MCD, and WAR are available from Home-Bank repository  ( VanDam et al. , 2016 ). Key properties of these corpora are summarized in Table  2. Each corpus consists of daylong (4-16 h) at-home recordings; each corpus samples from a unique community, with language varying across corpora and socioeconomic environment varying both within and across corpora. In each recording, the target child ( "participant ") wears a mobile recorder in a special vest throughout a normal day. BER, MCD, L05, and WAR recordings were collected with the LENA recorder, while CAS was recorded with Olympus WS-382 or WS-852, and ROS was recorded with a mix of Olympus, Panasonic, Sony, and LENA recorders.
First, all non-lexical transcript entries such as markers for incomprehensible speech, nonlinguistic communicative sounds, and all paralinguistic markers were discarded. In addition, for ready comparison to LENA, all transcribed words from non-adult speakers were discarded, even though the present WCE pipeline does not yet have a mechanism for separating speaker identities. As a result, only unambiguously transcribed word forms from adult talkers remain in the final gold standard dataset here. Every remaining orthographic entry separated by a whitespace was then considered as a word for adapting and testing of the WCE system.
OpenSMILE  SAD has the best average performance of all the alternatives whereas Noisemes SAD is clearly not as suitable for the present task. To-Combo-SAD would otherwise be on par with OpenSMILE, but the problems with WAR cause its mean performance to deteriorate substantially. As for the compared syllabifiers, the BLSTM generally outperforms the unsupervised methods thetaSeg and WN when either of the two wellperforming SADs or fixed-frame segmentation is used, especially when training data augmentation has been used. WN and thetaSeg perform approximately at the same level with the non-augmented BLSTMs with or without dropout with some variation across corpora and SADs, but fall behind the augmented BLSTMs in overall accuracy and consistency. This demonstrates that the multilanguage training of the BLSTM indeed works so that the models generalize to novel languages, and that the BLSTM is more tolerant against non-speech noise in the signals than the unsupervised methods.
We also verified that the WCE performance of the BLSTM system did not correlate with the average SNR of each corpus ( p > 0. 05, Pearson correlation), likely since the SNR differences between the corpora are small ( Section 3. 2 ). Finally, comparison to LENA shows that the TO-Combo-SAD + BLSTM system outperforms LENA on a number of varieties of English (American in BER, British in L05, and Canadian in MCD), even though LENA has been optimized for American English. Only in the case of WAR, performance is worse in the present system than in LENA, again suggesting that the present system's speaker attribution mechanisms are poorer than those of LENA, at least for the present task. Unfortunately, no LENA output data were available for the Spanish or Tseltal corpora to enable comparison on the non-English languages (but see Fig.
e. , excluding the subject itself from the dataset), and then transformed to the absolute value of the z-scores. As a result, 0 stands for test-case SAD performance typical to the adaptation data and positive values for increasingly different performance from the adaptation data. Participant-level relative WCE errors ( Eq. (  3 ) ) were also z-scored in an analogous manner across all the test folds to quantify the error on a given participant compared to the performance on other subjects in the pool (negative value for belowaverage and positive value for above-average WCE error). The normalized errors were averaged across all the six syllabifiers, and the data  from TO-Combo-SAD and OpenSMILE SAD were pooled before correlation calculation to focus on the shared effects of SAD behavior. Results of this analysis can be seen in Fig.
Conclusions and future work The aim of this study was to describe a basic framework for automatic word count estimation from daylong audio recordings of sound environments of language-learning infants, and to test its applicability to multiple languages and language environments. We also compared a number of speech activity detectors and automatic syllabifiers as potential modules in the pipeline and studied the applicability of supervised neural network training for language-independent syllabification of speech, as evaluated in terms of overall WCE system performance. One of the key aims was to have a system that performs similarly in high-and low-resource languages, as the existing commercially avail-able LENA system is expected to perform better on English than other data. To place the present work in a context, Fig.
8  shows the current WCE system performance (with OpenSMILE SAD and augmentedtraining BLSTM syllabifier) together with LENA performance metrics on English from the same study. In addition, LENA performance on a variety of other languages from a number of earlier publications is shown. With the exception of the WAR corpus, the present system achieves lower relative word count error rates on all tested language samples compared to what LENA has achieved in the previous studies. It also outperforms LENA on three of the four varieties of English tested in the present experiments. Importantly, the performance on English, Argentinian Spanish, and Tseltal is very similar despite the wide variation in language and recorder types. In contrast, LENA accuracy has historically varied enormously depending on the language being recorded, with non-English data having substantially worse word count accuracies than English data ( Fig.
8  ). This demonstrates that the basic approach consisting generic out-of-the-box SAD, language-independent syllabification, and domain-specific adaptation of a small number of parameters works both in principle and in practice. The current performance is far from ideal, but is still better than the static English-based acoustic model and mapping to word counts used in LENA. Unfortunately, our non-English corpora were not able to be processed with the LENA system, and therefore direct comparisons on exactly the same data is not possible. 9 It should also be noted that LENA word count estimates can also be linearly scaled to obtain more accurate word count estimates on novel languages and dialects, given that a suitable conversion coefficient is known or estimated from annotated data. In our experiments with the different varieties of English, linear scaling of LENA outputs based on the leave-one-subject-out adaptation protocol did not provide system-atic performance gains beyond the default LENA output ( Section 4.