dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgements The authors' research is supported by grants #  2012/50533-2, 2013/05083-1, 2006/06491-2 ,  2011/06179-7  and  2012/20110-2  from  São Paulo Research Foundation (FAPESP). We also thank  Agência UNESP de Inovação (AUIN)  for processing the national and international patents of the invention.
Compared to the current state of the art  [9] , our novel electro-optic phase architecture provides an improvement by a factor of 3, reaching a million words per second when tested on a standard speech-recognition database. As reservoir computing is a universal computational concept, many other complex problems can, in principle, be addressed by our approach, such as real-time information extraction from high bit-rate optical data flow or the prediction of a high-dimensional deterministic time series. The article is organized as follows: Section II introduces some basic concepts of reservoir computing. Section III bridges these neural-network-based concepts with a timedomain only hardware implementation in the particular case of the proposed electro-optic phase-delay oscillator. Section IVA reports the experimental and numerical results obtained on a first benchmark of spoken digit recognition (TI46).
This particular temporal task example gives us the opportunity to discuss the importance of the time-scale control revealed in our work, between the write-in (input layer) of the information to be processed, the intrinsic reservoir (internal layer), and the read-out (output layer). Section IV C is devoted to additional experimental and numerical speech-recognition results obtained from an extended complexity speech-recognition database, namely, the one used for the standard AURORA-2 benchmark. The last section summarizes the results and gives a perspective for future work for practical photonic RC processors.
To evaluate such computational performances, we have implemented classification tasks consisting of speech recognition. Two different speech databases have been used for testing. The first one corresponds to a test already widely used in several recent photonic RC demonstrations-the TI46 test  [5-9, 24,27,28]. This first test is important to compare with already available hardware RC results in the literature. The second test was also addressed in this work, based on a very similar task-however, with an extended database for which the classification problem is more realistic and thus also more complex. Ref. [28]  and reused is several recent RC hardware evaluation under the same condition; it consists in the recognition of pronounced digits between 0 and 9, with the database corresponding to 500 digits extracted from the TI46 speech corpus  [29].
Each response of the delay dynamics to each of the 500 encoded input spoken digit signals fu I σ ðnÞjn ∈ ½1; Ng is recorded by a real-time ultrafast digital scope (Lecroy WaveMaster 845Zi-A, 30-GHz analog bandwidth, 80 GS=s) as depicted in the bottom of Fig. 2 , with an oversampling allowing for further postprocessing [e. g. , averaging, or selection of interpolated samples according to the already mentioned asynchronous read-out sampling in Eq. (  9 ), i. e. , δτ R ¼ ð1 þ εÞδτ]. Each sampled response signal fx σ R k ðnÞjn ∈ ½1; Ng is conditioned numerically in the form of a K × N matrix M x , thus in a similar form compared to the previously introduced input signal matrix M in for u σ k ðnÞ. Processing all the digits of the database thus leads to 500 of such response matrices M x. 4. Learning the optimal read-out matrix W R A given response matrix M x is representative of the dynamical processing performed by the EO phase-delay dynamics.
A simple form for the construction of such a matrix M y is to set zeros on each (M -1) lines related to the wrong modality and to set ones on the line corresponding to the correct modality. Practically, the learning requires the use of a subset of digits from the whole database of 500 spoken digits, the complementary part of this subset being used later for the testing of untaught digits. One of the most attractive features of RC is its easy and always converging learning technique to compute the optimal read-out matrix W R opt. This is basically performed through a simple ridge regression minimizing the error for all possible W R matrices in FIG. 4. Illustration of the expected optimized read-out processing through a (M × K) matrix W R , left multiplying the transient response (K × N) matrix M x , thus resulting in an easy-tointerpret target (M × N) matrix M y.
Cross-validation was performed through the repetition of 20 learning procedures, according to the choice of 20 nonintersecting subsets of 25 testing digits, and their corresponding training subsets. This allowed us to obtain an average computational performance measure in which each of the 500 digits has been used once for testing. The optimal read-out matrix was calculated off-line using the experimentally recorded transient response matrices M x. This calculation was done by a Matlab routine executing the ridge regression through a standard More-Penrose matrix inversion, according to the following expression: W R opt ¼ M y • M ⊺ x ðM x M ⊺ x -λI K Þ -1 ; ð14Þ where I K is the K-dimensional unity matrix, and ð:Þ ⊺ refers to the matrix transpose.
C. Database from the AURORA-2 test The AURORA-2 test is a standardized benchmark developed in the framework of speech recognition for mobile-phone communications. The original acoustic waveforms are extracted from the TIDigit corpus database (from the Linguistic Data Consortium) as for the TI46 test. It is, however, much richer in terms of speaker variability  and thus more challenging in terms of recognition difficulty. The database is constructed of 326 speakers (111 men, 114 women, 50 boys, and 51 girls), each speaking approximately 77 digit sequences from one (isolated) to seven digits. The sequences were made up of the following digits: zero, oh, one, two, three, four, five, six, seven, eight, and nine. On this basis of a set of clean speeches, the AURORA-2 test consists in the construction of test and training subsets, in which different kinds of noise can be added with different noise levels.
It is worth noticing that performance on clean sets is available in the literature  [30]  and, moreover, also in the recent case of an advanced RC-based recognition algorithm  [31]. In the latter reference, a complex multistate architecture and very high number of nodes were used (up 20 thousand), resulting in performances very close to the state of the art, with the best WER of about 1% on clean digits conditions. We have performed, both numerically and experimentally, a speech recognition test on the clean and isolated digits subset of the AURORA-2 database. The parameters of our photonic reservoir are the following: number of nodes ¼ 1000, the feedback gain is β ¼ 0. 7, the offset phase is ϕ ¼ 2π=5, and the scalar factor is ρ ¼ 1. We have also used 3 times EDM so that we have indeed 1000 × 3 ¼ 3000 nodes per time delay.
ACKNOWLEDGMENTS This work was supported by the  European  project  PHOCUS  ( FP7  Grant No. 240763 ), by the  Labex ACTION program  (Contract No. ANR-11-LABX-01-01 ), by the  BiPhoProc ANR  project ( ANR-14-OHRI-0002-02 ), and by  CONACyT/MÉXICO  under Project No. 290788  and Grant No. 314603.
Abstract. Interannual variability in desert dust is widely observed and simulated, yet the sensitivity of these desert dust simulations to a particular meteorological dataset, as well as a particular model construction, is not well known. Here we use version 4 of the Community Atmospheric Model (CAM4) with the Community Earth System Model (CESM) to simulate dust forced by three different reanalysis meteorological datasets for the period 1990-2005. We then contrast the results of these simulations with dust simulated using online winds dynamically generated from sea surface temperatures, as well as with simulations conducted using other modeling frameworks but the same meteorological forcings, in order to determine the sensitivity of climate model output to the specific reanalysis dataset used.
g. ,  Miller and Tegen, 1998b; Mahowald et al. , 2002 Mahowald et al. , , 2003;; Ginoux et al. , 2004; Ridley et al. , 2014). Thus an open question remains as to how robust our simulations of inter-annual variability (IAV) are, and how sensitive they are to the meteorological dataset vs. the modeling framework. Characterizing how well IAV is simulated in models allows a better understanding of how much we should trust model output in studies directed at understanding the role of dust IAV in contributing to IAV in total aerosol optical depth (AOD) variability (e. g. ,  Streets et al. , 2009)  or in dust impacts on ocean biogeochemistry IAV (e. g. ,  Doney et al. , 2009). A related question is how much observational data do we need in order to correctly characterize the mean dust amount, based on how much interannual variability we think exists in different locations.
Note that the global model simulations used here may miss small-scale features that may be important for IAV, such as dust devils or moist convective events (e. g. ,  Renno et al. , 2000; Marsham et al. , 2011). Here in this study, we use three different reanalysis meteorological datasets, online dynamic winds and different modeling frameworks to try to understand how robust interannual variability in simulated dust is across 1990-2005. Our emphasis is on conducting sensitivity studies comparing the importance of different model meteorological datasets and frameworks, but we do include some comparison to observations. While we focus on interannual variability, we will contrast that to seasonal variability, which has been commonly evaluated in models and previous intermodel comparisons  (Huneeus et al.
Model description Several models are used in this study, all of which include prognostic dust. The atmospheric component of an Earth system model, the Community Atmospheric Model (CAM4) of the Community Earth System Model (CESM)  (Neale et al. , 2013; Hurrell et al. , 2013)  is capable being forced either by online-calculated dynamical winds or by reanalysis datasets and is the model used for the bulk of the analysis to test sensitivity to which reanalysis winds are used (Sects. 2. 1. 1 and 2. 1. 2). To contrast with this model, results from other models, including the CAM5 version of the same model, and two chemical transport models, driven by reanalyses, are also used (Model of Atmospheric Transport and Chemistry, or MATCH, and GEOS-chem) (Sect. 2. 1. 3). More details are described below, along with a summary of the model simulations (Table  1 ).
CAM4 dust model For the bulk of the analysis in this study, simulations using the CESM were conducted focusing on the atmospheric   Y 1990 Y -2008 Y Albani et al. (2014) )  model, CAM4  (Neale et al. , 2013; Hurrell et al. , 2013). This model is capable of simulations based on prognostic dynamic meteorology, as conducted for long climate simulations, or simulations forced to follow specific meteorological events, which allows comparison to specific observational data or field campaigns  (Neale et al. , 2013). The CESM model includes four main global climate model (GCM) components: atmosphere (in this case, CAM4), land, ocean, and sea ice, all linked by a flux coupler. However, only the land and atmosphere components were prognostic for these simulations, with prescribed ocean and sea ice being used instead.
Meteorological forcing datasets In the CAM model, the reanalysis forcing can be used to nudge the model close to specific weather patterns so that events can be simulated  (Lamarque et al. , 2011)  using a similar methodology to one used elsewhere (e. g. , MATCH;  Rasch et al. , 1997; Mahowald et al. , 1997). In this procedure the horizontal wind components, air temperature, surface temperature, surface pressure, sensible and latent heat flux, and wind stress are read into the model simulation from the input meteorological dataset. These fields are subsequently used to internally generate (using the existing CAM4 parameterizations) the variables necessary for (1) calculating subgridscale transport including boundary-layer transport and convective transport; (2) the variables necessary for specifying the hydrological cycle, including cloud and water vapor distributions and rainfall (see  Lamarque et al.
, 2011  for more  details; developed in Rasch et al. , 1997; Mahowald et al. , 1997). While this approach has the advantage of increased versatility with the input reanalysis dataset, there are inconsistencies between the model and reanalyses or observations, which leads to the model being nudged back towards the reanalyses. These inconsistencies mean the model has an anomalous source of energy or water. The approach used in MATCH and CAM was developed to minimize these inconsistencies (e. g. ,  Mahowald et al. , 1995 Mahowald et al. , , 1997)). Previous studies have shown that the MATCH-CAM framework can reproduce the reanalysis precipitation to a very high degree at the sub-daily to monthly to annual timescales  (Mahowald et al. , 1997; Mahowald, 1996). Three different reanalysis meteorological datasets were used to simulate dust entrainment, transport and removal in the CAM4 simulations: MERRA (Modern Era-Retrospective Analysis for Research and Applications version 1;  Rienecker et al.
, 2011) , NCEP (National Centers for Environmental Prediction)-NCAR (National Center for Atmospheric Research) 50-year reanalysis  (Kistler et al. , 2001) , and ECMWF (European Center for Medium-Range Weather Forecasts) ERA-Interim  (Dee et al. , 2011). Within the meteorological literature there are many studies contrasting these datasets to available observations, and showing the errors in the reanalyses, especially the moisture transports and precipitation  (Trenberth and Guillemot, 1998; Trenberth et al. , 2000 Trenberth et al. , , 2011;; Trenberth and Fasullo, 2013). Reanalyses should not themselves be considered observations, but are the closest representation we have to observed meteorology, which can drive chemical transport models, and thus they represent an important resource. Here in this paper we supplement the meteorological analysis of different reanalysis datasets by contrasting how they impact dust emission, transport and deposition.
Not all of the input reanalysis data were available in the format required for CAM4 for the entire satellite era, so most of the analysis was conducted over a time period of 15 years when all the input data were accessible. There are parts of the analysis for which data outside this time period was used, but this is always indicated in the results. A summary of the model simulations and time periods of data availability are shown (Table  1 ). The first year of each simulation was neglected to allow for spinup. For two of the model simulations used here -CAM4 (MERRA) and CAM4 (NCEP) -additional aerosol species were available for analysis -sea salts, black carbon (BC), organic carbon (OC), and sulfate (SO 4 ) -and we use these to contrast the correlations between dust aerosols and other aerosols for a sensitivity study, referenced in Sect.
Other dust models In this paper, we also include sensitivity studies, where additional models are used, to contrast the importance of meteorological datasets with model construction. We briefly describe here the other models used in the study, with an emphasis on contrasting the differences in the model construction, but refer the interested reader to the specific model descriptions elsewhere. An AMIP-style simulation was also conducted using the CAM5 model for comparison with the CAM4 AMIP simulation. While the dust generation module in the CAM5 is identical to the CAM4, there are significant differences in the physics  (Hurrell et al. , 2013) , as well as the aerosol formulation, which has implications for dust  (Albani et al. , 2014). The CAM5 model includes new planetary boundary layer, radiation, and moist convective parameterizations  (Hurrell et al.
5 • , and were interpolated onto the CAM grid for analysis in the paper. This model uses a similar dust entrainment scheme as CAM, but has different size distribution and deposition mechanisms, as well as different boundary layer and moist convection physics. It should also be noted that there is a difference between how the GCHEM (MERRA) model and CAM models incorporate reanalysis meteorology. The GCHEM (MERRA) model reads in all meteorological parameters from the reanalysis datasets, including turbulent and moist convective mixing, as well as the hydrology. This has the advantage, which the chemical transport model does not have, to rederive the hydrological cycle or mixing, which can be difficult (e. g. ,  Mahowald et al. , 1995; Rasch et al. , 1997). On the other hand, it makes the model less flexible, as it can only be operated using reanalysis datasets with mixing parameters output at the right frequency, in contrast to the MATCH or CAM framework (e.
g. ,  Rasch et al. , 1997; Mahowald et al. , 1997). This means that although both the CAM4 (MERRA) and GCHEM (MERRA) models are forced by MERRA winds, the surface winds and transport may still be slightly different. Finally, simulations using the MATCH  (Rasch et al. , 1997)  with NCEP reanalysis data  (Mahowald et al. , 1997; Kalnay et al. , 1996)  are included. These simulations use a similar entrainment and deposition scheme  (Zender et al. , 2003a) , with a simple wet removal scavenging coefficient  (Luo et al. , 2003) , and have been extensively compared against observations  (Luo et al. , 2003 (Luo et al. , , 2004;; Mahowald et al. , 2003). In contrast to the CAM models discussed earlier, there is no vegetation phenology included. Instead, the preferential source term from  Ginoux et al.
Observational data description For completeness, we compare standard annual means from each simulation to available data (e. g. ,  Ginoux et al. , 2001; Huneeus et al. , 2011)  in order to show that the mean dust cycle is reasonable in our models. In this study, we use the annual-mean surface concentration, AOD and deposition compilations from  Albani et al. (2014). For the variability studies, data were chosen for overlap with model runs and availability of longer datasets to evaluate interannual variability. In situ observations from the University of Miami network  (Prospero and Nees, 1986; Prospero et al. , 1996; Arimoto et al. , 1990 Arimoto et al. , , 1997) )  were used here, as compiled in  Luo et al. (2003)  and  Mahowald et al. (2003)  and updated at some sites  (Prospero and Lamb, 2003).
In addition, in situ observations from three sites in northern Africa from the AMMA (African Monsoon Multidisciplinary Analysis) campaign were included  (Marticorena et al. , 2010). Details on the individual sites and locations are included in Table 2, and are shown in Figs. S1, S3 and S5 in the Supplement. Sun-photometer-derived AOD is included from the AERONET (Aerosol Robotic Network) database  (Holben et al. , 1998). Only sites where more than 50 % of the modeled AOD was from dust are included in this comparison (as filtered in Fig. 1  using in  Mahowald et al. , 2007) , and only sites with more than 18 months of data are used here to estimate variability (Table  2 ). Because both sea salt and dust occur in the coarse mode, we cannot use remote sensing measure-ments of the coarse vs. fine mode to identify dust-dominated stations.
Two sites in the Southern Hemisphere are included (bottom of Table  2 ): surface concentration data from Rio Gallegos  (Zihan, 2016)  and deposition data from Kerguelen  (Heimburger et al. , 2012). Because of the limited datasets in the Southern Hemisphere, we will consider these sites separately in Sect. 3. 3. Although there are no AERONET data using these Northern Hemisphere criteria available in the Southern Hemisphere, if we focus on the coarse mode, there is one station which is dominated by dust (i. e. , far from the coasts, where sea salts would make up a large percentage of the total aerosol load): Tinga Tingana in southeastern Australia. Located in a region of predominately westerly winds, Tinga Tingana lies downwind of the central Australian desert, allowing desert dust to feature prominently in its aerosol load.
Tinga Tingana also has a data record beginning in September of 2002, which overlaps our chosen time period by more than 3 years, making this the best Southern Hemisphere site to evaluate our modeled AOD variability. For evaluation of the models' precipitation, we use the CPC Merged Analysis of Precipitation (CMAP) (http: //www. esrl. noaa. gov/psd/data/gridded/data. cmap. html) (Xie and Arkin, 1997). This is a combination of in situ and satellite observations, as well as models, to present the best estimate of precipitation.
(Table There is also a correlation between Sahel precipitation and Barbados concentrations  (Prospero and Lamb, 2003) , or precipitation and visibility in the Sahel (e. g. ,  Mbourou et al. , 1997; Mahowald et al. , 2007) , so the other driver could be precipitation. In some of the CAM4 model simulations, the IAV of the source strength does feature significant correlations with both LAI and precipitation, but in general those same cases feature even stronger correlations with surface winds (Table  S6 ). Although the quality of the surface wind data precludes us from evaluating the reanalysis for surface winds, as discussed in  McVicar et al. (2012) , we can evaluate the precipitation, which is commonly done (e. g. ,  Trenberth and Guillemot, 1998). When we do, we see that the reanalysis precipitation datasets are not capturing either interannual variability in precipitation or the slope in the pre-   cipitation, compared to the CMAP precipitation compilation (more details in Sect.
Note, however, that some of the CAM4 models (CAM4 (MERRA) and CAM4 (NCEP)) have similar variability as the non-CAM4 models (GCHEM (MERRA) and MATCH (NCEP)), especially at Kerguelen and Tinga Tingana, suggesting that some of this variability may also be associated with the meteorological dataset. We will explore the ramifications for variability predictions in Sects. 3. 4 and 3. 5. Further downstream of the sources, the amount of variability in deposition observed at Kerguelen is 0. 10, which is overpredicted in most of the models (Table 5), while at Tinga Tingana, the IAV variability is 0. 42 in the observations, and tends to be smaller in the models. At both Rio Gallegos and Tinga Tingana, the ratio of the IAV variability to the seasonal variability is 1. 4 and 1. 5, which are on the lower side of the observations in the Northern Hemisphere (Fig.
These show correlations similar to previous studies (e. g. ,  Moulin et al. , 1997; Mahowald et al. , 2003) , and suggest some correlation between NAO and ENSO. Because we are using a relatively short time period (16 years), these signals do not show as statistically significantly as if we use a longer time period, but the same models (e. g. , MATCH in  Mahowald et al. , 2003, included 22 years)  have shown a similar pattern in previous studies. There is much more coherence in the simulated variability from the reanalysis winds, than seen in the NAO or ENSO (Fig. 8a  vs. Fig. 8c  and d ). This is also consistent with the lower correlation between the SST-driven model simulations (AMIP-style) (Fig. 8e ). There are much higher correlations between model results when we use reanalysis winds compared to forcing with only sea-surface temperatures, indicating the value of using reanalysis datasets to obtain more robust results.