dataesr/scientific-paragraphs-categorization huggingface dataset:
Comparing to the PLS regressor, SVMs, and deep neural network models, the proposed method achieves the lowest diagnostic error rate in the experiment of this paper. Additionally, the hardware performance evaluations of the RTL implementation indicate that the optimizations used in this paper achieve a speed up of 10. 89× relative to that of the CPU. The main novelties of this work are summarized as follows: (a) An FPGA-suitable CAPT framework is conceived and trained, in which the phoneme pronunciation diagnostic algorithm is based on the partial least squares regression method and an improved support vector machine, so that it could raise the accuracy performance of the framework by suppressing the collinearity problem among the predictors. (b) The phoneme diagnostic core is implemented at the register-transfer level (RTL) via the recently-proposed Matlab-to-RTL SW/HW co-design flow for the purpose of high development productivity and maintainability.
The implementation is further accelerated at the instruction-level, and a speedup of 10. 89× is achieved relative to its CPU implementation. (c) The proposed RTL implementation of the CAPT framework is functionally verified and evaluated by using a French phoneme utterance database, demonstrating its application values. The remainder of this paper is organized as follows: Section 2 describes the proposed embedded CAPT framework and explains how it is trained; Section 3 presents the implementation and optimization processes of the proposed CAPT framework; Section 4 analyzes the evaluation experiment results; and finally, Section 5 gives the final conclusion of this work.
Prototyping of the Proposed CAPT Framework This paper prototypes the proposed CAPT framework by using a VHLS-based SW/HW co-design flow. As shown in Figure  2 , this workflow allows one to synthesize the algorithm behavior from a high-abstract program language level (Matlab) down to low ones (register transfer languages) via intermediate C++ code. More precisely, the algorithm behavior is specified in Matlab, then automatically transformed to intermediate C++ code by using Matlab Coder. The generated C++ function is further verified and optimized manually in GCC. Finally, the desired RTL implementation is generated and evaluated in Vivado HLS (formerly AutoPilot from AutoESL)  [50]. The methodology of this work was approved in writing by the Ethics Committee of the Foreign Language College, Capital University of Economics and Business.
Original Implementation Considering that the diagnosing tasks are activated when the interested signals are segmented and pre-processed, the proposed CAPT detector is prototyped as a slave IP core, allowing a master module to invoke it at any time. In order to facilitate the updating of the network parameters, parameter ports are designed. Moreover, the data ports should be parallelizable, allowing parallelizing the computing by accessing multiple data sets simultaneously. The final interface protocol is shown in Table  1. It includes a group of logical control signals (CLK, RST, START, DONE, IDLE, READY, and RETURN) and the data ports (x_ * , W1_ * , W2_ * , B and ETA). The parameter ports are implemented in the classical memory protocol, allowing the core to access the external memory with an index when needed.
x_ * is the utterance sample to be diagnosed, and its size is determined by the sampling precision of the system. In the case of this paper, we set n as 14, allowing a sampling frequency of 2 n = 16384 Hz. W1_ * and W2_ * are the parameter matrices of the first and second layers of the proposed framework, respectively. The size of W2_ * is set as 2 6 = 64, and W1_ * is therefore a 2 n+6 = 1048579-element array, which can cover the 35 phonemes in the case of this paper. If desired, the data and parameter ports can be expanded to accelerate the processing speed by communicating parallelly. B is the bias value of the second layer. ETA is the threshold value of the decision cycle. If the diagnosis result is positive, RETURN output true, otherwise false when negative. Algorithm 2 shows the Matlab pseudocode of the proposed CAPT implementation.
Because Matlab provides a vector-available programming environment, the target behavior can be efficiently described. The algorithm's behavior starts the diagnostic processing after parameter initializations. Lines 2 and 4 correspond to the regression operations of the first and second layers (see (  2 ) and (  3 )), whereas Lines 3 and 5 correspond to the ReLU and sigmoid active functions. The final decision is made in Line 6. The operator ". ×" returns a matrix whose elements are the products of the corresponding elements of the two input matrices. Next, we transform the algorithm behavior from Matlab into C++ code via the sourceto-source compiler Matlab Coder. For the reason that C++ is only available for scalar variables and operations, the matrix computations in Matlab are mapped into loops line by line.
The pseudocode of the generated C++ behavior is shown in Algorithm 3. The first nest loop (Lines 2-8) computes the PLS regressions (see  (2) ) with the help of a newly-allocated register. The second loop with a "if" body (Lines 9-15) corresponds to the ReLU function. Lines 17-20 compute the SVM regressions (see  (3) ). Here, the buffer of the first loop is reused to save the hardware sources. Line 21 is the sigmoid function, and the decision is made within Lines 22-26. From this point, we can start to synthesize the C++ code down to the register level through the HLS process. HLS extracts the source code as a control-and-datapath flow graph (CDFG)  [51]  and then represents it as a finite state machine (FSM)  [52]. Figure  3  shows the diagram of the extracted FSM, in which each node is a state including a series of subsequent operations, and the arrows are the operating orders.
Optimizations Automatically synthesizing the behavior from high to low abstract levels allows fast algorithm prototyping, but there are still performance gaps between it and manual implementations in terms of time control, execution speed, consumption, and other factors  [53, 54]. As indicated in ref. [52] , the quality of the HLS-based implementations is impacted by the following three factors: high-level description of language, optimization forms, and applying orders of optimization forms  [55, 56]. In the case of this paper, multiple optimization forms are made successively at the interface, loop, and instruction levels in order to accelerate the implementation by improving the parallelism of the code. According to the estimations, the first loop nest costs almost all the clock cycles in the original implementation (7454790 vs.
7455344 cycles), so the main goal of the optimization work is to accelerate this part. First, the memory ports of the target core are optimized in order to mitigate access conflicts and reduce consumption due to the logic controls. According to Algorithm 3, x x x and W1 W1 W1 are the frequently-activated ports in the first loop nest, and this will lead to access conflicts when parallelizing. We therefore partition the related arrays from a single to multiple ones to expand the bus width. An index D opt = 4, 8, 16,. is defined to present the optimization depth. The optimized memory port protocol is shown in Table  2 , demonstrating that the band of the two ports is multiplied by D opt ×. With Vivado HLS, this optimization is made by inserting the directives of array_partition into the code (see Lines 1 and 2 of Algorithm 4).
We insert the "pipeline" directives into the loops of the optimized code (see Lines 6 and 13 of Algorithm 4) in order to accelerate the iterations. Finally, the code of Algorithm 4 is optimized at the instruction level depending on the symbolic expression manipulation strategies presented in  [52]. In the original C++ code, the iterations of Line 5 have dependent relationships between each other, so they cannot be parallelized only via loop unroll. We therefore partially unroll the loops with a factor of D opt , and then re-specify the iteration information and body manually. As shown in Lines 5-10 of Algorithm 4, the loop body is repeated D opt times during each iteration, and the iteration number is reduced by D opt accordingly. The body operations are described in polynomial form: h1 h1 h1 = (i+1)×D opt -1 ∑ k=i×D opt x x x(k) × W1 W1 W1(i, k) (12) The re-specified code avoids the dependence between the loops, enabling parallelization of the operation schedule.
Function Verifications This subsection verifies the function of the VHLS implementation. The experiments are conducted by the CUEB French Phoneme Database 1. 0 established by the Capital University of Economics and Business and the Institute of Acoustics CAS  [27] , which includes 35 phonemes × 6 sessions × 23 participants = 4830 samples. The participants in the data collection are asked to read the French phonemes shown in Table  3  six times to perform six different data sessions.
consonants Deaf consonants: [p], [t], [k], [f], [s], [S] Sound consonants: [b], [d], [g], [v], [z], [Z] Lateral consonants: [l], [r] Nasal consonants: [m], [n], [N] Figure  6  shows the test bench of the experiments. The preprocessing cycles of the input phoneme waveform include band filtering, Fourier transforming, and normalizing. The normalized frequency spectrum is used as the predictor vector for the detectors. We verify the functions of the proposed implementation (capt_vhls) by comparing it with its Matlab prototype (capt_matlab) and C++ routine (capt_cpp). Moreover, in order to evaluate the diagnostic error rate of the selected algorithm, the PLS regressor (pls_matlab), hardmargin SVM (hmsvm_matlab), soft-margin SVM (smsvm_matlab), and deep neural network (dnn_matlab) are implemented in Matlab as references.
All the reference implementations are specified in Table  4. All the implementations are considered to be constructed by input, hidden, and output layers. The hidden layer numbers of capt implementations and dnn_matlab are two and three, and those of the other implementations are one. The lost value and maximal iteration number are set as 1 × 10 -6 and 1000, respectively. The ReLU and sigmoid functions are used as the activate functions of dnn_matlab. Considering that machine learning modules necessitate high data precision and the values of speech signals usually have a large dynamic range, the data type of the proposed implementation is set to floating-point one. We also tried to use the fixed-point data to realize the same function and found that all the threshold values must be carefully set if the similar accuracy performance are desired.
That is, fixed-point implementation may lead to unknown risks in the case of this paper. We divide the data set into two groups for training and testing, each with different size ratios. The size ratios are R = 1:5, 2:4, 3:3, 4:2 or 5:1. The average diagnostic error rate of three measurements is used to determine the final evaluation result. Figure  7  compares the diagnostic error rates of all the implementations. First, it demonstrates that the diagnostic error rates decrease with the increases in the training set size. This is because enough training data improves the machine learning classifiers by overcoming the over-fitting problems. Second, when the different implementations have the same training set size, the implementations of capt_ * achieve similar diagnostic results, indicating that the VHLS procedure used in this paper correctly synthesize the Matlab prototype to the registertransfer level automatically.
Third, among the PLS-only, SVM and DNN implementations, smsvm_matlab achieves the best accuracy performance at R = 1:5, 2:4 and 3:3, whereas the dnn_matlab at R = 4:2 and 5:1. Comparing to them, capt_vhls improves it by 1. 24%, 0. 79%, 1. 33%, 0. 97% and 0. 89%. That is, the CAPT framwork of this paper possesses the best accuracy performance in this experiment. Finally, it should be noted that it shows that capt_vhls achieves similar diagnostic error rate corresponding to capt_matlab and capt_cpp, but that does not mean that the new implementation possesses lower accuracy performance. This tiny difference is caused by dividing the data set randomly into the training and testing groups, which introduces random factors in the evaluations. 4  for the definition of acronyms).
Running-Time Performance This subsection first analyzes the operation schedules of the proposed implementation, then evaluate its running-time performance by comparing it with multiple reference implementations. For the purpose of fairness, all the references are implemented in high abstract environments, including Matlab and C++. This allows us to evaluate the implementations within the similar development productivity constraint. The optimizing depth is set as Dopt = 32 in order to well balance the hardware consumptions and running time efficiency. For better understanding, the optimized behavior code, Algorithm 4, is divided into the following four scopes to analyze: (a) Scope 1: Line 3, (b) Scope 2: Lines 4-11, (c) Scope 3: Lines 12-17 and (d) Scope 4: Lines 18-24. The four scopes execute in sequence, so the total latency cost L opt can be estimated as L opt = L s1 + L s2 + L s3 + L s4 ( 19 ) where L s1 , L s2 , L s3 and L s4 are the latency costs of the four scopes.
It should be noted that the value of P opt varies with the optimizing depth D opt. In this case, the evaluation report for Vivado HLS indicates that P opt = 8. 63 ns at D opt = 32, so its running time is T opt = 8. 63 ns × 663,320 cycles = 5. 72 ms. For the purpose of unbiased evaluation results, we compared the running-time performance of the final implementation with multiple references. Table  7  specifies their developing environments. capt_matlab is the very original version for algorithm verifications. capt_cpp is developed from the Matlab-to-C transforming version, which is used as the input of the high-level synthesis process. capt_vhls_ori is generated from the capt_cpp code without any optimizations. capt_vhls_opt_D opt (D opt = 4, 8, 16, or 32) are the versions optimized from capt_vhls_ori with different optimizing depths.
capt_vhls_opt_32 is the proposed CAPT prototype of this paper. Figure  11  plots the acceleration ratios over the different implementing versions, in which capt_matlab is set as the base. It illustrates that the C++ version achieves a speedup of 1. 8×, whereas the original VHLS version 0. 97×. This result means that synthesizing the Matlab code directly down to register-transfer levels do not obtain acceleration gains corresponding to today's commonly-used processors and developing environments. However, the optimizing methods used in this paper effectively improve the running-time efficiency of the VHLS implementations. Comparing with capt_vhls_ori, the optimizing forms made in this paper accelerate the implementations by 2. 36×, 3. 85×, 6. 50× and 11. 28× at D opt = 4, 8, 16 and 32. Comparing with the two CPU-based implementations capt_matlab and capt_cpp, the proposed CAPT prototype capt_vhls_opt_32 achieves speedups of 10.
Considering that a classical DNN model include 5 layers at least (1 input, 3 hidden and 1 output layers), whereas the proposed one only 4 (1 input, 1 PLS feature extracting, 1 SVM classifying and 1 output layers), the latter is more suitable for the systems on chips. As far to the register-transfer level implementing of the design, we prototype it via a new-proposed VHLS SW/HW co-design flow in order to facilitate the development works and maintenances. During this work, it is found that synthesizing directly the behavior from Matlab down to RTL prevents the implementation from benefiting from the running-efficiency advantages of FPGAs, a series optimizing forms are therefore made in the loop and instrument levels. The CUEB French Phoneme Database is used to evaluate the achievements of this work.
The experiments results verify the basic function of the new implementation by comparing it with its Matlab and C++ implementations. The hardware evaluation experiments demonstrate that the prototype of this paper make efficient use of the given hardware resources, and achieves a speedup of 10. 89×, which making better use of the hardware resources. Despite of many benefits of development productivity and easy maintenances, it should note that high level synthesis seriously constrain the performance of FPGA implementations in the terms of hardware costs and running-efficiency comparing to the low abstract level implementations. If high performance is desired, some more bottom optimizations are still required, especially when the constraints of Placing and Routing cycle is taken into account.
We have solved explicitly the recurrence relations for the polynomials a i,K (z) and b K (z), up to K = 100. In the anciliary file we provide a Mathematica code that constructs the solutions recursively. The procedure is completely algebraic and it only requires solving a system of linear equations with rational coefficients at the K-th step. Its solution is then fed in the linear system of the (K + 1)-th step, etc. Unlike the one-loop case, we have not attempted to find a closed-form expression for F K (z) for arbitrary K. Instead, we used the obtained solutions for several small values of K to deduce the asymptotic behavior of F (2) K (z) as z → 0 for any K, F (2) K (z) = 3 K + 1 log(z) z + c (2) 1/z,K K + 1 1 z + O(log z) , (4. 38) where c (2) 1/z,K = - 3 2 ζ 3 + 3ζ 2 - 7 2 - 3 K -1 - 3 K - 3 K + 1.
Acknowledgements We thank  Benjamin Basso ,  Joao Caetano ,  Hao Chen ,  Vasco Goncalves ,  Jack Holguin ,  Shota Komatsu ,  Cyrille Marquet ,  Sasha Monin ,  Ian Moult ,  Baur Mukhametzhanov ,  Kyriakos Papadodimas ,  Riccardo Rattazzi ,  Kai Yan  and  HuaXing Zhu  for useful discussions. This project has received funding from the  European Research Council (ERC)  under the European Union's  Horizon 2020 research and innovation programme  (grant agreement number  949077 ). DC is supported by the  French National Research Agency  in the framework of the " Investissements d'avenir" program  ( ANR-15-IDEX-02 ).
To select initial conditions for the nonadiabatic dynamics, we used data from our previous work on ASR where we modeled pH-dependent absorption spectra. 14  In said work, we extracted 20,000 snapshots (intended as a geometry and a distribution of charges representing the corresponding protonation microstate) from each CpHMD simulation at pH=3, pH=5 and pH=7 and for both isomers, and computed the excitation energy using PM7 to model the electronic structure of the retinal chromophore. Therefore, for each isomer and pH value, we have a full dataset D consisting of 20,000 excitation energies. The initial condition selection procedure was conducted as follows: 1. We computed the average λD max and standard deviation σ D for the excitation energies in D. 2. We created a smaller dataset d containing 1,000 snapshots for each D.
The QM region is limited to the ASR retinal chromophore; the MM interactions are modeled using the Amber forcefield  38  and the QM/MM ones with an electrostatic embedding approach, i. e. , including MM point charge electrostatic potential in the semi-empirical Hamiltonian, as implemented in COBRAMM. 29  The resulting data sets are labeled as pH3-AT, pH3-13C, pH5-AT, pH5-13C, pH7-AT, and pH7-13C. Notice that about 80% of the trajectories have been used for later analysis, the rest is discarded for reasons explained in Supporting Information, section 1. We analyzed each ensemble of trajectories using three different statistical properties. The first one, the isomerization quantum yield (IQY), characterizes the efficiency of the retinal photoisomerization. In principle, this quantity can be directly compared with the experiment, if the number of trajectories is large enough (i.
Interest-Table  3 : Impact (in %, relative to the value in the protonated form, see Supporting Information) on thop , IQY and fitted time constant τ f ast of the deprotonation of several titrated amino acids, whose deprotonated fractions at the considered pH are indicated as x dep. In the case of D98, the deprotonated dataset size at pH3 for the 13C isomer is too small to allow reliable fitting. ingly, in the present simulations, D120 is 90% deprotonated at pH=3 in the AT conformation, while it is only 17% deprotonated in 13C. This allows us to analyze the deprotonation effect at the same pH value, revealing a conformation-independent 20% increase of the thop. Since this value cannot distinguish between the decay channels, we hypothesize that the D120 deprotonation results in a flatter S 1 potential energy surface at pH=3.
Exploration of most populated protonation microstate and charge state As mentioned in the introduction, currently the standard procedure to model the nonadiabatic dynamics of a protein at a given pH relies on picking and fixing the protonation state of every titratable amino acid, or, in other words, using only one protonation microstate. In this work, we explored the advantages and drawbacks of using a more realistic microstate distribution. The resulting large dataset allows us to compare the results and accuracy of the single microstate picture compared to the full set. We start by examining how representative of the entire system the most populated protonation microstate is (see Figure  7B ). Our CpHMD simulations indicate that at pH values where not many amino acids titrate, the most populated microstate accounts for more than 50% of the population.
Our large dataset also offered the chance to analyze the impact of protonation state changes in individual amino acids. This revealed that some residues, like D57, affect the photochemical properties somewhat drastically. However, trends are sometimes difficult to extract, especially when the changes in the protonation state in one residue are correlated with other residues (i. e. , interacting amino acids). As a matter of fact, such clusters would need to be considered as a poly-acid featuring several effective pK a values that cannot be assigned to any particular amino acid. The "protonation microstate" and the "protein charge state" are two useful ways to conceptualize and investigate pH-dependent properties in complex systems. We showed that the former, while widely used to model proteins, may prove insufficient to accurately describe the problem in certain situations, mainly due to the small fraction of the total population it may represent.
Acknowledgement EP and NF thank the  French Agence Nationale de la Recherche  for funding (grant  ANR-14-CE35-0015-02 , project  FEMTO-ASR ). The  Centre de Calcul Intensif d'Aix-Marseille  is acknowledged for granting access to its high performance computing resources. Computational support and infrastructure was also provided by the  "Centre for Information and Media Technology (ZIM) " at the  Heinrich Heine University.
Hydrological modelling is at the core of most studies related to water, especially for anticipating disasters, managing water resources, and planning adaptation strategies. Consequently, teaching hydrological modeling is an important, but difficult, matter. Teaching hydrological modeling requires appropriate software and teaching material (exercises, projects); however, although many hydrological modeling tools exist today, only few are adapted to teaching purposes. In this article, we present the airGRteaching package, which is an open-source R package relying on the GR rainfall-runoff models. In this package, thanks to a graphical user interface and a limited number of functions, numerous hydrological modelling exercises representing a wide range of hydrological applications are proposed.