dataesr/scientific-paragraphs-categorization huggingface dataset:
CIFAR-10 RESULTS CIFAR-10 dataset consists of 60,000 32 × 32 color images (50,000 training images and 10,000 test images) representing 10 objects class like airplanes, trucks or birds. Images vary greatly within each class. See  [25]  for more information. The baseline network is the one implemented in  [26]  where we have replaced the average pooling layers with max pooling. The network has three convolutional layers with ReLU activation, all followed by max pooling. All filters have size 5×5 and stride 1 with 32 filters at the first two layers and 64 at the last one. The pooling windows have size 3 × 3 and stride 2 at all layers. The last max pooling layer is followed by two fully connected layers before a softmax. The accuracy score obtained using this simple CNN is 74. 44% on the test set.
Only the number of convolutional filters varies from a method to another. We report in table 1 the accuracies obtained for different numbers of parameters for both simple CNN and MaxMin networks. We observe that the MaxMin network systematically outperforms the CNN whatever the number of parameters is with a best performance of 82. 98%. This shows the robustness of our method to different network parametrization and demonstrates the ability of MaxMin networks to exploit filter information. Boosting performances: Current deep models on CIFAR-10 use several kinds of learning tricks such as data augmentation, image preprocessing, dropout, to improve final classification performances. Our MaxMin deep convolutional model may benefit of the same tricks. As the learning becomes quite more complex and much more time consuming, we do perform only once these optimizations for our MaxMin architecture model on CIFAR-10.
We thus apply some translations and flips for data augmentation, zca whitening referenced in  [27]  to preprocess the images, and dropout from  [23]  to boost the learning. We use the same network described above with additional local contrast normalisation layer after each pooling. With this training of our network, we do obtain the accuracy of 90. 03% on the test set. This score beats the results reported by  [8]  that uses a deeper architecture (89%) and reported by  [25]  that uses 8 networks for prediction (88. 79%). On the contrary, our MaxMin scores are obtained with only one network. Our method could also benefit from the use of several nets for prediction in order to reach the performances of recent very deep architectures as  [28]  or  [29]  with an accuracy of 92. 40%.
CONCLUSION In this paper, we present our new deep CNN architecture, MaxMin-CNN, to better encode both positive and negative filter detections in the net. MaxMin strategy aims at preserving and propagating significant negative detection values through the net. This difference with standard CNN enables us to preserve and transfer more information through the network. We evaluate and compare our strategy to classical deep CNNs on two benchmarks CIFAR-10 and MNIST. Results demonstrate that our MaxMin networks perform better than CNNs, whatever the configuration considered. Finally, our MaxMin strategy reaches very good performances on CIFAR-10 outperforming several recent and much more complex deep architectures. The principle is very simple and many deep networks could benefit of this MaxMin strategy with only minor adaptations.
Different from most conventional offshore floating structures, floating wind turbines (FWTs) are relatively small bodies which can exhibit stronger nonlinearities in their dynamic behaviour. Moreover, they are designed with the purpose of maximising the aerodynamic interaction related to wind energy extraction, which gives raise to unusually large aerodynamic load to displacement ratios. This constitutes an important source of dynamic coupling, especially as FWT platforms tend to evolve toward more optimised, lightweight solutions. Characterising the mechanical behaviour of a floating wind turbine for design and verification purposes requires the coupling of wind turbine aerodynamics and control with offshore hydromechanics. The understanding of such coupled dynamics under complex met-ocean loading has recently been the driver of a novel generation of coupled offshore dynamic models designed for the requirements of FWT mechanical simulation, such as FAST [6e8], HAWC2  [8, 9] , FloVAWT  [10] , Simo-Riflex  [8, 11] , and CALHYPSO of EDF R&D, the software used in the present study.
Methodology A program named CALHYPSO (CALcul HYdrodynamique Pour les Structures Offshore) has been developed at EDF R&D that incorporates the aerodynamic, hydrostatic, hydrodynamic, inertial, and mooring forces exerted on a FWT to resolve its dynamics in the time domain. Whilst HAWT aerodynamics are here represented via a reactive thrust module, CALHYPSO also features a fully-fledged double-multiple streamtube representation of VAWT aerodynamics. This simulation tool has been both verified and validated in the past; simple code-to-code verification examples are available in Ref. [35] , whilst the experimental validation of some of the inclination effects described here is scheduled for publication in 2016 (two related articles have been proposed to  [36] ). The combination of modelling methods adopted in this study, constituting a subset of the CALHYPSO framework, will be explained in the present section.
Building these resources, acoustic models and language models for Fongbe ASR becomes a challenging task. For this, we used Kaldi toolkit 2  that has allowed us to train our acoustic models on speech data that we have collected ourselves. For the language modeling, we used SRILM toolkit  3  to built trigram language models that we trained on collected text data. To enhance performance of our ASR, we subsequently transformed the vowels by normalizing different tones of Fongbe. Experiments have shown a significant improvement in the results given by the world error rates (WER). The remainder of this paper is organized as follows. The next section describes the target under-resourced language that is Fongbe. Section 3 describes how text and speech corpora have been collected. Section 4 and 5 focus respectively on language modeling and acoustic modeling.
A. Speech corpus As an audio corpus is not available for Fongbe, we proceeded to the speech signals collection to build the audio data for the system. We thus conducted the tedious task of recording the texts pronounced by native speakers (including 8 women and 20 men) of Fongbe in a noiseless environment. We have recorded at 16Khz 28 native speakers who have spoken around 1500 phrases (from daily living) grouped into 3 categories. A category is read by several speakers and contains texts that are different from contents of other categories. These recordings were made with an android application referred to as LigAikuma  [8]  which is developed by GETALP group of Grenoble's Computer Science Laboratory. Overall, there are around 10 hours of speech data that have been collected. First, we split the data by categories leading to a first configuration FC1: 2 categories for training (8 hours) and 1 category for testing (2 hours).
We used SRILM toolkit to train the two languages models. LM1 and LM2 were trained on 995,338 words (10,095 unigrams) by using the training data from text corpus (1,054,724 words, 33,153 sentences) without utterances used for the speech corpus (5,490 words and 1,500 sentences removed). LM1 was trained with the original texts while LM2 was trained with the modified texts by vowel normalization. To represent the uncertainty of our language models, we calculate the perplexity values of all the utterance transcriptions from speech corpus that are not contained in the various text corpus and which represents our test data to evaluate the performance of the two language models. Table  V  shows the perplexity values. The vowel normalization after the original text modification has positive impact on the quality of the language model by reducing in the OOV from 9.
V. ACOUSTIC MODELING In this section, we describe the methods that we used for training and testing our 2 configurations (FC1 and FC2) and present in the next section the obtained results. The recordings and their transcriptions are used for acoustic modeling. The Acoustics models (AMs) are trained and tested on acoustic data from both FC1 and FC2 by using Kaldi acoustic modeling scripts that we have adapted to produce Kaldi scripts for Fongbe. We not only explored AM training methods but also experimented the impact of presence of tones in the utterances transcription from speech corpus by using LM1 (with tones) or LM2 (no tones). Thus, FC1 and FC2 training are performed not only with the same scripts but also by using both pronunciation dictionary. The pronunciation dictionary based grapheme that is used with LM1 contains 49 graphemes while the dictionary used with LM2 contains 28 graphemes.
VI. EXPERIMENTAL RESULTS The experiments focus on comparing the quality of ASR hypothesis measured by WER on AMs trained by different methods. To obtain the best path, we followed the standard The experiments were performed first on LM1 built with the original texts and using both speech data configurations. Then we conducted experiments based on the same procedures on LM2 including texts without diacritics. The interest is to measure the impact of using diacritics in language modelling from the results given by the WER. We also showed how the data speech configuration influence the quality of AMs measured by WER.
VII. CONCLUSION In this work we introduced the first system of Fongbe continuous speech recognition by training different acoustic models using Kaldi scripts and different language models using SRILM toolkit. We also demonstrated the effect of tones on the quality of the recognition system. This leads us to conclude that with the current state of our system, the language modelling without diacritics improves significantly the recognition performances by decreasing the WER by 15. 23% for speech data divided by speakers and 28. 93% for speech data divided by category. Using the Kaldi recipe and the language resources we provide, researcher can build a Fongbe recognition system with the same WER obtained in this paper. For future work, firstly we will enhance the speech and text data and introduce other training techniques to further improve the performance of this first system of Fongbe recognition.
Conventional analysis of functional magnetic resonance imaging (fMRI) data using the general linear model (GLM) employs a neural model convolved with a canonical hemodynamic response function (HRF) peaking 5 s after stimulation. Incorporation of a further basis function, namely the canonical HRF temporal derivative, accounts for delays in the hemodynamic response to neural activity. A population that may benefit from this flexible approach is children whose hemodynamic response is not yet mature. Here, we examined the effects of using the set based on the canonical HRF plus its temporal derivative on both first-and second-level GLM analyses, through simulations and using developmental data (an fMRI dataset on proprioceptive mapping in children and adults). Simulations of delayed fMRI first-level data emphasized the benefit of carrying forward to the second-level a derivative boost that combines derivative and nonderivative beta estimates.
Therefore, the goal of this paper is to investigate the effect of using the informed basis set to model the fMRI signal in analysis of task-based brain activation, especially in the context of special populations such as children. Our objective is not to derive complex methodological advances, but to provide elements to consider for the users of standard fMRI analysis software suites, especially SPM. As a first step, we examined the overall impact of using the informed basis set on individual simulated data. This simulation step was intended to show the benefit of translating to the second-level of analysis the derivative boost and not only the parameter estimate related to the canonical HRF. Second, we fitted the canonical HRF alone and the informed basis set (while varying the time-topeak ranges of interest) to children and adults' data acquired during sensory (proprioceptive) stimulations.
Specifically, they are combined voxel-wise into an estimator (e. g. , t-statistic) relating the overall effect size (mean parameter estimate across subjects) to the between-subject variability, which is used to make an inference about the significance of the effect at the voxel or the cluster level (e. g. , whether or not there is voxelwise or clusterwise activation in case of a one-sample t-test). While the random effect model via summary statistics is in widespread use and is the default procedure in SPM, the generalized mixed-effect model is implemented by default in other software packages (e. g. , FSL) to account for settings where first level variances would not be homogeneous (more generally the sphericity assumption not met), by relating the variance associated with the group effect estimate to a mixture of both within-and between-subject variability  (Mumford and Nichols, 2006, 2009)   1.
Simulations A slow event-related design dataset including 10 s-events interspersed by 10 s-rest periods was simulated using Matlab codes adapted from  Pernet (2014). Specifically, the simulated BOLD time series, y(t), was obtained by convolving hypothetical neural events of various height [i. e. , u(t), the boxcar function] with the standard HRF [i. e. , h(t), the double-gamma function] using a time resolution of 0. 5 s. Our simulation mimicked a slow event-related design, specifically a periodic event presented at 0. 05 Hz, to approach the experimental design described later. To demonstrate the impact of adding one basis function on parameter estimates and model accuracy in case of temporal variability, events were afterwards modeled with temporal shifts (from -2 to 2 s, time step of 0. 5 s) relative to the design matrix and convolved by the HRF alone vs.
Experimental Data Data are from a sensory experiment during which participants experienced vibrations of the tibialis anterior muscles (i. e. , proprioceptive stimulation). The sample included 17 adults (mean age ± SD: 32. 2 ± 4. 5 years; 9 females) and 19 children (mean age ± SD: 8. 7 ± 1. 2 years; 11 females). Adult participants and parents of minors who participated in the study gave written informed consent. The study was approved by the research ethics committee CPP Sud-Méditerranée 1. The scanning session was composed of five runs including 12-s long conditions (epochs) of vibration on the right and left tendons at 30 Hz (R30, L30) or 100 Hz (R100, L100). Each vibration condition was repeated three times per run. The order of vibration conditions was randomized within a run and REST epochs (12-s long on average) were inserted between all vibration conditions.
Complete details on the protocol are provided in  Cignetti et al. (2014). fMRI time series were acquired using a 3-T fMRI scanner (Medspec 30/80 AVANCE, Bruker, Ettlingen, Germany) with a T2 * -weighted gradient echoplanar imaging sequence (42 interleaved axial slices acquisition; 3 mm thickness; 0. Image preprocessing and statistical fMRI data analysis were conducted with SPM8 (Wellcome Department of Imaging Neuroscience, London, UK) running in Matlab 7. 5 environment (Mathworks Inc. , Sherbon, MA, USA) and custom-made Matlab scripts. Each run included 113 images, including 6 dummy images acquired before magnetic field saturation was reached, which were discarded. The remaining images were (i) slice-time corrected, (ii) realigned to the first image of the time series to correct for head movement between scans, (iii) unwarped to remove residual movement-related variance  (Andersson et al.
The core code (spmup_hrf_boost. m) we used is available at the GitHub repository: https://github. com/CPernet/spmup/blob/ master/spmup_hrf_boost. m. One-sample t-tests using the first-level 100 Hz and 100 minus 30 Hz contrast images were finally conducted both in children and adults, for each of the three modeling approaches (i. e. , canonical HRF and basis set with temporal restriction or not). This was intended to evaluate the extent to which average group activation (reflecting here sensory mapping) was impacted by using the derivative boost. In practice, second-level onesample t-test is implemented in SPM following a random-effects approach via summary statistics, which pools voxelwise the firstlevel contrast estimates c β 1 ,. , c β N from N subjects into the t-statistic t = c β G / Var c β G (5) where c β G and Var c β G represent the average group contrast estimate and the across-subject variance in contrast estimate, respectively (e.
The camera was controlled by a computer through DSLR Remote Pro software. From each captured image, sperm nuclear morphometry was automatically analyzed by ImageJ open software (available on-line at http://rsbweb. nih. gov/ij/download. html). 7 Each sperm nucleus was measured for four primary parameters and four derived parameters for nuclear shape. Primary parameters were Area (A, µm 2 , as the sum of all pixel areas contained within the boundary), Perimeter (P, µm, as the sum of external boundaries), Length (L) and Width (W, µm, the highest and lowest values, respectively, of the Feret diameters, i. e. , the projection of the sperm head on the horizontal axis measured at angles of rotation of 0°, 30°, 60°, 90°, 120°, and 150°). Derived nuclear shape parameters were Ellipticity (L/W), Rugosity (4πA/P 2 ), Elongation ([L -W]/[L + W]), and Regularity (πLW/4A).
Statistical analysis Statistical analyses were performed by using the SPSS package, version 15. 0 (SPSS Inc. , Chicago, IL, USA). Normality distributions and variance homogeneity of the median value score for each set were checked by the Kolmogorov-Smirnov and Levene tests, respectively. As all data were normally distributed, parametric tests were used throughout. In Experiment 1, discriminant analysis was performed with the linear stepwise procedure to identify the most useful parameters for the classification of SX and SY spermatozoa. Variables were added one by one to the discriminating functions until the addition of a new variable did not give a better discrimination. Wilk's lambda was used to compare the fraction of the total dispersion of data not accounted for. Both in Experiments 1 and 2, differences in sperm nuclear morphometric parameters between groups were examined through analysis of variance (ANOVA) by using generalized linear models.
CONCLUSION The fluorescence-based CASA-Morph method has the potential to distinguish differences between X-and Y-chromosome-bearing spermatozoa in the bovine species although more studies are needed to increase the precision of sex determination by this technique. These studies should be orientated toward the reduction of the factors other than the gonosomes potentially affecting sperm morphometry, improvements in the equipment and data analysis. fluorescence intensity between SX and SY although more studies are needed for a successful classification of spermatozoa according to their sex. 5] [6]  Discrepancies might be attributable to the different techniques of sperm sex characterization and sperm morphometric evaluation. In the present study, significant differences in sperm nuclear morphometry between SX and SY were obtained with CASA-Morph after sex determination by FISH.
Different solution concepts can be proposed by individual designers, or design groups, according to their background knowledge and professional experience.  Secondly, how to evaluate the different solution concepts proposed.  Finally, because of the complexity of a CPS, it is impossible to find the optimum solution concepts without iteration and there is a requirement to optimise solution concepts during the early part of the design process. The Systems Modelling Language (SysML), has been established by the International Council of Systems Engineering (INCOSE) and the Object Management Group (OMG)  [OMG09]  based on unified modelling language (UML) to support Model Based Systems Engineering (MBSE). Other modelling languages have been proposed to model a CPS. Venter and Ehlers  [VeEh12]  believe that it is not possible to formally describe dynamic a CPS due to the flexibility in which incomplete UML diagrams can be defined, so they propose an Architecture Description Language (ADL) for dynamically modelling architectures with specific reference to CPSs.
MDE of Self-configuration agent-based CPPS To achieve an intelligent reconfigurable CPPS, a model based approach may well be beneficial [Vya11, Thr13, LZV+11, DOB+13, LeRo12, BFS12, CPdA+12, PSL11, LSVH13, SWL+13]. Here we introduce an SysML based approach to developing self-configuration abilities of field level functions using indirect redundant information or devices, i. e. in case of a fault, the system is able to reconfigure and continue operation to avoid down time using already available neighboured values or related values and devices. In the SysML-based approach, single components of a CPPS are modelled as an agent using the SysML element Block to encapsulate an agent's description model. This model covers information that describes the basic functionality a particular software component fulfils within the CPPS, e.
Since the diagnosis algorithm has been developed to work independently from the number of devices in a component (cf. [SWL+13] ) it does not need to be generated from the SysML model but can be placed inside the software component as static code (cf. Figure  10 , right). In the last step, the implementation code that realizes the basic functionality of a CPPS component is generated from its description, e. g. inside a State Chart. The required SysML model will be discussed in the following paragraphs and illustrated by an application example from process industry. This considers a tank (see Figure  11 ) equipped with two sensors to detect the maximum and minimum filling levels. The tank is used to store liquids in a production process Once the minimum filling level of the tank is reached (detected by the lower tank sensor, sensor 101.
The real level sensor can be approximated by a virtual sensor calculated during runtime. If the filling level sensor is effected by a fault, the faulty situation may be detected. At first the redundancy model has to be developed in the design phase by taking functional dependencies into account and formulated as physical equation (cf. Figure  12 , left). To map the redundancy model onto the SysML based notation of the approach using the Parametric Diagram (PAR), each real sensor of the plant (Figure  12 , left, ovals) is modelled as a Block inside the PAR. For the physical equations which represent the functional dependency between the sensor values a Block has also to be modelled inside the PAR. In the code generation process, both types of Block are translated into specific sections of the generated software component that either handle the pre-processing of real sensor values or calculate redundant sensor values based on the modelled physical equations.
Since the redundancy model has to be implemented on the PLC to allow decisions during runtime, the code generation of the SysML based approach supports (cf. Figure  9 ) the transformation from the information modelled within the PAR into software components according to IEC 61131-3, the leading standard for PLC programming and the standard hardware platform for automated Production systems. As the basic step of the code generation process an IEC 61131-3 Function Block (FB) is created as from the agent modelled in SysML for the software component that is implemented inside the CPPS (Figure  9 , right). The process of the code generation consists of four further steps to completing the generation of the FB (cf. Figure  9 , middle). As the first step the necessary declarations for the virtual sensors and real sensors are generated and a redundancy matrix created as a code variable array inside the declaration part of the FB.