Miracle-dz/UniFAR-FTUs huggingface dataset:
A Compact and Configurable Long Short-Term Memory Neural Network Hardware Architecture

Neural network has been one of the most useful techniques in the area of image analysis and speech recognition in recent years. Long Short-Term Memory (LSTM), a popular type of recurrent neural networks (RNNs), has widely been implemented on CPUs and GPUs. However, software implementation cannot offer large parallelism for the complicated computation of LSTM, and most of the LSTM hardware implementations proposed yet are intensive and non-configurable. In order to accelerate the computation and reduce the resources consumption, in this work, we present a compact and configurable LSTM neural network hardware architecture. To meet the requirements of different networks, we set a wide array of hardware parameters that can be configured to balance area, power and performance. And we adopt the second-order polynomial to approximate the activation functions in LSTM, which balances the computational accuracy and resource utilization, Implemented on XCZU6EG FPGA running at 238 MHz, our work has a performance of 7.64 GOP/s. Compared to the implementation on Intel Xeon E5-2620 CPU at 2.10 GHz, our parallel hardware architecture achieves $90\times$ speedup for a small network and 25 x speed-up for a large one. The total consumption of resources is 77% less than the state-of-the-art works', which implies the compactness of our work.
Articulatory and Stacked Bottleneck Features for Low Resource Speech Recognition

In this paper, we discuss the beneﬁts of using articulatory and stacked bottleneck features (SBF) for low resource speech recognition. Articulatory features (AF) which capture the underlying attributes of speech production are found to be robust to channel and speaker variations. However, building an efﬁcient articulatory classiﬁer to extract AF requires an enormous amount of data. In low resource acoustic modeling, we propose to train the bidirectional long short-term memory (BLSTM) articulatory classiﬁer by pooling data from the available low resource Indian languages, namely, Gujarati, Tamil, and Telugu. This is done in the context of Microsoft Indian Language challenge. Similarly, we train a multilingual bottleneck feature extractor and an SBF extractor using the pooled data. To bias, the SBF network towards the target language, a second network in the stacked architecture was trained using the target language alone. The performance of ASR system trained with stand-alone AF is observed to be at par with the multilingual bottleneck features. When the AF and the biased SBF are appended, they are found to outperform the conventional ﬁlterbank features in the multilingual deep neural network (DNN) framework and the high-resolution Mel frequency cepstral coefﬁcient (MFCC) features in the time-delayed neural network(TDNN) framework.
Margin-Based Sample Filtering for Image Classification Using Convolutional Neural Networks

Deep convolutional neural networks have become the state of the art methods for image classification after demonstrating very good performance on very large datasets with general visual content. Amongst the problems for training deep CNN architectures is the heavy computational cost and the large memory requirements. In this work we exploit the fact that many training samples are correctly classified in the early stages of learning and therefore they can be skipped in further training epochs without noticeable performance degradation. We employ sample filtering to determine which samples are used in each epoch. We introduce the “multi-class margin” index to measure how safely a sample is classified to the correct class. The multi-class margin is closely connected with the output of the final Softmax layer. Using filtering we gradually reduce the number of usable samples to 40% of the original dataset. At the same time we gain around 18% in time compared to using all the samples in every epoch, while the performance degradation is around 1%.
Acoustic and Perceptual Characteristics of Mandarin Speech in Homosexual and Heterosexual Male Speakers

The present study investigated both acoustic and perceptual characteristics of Mandarin speech in homosexual and heterosexual male speakers. Acoustic analyses of monosyllabic words showed significant differences between the two groups in F0 features (including the mean, the max, and the range), F1 and F2 of vowels, aspiration/frication duration of consonants, and center of gravity as well as skewness for /s/. Especially, the patterns were found to be opposite between Mandarin and American English speakers, which might be due to social psychological differences between the two societies. The perceptual experiment showed that the perceived score of gayness differed significantly between the speeches of the two groups. Among those acoustic parameters showing significant differences, fricative duration may be the most salient cue for sexual orientation of Mandarin male speakers.
Low Self-Esteem for Physical Appearance Mediates the Effect of Body Mass Index on Smoking Initiation Among Adolescents

Objective Adolescence is a period during which youth may begin experimenting with substances. Youth with overweight or obesity may be at increased risk for substance use, including cigarette smoking. Understanding the associations between smoking and excess weight and the pathways associated with increased likelihood for smoking initiation is of particular importance given the increased risk for negative health outcomes associated with each. Methods Using longitudinal panel data from 1,023 middle school youth (baseline age M = 12.5, 52% female), we tested whether smoking initiation was concurrently and prospectively predicted by self-reported body mass index (BMI) and whether self-esteem for physical appearance (SEPA) mediated the effect of BMI on risk of early initiation. Results BMI predicted smoking initiation concurrently and prospectively in unadjusted models. In adjusted models, SEPA mediated the effects of BMI on smoking initiation. Bootstrapped mediation results indicated that the positive relationship between BMI and subsequent smoking initiation was significantly mediated by lower SEPA (B =.10, 95% confidence interval [0.01, 0.22]). Conclusions Adolescents who have overweight or obesity are more likely to feel negatively about their appearance and bodies, and this negative perception may result in experimentation with cigarettes. Cigarettes may be used by youth with overweight or obesity in an effort to manage weight, to cope with low self-esteem or for other reasons. Future research should explore the motives and psychosocial context of smoking initiation among adolescents with overweight/obesity further (e.g., with whom they first try smoking, perceived benefits of smoking).
Combined Speaker Clustering and Role Recognition in Conversational Speech

Speaker Role Recognition (SRR) is usually addressed either as an independent classiﬁcation task, or as a subsequent step after a speaker clustering module. However, the ﬁrst approach does not take speaker-speciﬁc variabilities into account, while the second one results in error propagation. In this work we propose the integration of an audio-based speaker clustering algorithm with a language-aided role recognizer into a meta-classiﬁer which takes both modalities into account. That way, we can treat separately any speaker-speciﬁc and role-speciﬁc characteristics before combining the relevant information together. The method is evaluated on two corpora of different conditions with interactions between a clinician and a patient and it is shown that it yields superior results for the SRR task.
Two-Stream Designed 2D/3D Residual Networks with Lstms for Action Recognition in Videos

Convolutional Neural Networks(CNNs) have achieved great success for object recognition in still images. However, CNNs can't make evident improvement for action recognition in videos, one reason is that many current network architectures are relatively shallow compared with deep models in image domain, and the other reason is that CNNs can't capture effective long-term motion information from videos. Encouraged by the good performance of Residual Network-s(ResNets) for training extremely deep models, and Long-term Recurrent Convolutional Networks(LSTMs) for dealing with tasks involving sequences, we presented an action recognition method based on a two-stream architecture, with 2D ResNets with LSTMs in one stream and designed 3D ResNets with LSTMs in the other stream, which can combine appearance and motion information better. Especially, our proposed method first learns spatiotemporal features of videos through the Residual networks, then models complex temporal dynamics by the Long-term Recurrent Convolutional networks, and with a softmax layer on the top of two streams, the final classification results can be predicted by fusing scores of each stream with weights on score distribution. Furthermore, for better reducing the influence of redundant background information in videos for recognition results, we also applied a center extraction method to generate central regions of videos instead of an entire video into a visual representation. On two video action benchmarks of UCF101 and HMDB51, our method achieved promising performance compared with state-of-the-art.
Implementation of Digital Hearing Aid as a Smartphone Application

Hearing aids for persons with sensorineural hearing loss aim to compensate for degraded speech perception caused by frequency-dependent elevation of hearing thresholds, reduced dynamic range, abnormal loudness growth, and increased temporal and spectral masking. A digital hearing aid is implemented as a smartphone application as an alternative to ASIC-based hearing aids. The implementation provides user-configurable processing for background noise suppression and dynamic range compression. Speech enhancement technique using spectral subtraction based on geometric approach and noise spectrum estimation based on dynamic quantile tracking is implemented to improve speech perception. To compensate for reduced dynamic range and frequency-dependent elevation of hearing thresholds, a sliding-band dynamic range compression technique is used. Both processing blocks are implemented for real-time processing using single FFT-based analysis-synthesis. Implementation as a smartphone application has been carried out using Nexus 5X with Android 7.1 Nougat OS. A touch-controlled graphical user interface enables the user to fine tune the processing parameters in an interactive and real-time mode. The audio latency is 45 ms, making it suitable for face-to-face communication.
DNN-based Speech Synthesis for Small Data Sets Considering Bidirectional Speech-Text Conversion

In statistical parametric speech synthesis, approaches based on deep neural networks (DNNs) have improved qualities of the synthesized speech. General DNN-based approaches require a large amount of training data to synthesize natural speech. However, it is not practical to record speech for many hours from a single speaker. To address this problem, this paper presents a novel pre-training method of DNN-based speech synthesis systems for small data sets. In this method, a Gaussian-Categorical deep relational model (GCDRM), which represents a joint probability of two visible variables, is utilized to describe the joint distribution of acoustic features and linguistic features. During the maximum-likelihood-based training, the model attempts to obtain parameters of a deep architecture considering the bidirectional conversion between 1) generated acoustic features given linguistic features and 2) re-generated linguistic features given acoustic features generated from itself. Owing to considering whether the generated acoustic features are recognizable, our method can obtain reasonable parameters from small data sets. Experimental results show that pre-trained DNN-based systems using our proposed method outper-formed randomly-initialized DNN-based systems. This method also outperformed DNN-based systems in a speaker-dependent speech recognition task.
The INTERSPEECH 2018 Computational Paralinguistics Challenge: Atypical & Self-Assessed Affect, Crying & Heart Beats

The INTERSPEECH 2018 Computational Paralinguistics Challenge addresses four different problems for the ﬁrst time in a research competition under well-deﬁned conditions: In the Atypical Affect Sub-Challenge, four basic emotions annotated in the speech of handicapped subjects have to be classiﬁed; in the Self-Assessed Affect Sub-Challenge, valence scores given by the speakers themselves are used for a three-class classiﬁ-cation problem; in the Crying Sub-Challenge, three types of infant vocalisations have to be told apart; and in the Heart Beats Sub-Challenge, three different types of heart beats have to be determined. We describe the Sub-Challenges, their conditions, and baseline feature extraction and classiﬁers, which include data-learnt (supervised) feature representations by end-to-end learning, the ‘usual’ ComParE and BoAW features, and deep unsupervised representation learning using the AU D EEP toolkit for the ﬁrst time in the challenge series.
Feature Representation of Short Utterances Based on Knowledge Distillation for Spoken Language Identification

The performance of spoken language identiﬁcation (LID) on short utterances is drastically degraded even though model is completely trained on short utterance data set. The degradation is because of the large pattern confusion caused by the large variation of feature representation on short utterances. In this paper, we propose a teacher-student network learning algorithm to explore discriminative features for short utterances. With the teacher-student network learning, the feature representation for short utterances (explored by the student network) are normalized to their representations corresponding to long utterances (provided by the teacher network). With this learning algorithm, the feature representation on short utterances is supposed to reduce pattern confusion. Experiments on a 10-language LID task were carried out to test the algorithm. Our results showed the proposed algorithm signiﬁcantly improved the performance.
Identifying unmaintained projects in github

Background: Open source software has an increasing importance in modern software development. However, there is also a growing concern on the sustainability of such projects, which are usually managed by a small number of developers, frequently working as volunteers. Aims: In this paper, we propose an approach to identify GitHub projects that are not actively maintained. Our goal is to alert users about the risks of using these projects and possibly motivate other developers to assume the maintenance of the projects. Method: We train machine learning models to identify unmaintained or sparsely maintained projects, based on a set of features about project activity (commits, forks, issues, etc). We empirically validate the model with the best performance with the principal developers of 129 GitHub projects. Results: The proposed machine learning approach has a precision of 80%, based on the feedback of real open source developers; and a recall of 96%. We also show that our approach can be used to assess the risks of projects becoming unmaintained. Conclusions: The model proposed in this paper can be used by open source users and developers to identify GitHub projects that are not actively maintained anymore.
Turning Vice into Virtue: Using Batch-Effects to Detect Errors in Large Genomic Data Sets

Abstract It is often unavoidable to combine data from different sequencing centers or sequencing platforms when compiling data sets with a large number of individuals. However, the different data are likely to contain specific systematic errors that will appear as SNPs. Here, we devise a method to detect systematic errors in combined data sets. To measure quality differences between individual genomes, we study pairs of variants that reside on different chromosomes and co-occur in individuals. The abundance of these pairs of variants in different genomes is then used to detect systematic errors due to batch effects. Applying our method to the 1000 Genomes data set, we find that coding regions are enriched for errors, where ∼1% of the higher frequency variants are predicted to be erroneous, whereas errors outside of coding regions are much rarer (<0.001%). As expected, predicted errors are found less often than other variants in a data set that was generated with a different sequencing technology, indicating that many of the candidates are indeed errors. However, predicted 1000 Genomes errors are also found in other large data sets; our observation is thus not specific to the 1000 Genomes data set. Our results show that batch effects can be turned into a virtue by using the resulting variation in large scale data sets to detect systematic errors.
Compressing End-to-end ASR Networks by Tensor-Train Decomposition

End-to-end deep learning has become a popular framework for automatic speech recognition (ASR) tasks, and it has proven itself to be a powerful solution. Unfortunately, network structures commonly have millions of parameters, and large computational resources are required to make this approach feasible for training and running such networks. Moreover, many applications still prefer lightweight models of ASR that can run efﬁciently on mobile or wearable devices. To address this challenge, we propose an approach that can reduce the number of ASR parameters. Speciﬁcally, we perform Tensor-Train decomposition on the weight matrix of the gated recurrent unit (TT-GRU) in the end-to-end ASR framework. Experimental re-sults on LibriSpeech data reveal that the compressed ASR with TT-GRU can maintain good performance while greatly reducing the number of parameters.
Unsupervised Vocal Tract Length Warped Posterior Features for Non-Parallel Voice Conversion

In the non-parallel Voice Conversion (VC) with the I terative combination of N earest Neighbor search step and C onversion step A lignment (INCA) algorithm, the occurrence of one-to-many and many-to-one pairs in the training data will deteriorate the performance of the stand-alone VC system. The work on handling these pairs during the training is less explored. In this paper, we establish the relationship via intermediate speaker-independent posteriorgram representation, instead of directly mapping the source spectrum to the target spectrum. To that effect, a Deep Neural Network (DNN) is used to map the source spectrum to posteriorgram representation and another DNN is used to map this posteriorgram representation to the target speaker’s spectrum. In this paper, we propose to use unsupervised Vocal Tract Length Normalization (VTLN)- based warped Gaussian posteriorgram features as the speaker-independent representations. We performed experiments on a small subset of publicly available Voice Conversion Challenge (VCC) 2016 database. We obtain the lower Mel Cepstral Distortion (MCD) values with the proposed approach compared to the baseline as well as the supervised phonetic posterior-gram feature-based speaker-independent representations. Furthermore, subjective evaluation gave relative improvement of 13.3 % with the proposed approach in terms of Speaker Similarity (SS).
Acoustic-prosodic Entrainment in Structural Metadata Events

This paper presents an acoustic-prosodic analysis of entrainment in a Portuguese map-task corpus. Our aim is to analyze how turn-by-turn entrainment varies with distinct structural metadata events: types of sentence-like units (SU) in consecutive turns (e.g. interrogatives followed by declaratives, or both declaratives), and with the presence of discourse markers, afﬁrmative cue words, and disﬂuencies in the beginning of turns. Entrainment at turn-exchanges may be observed in terms of pitch, energy, duration, and voice quality. Regarding SU types, question-answer turns are the ones with stronger similarity, and declarative-interrogative pairs are the ones where less entrainment occurs, as expected. Moreover, in question-answer pairs, there is also stronger evidence of entrainment with Yes/No and Tag questions than with Wh- questions. In fact, these subtypes are coded in distinctive prosodic ways (moreover, the ﬁrst subtype has no associated lexical-syntactic cues in Portuguese, only prosodic). As for turn-initial structures, entrainment is stronger when the second turn begins with an afﬁrmative cue word; less strong with ambiguous structures (such as ‘OK’), emphatic afﬁrmative answers, and negative answers; and scarce with disﬂuencies and discourse markers. The different degrees of local entrainment may be related with the informative structure of distinct structural metadata events.
Examining gender health inequalities in Europe using a Synthetic Health Indicator: the role of family policies

BACKGROUND Family policies have been shown to affect men's and women's health throughout the European Union, though particularly in the case of women. This paper seeks to analyze this effect by creating two Synthetic Health Indicators (SHIs) differentiated by gender applying the P2 distance method. We also aim to compare the results obtained by means of our SHI and the Self-rated health (SRH) indicator, as it is the most common indicator used to approach individuals' health. METHODS The 2012 European Quality of Life Survey was used to construct the SHI. Dimensions such as physical health, mental health, psychosocial resources, the capacity of health services and community health status were included in our SHI. All proved to be relevant when measuring individuals' health. RESULTS Gender health inequalities differed depending on the indicator used in each European country. When considering family policy, we found significant gender health gaps that favoured men in traditional countries, regardless of the indicator used (SHI or SRH). Moreover, with our SHI we found significant gender health inequalities in market-oriented and dual-earner countries that were not revealed by the SRH indicator, highlighting the fact that current equality policies are necessary but insufficient to overcome the gender health gap. CONCLUSIONS The SHI was more sensitive to gender differences and evidenced certain advantages compared to the SRH. The SHI includes a wide range of variables that incorporated large amounts of fresh information, allowing men and women to have a personalized design of their own health.
Gated Square-Root Pooling for Image Instance Retrieval

Recently Convolutional Neural Networks (CNNs) have achieved great success in different fields including image instance retrieval. However traditional global pooling approaches fail to capture all possible discriminative information of CNN activations and treat activations over channels equally regardless of the different importance between channels. In this work, we focus on the mentioned problem of global feature pooling over CNN activations for image instance retrieval. We make two contributions. First, we introduce a channel-wise SQUare-root (SQU) pooling (2-norm) approach, which makes better use of information over activation maps and is superior to Average (1-norm) and Max pooling (infinity norm), in the context of instance retrieval. Second, we further improve SQU by learning a gating function that weights the contributions of different channels, in an end-to-end manner. Extensive experiments on 6 benchmark datasets show that the proposed strategies achieve considerable improvements over state-of-the-art.