OpenRaiser/Intern-Atlas huggingface dataset:
ORGB: Offset correction in RGB color space for illumination-robust image processing
Venue: Conference ICASSP 2017
Year: 2017

Single materials have colors which form straight lines in RGB space. However, in severe shadow cases, those lines do not intersect the origin, which is inconsistent with the description of most literature. This paper is concerned with the detection and correction of the offset between the intersection and origin. First, we analyze the reason for forming that offset via an optical imaging model. Second, we present a simple and effective way to detect and remove the offset. The resulting images, named ORGB, have almost the same appearance as the original RGB images while are more illumination-robust for color space conversion. Besides, image processing using ORGB instead of RGB is free from the interference of shadows. Finally, the proposed offset correction method is applied to road detection task, improving the performance both in quantitative and qualitative evaluations.
Permutation invariant training of deep models for speaker-independent multi-talker speech separation
Venue: Conference ICASSP 2017
Year: 2017

We propose a novel deep learning model, which supports permutation invariant training (PIT), for speaker independent multi-talker speech separation, commonly known as the cocktail-party problem. Different from most of the prior arts that treat speech separation as a multi-class regression problem and the deep clustering technique that considers it a segmentation (or clustering) problem, our model optimizes for the separation regression error, ignoring the order of mixing sources. This strategy cleverly solves the long-lasting label permutation problem that has prevented progress on deep learning based techniques for speech separation. Experiments on the equal-energy mixing setup of a Danish corpus confirms the effectiveness of PIT. We believe improvements built upon PIT can eventually solve the cocktail-party problem and enable real-world adoption of, e.g., automatic meeting transcription and multi-party human-computer interaction, where overlapping speech is common.
A comprehensive study of deep bidirectional LSTM RNNS for acoustic modeling in speech recognition
Venue: Conference ICASSP 2017
Year: 2017

We present a comprehensive study of deep bidirectional long short-term memory (LSTM) recurrent neural network (RNN) based acoustic models for automatic speech recognition (ASR). We study the effect of size and depth and train models of up to 8 layers. We investigate the training aspect and study different variants of optimization methods, batching, truncated backpropagation, different regularization techniques such as dropout and $L_2$ regularization, and different gradient clipping variants. The major part of the experimental analysis was performed on the Quaero corpus. Additional experiments also were performed on the Switchboard corpus. Our best LSTM model has a relative improvement in word error rate of over 14\% compared to our best feed-forward neural network (FFNN) baseline on the Quaero task. On this task, we get our best result with an 8 layer bidirectional LSTM and we show that a pretraining scheme with layer-wise construction helps for deep LSTMs. Finally we compare the training calculation time of many of the presented experiments in relation with recognition performance. All the experiments were done with RETURNN, the RWTH extensible training framework for universal recurrent neural networks in combination with RASR, the RWTH ASR toolkit.
Image denoising via group sparsity residual constraint
Venue: Conference ICASSP 2017
Year: 2017

Group sparsity has shown great potential in various low-level vision tasks (e.g, image denoising, deblurring and inpainting). In this paper, we propose a new prior model for image denoising via group sparsity residual constraint (GSRC). To enhance the performance of group sparse-based image denoising, the concept of group sparsity residual is proposed, and thus, the problem of image denoising is translated into one that reduces the group sparsity residual. To reduce the residual, we first obtain some good estimation of the group sparse coefficients of the original image by the first-pass estimation of noisy image, and then centralize the group sparse coefficients of noisy image to the estimation. Experimental results have demonstrated that the proposed method not only outperforms many state-of-the-art denoising methods such as BM3D and WNNM, but results in a faster speed.
On classification of distorted images with deep convolutional neural networks
Venue: Conference ICASSP 2017
Year: 2017

Image blur and image noise are common distortions during image acquisition. In this paper, we systematically study the effect of image distortions on the deep neural network (DNN) image classifiers. First, we examine the DNN classifier performance under four types of distortions. Second, we propose two approaches to alleviate the effect of image distortion: re-training and fine-tuning with noisy images. Our results suggest that, under certain conditions, fine-tuning with noisy images can alleviate much effect due to distorted inputs, and is more practical than re-training.
Encoder-decoder with focus-mechanism for sequence labelling based spoken language understanding
Venue: Conference ICASSP 2017
Year: 2017

This paper investigates the framework of encoder-decoder with attention for sequence labelling based spoken language understanding. We introduce Bidirectional Long Short Term Memory - Long Short Term Memory networks (BLSTM-LSTM) as the encoder-decoder model to fully utilize the power of deep learning. In the sequence labelling task, the input and output sequences are aligned word by word, while the attention mechanism cannot provide the exact alignment. To address this limitation, we propose a novel focus mechanism for encoder-decoder framework. Experiments on the standard ATIS dataset showed that BLSTM-LSTM with focus mechanism defined the new state-of-the-art by outperforming standard BLSTM and attention based encoder-decoder. Further experiments also show that the proposed model is more robust to speech recognition errors.
Advances in all-neural speech recognition
Venue: Conference ICASSP 2017
Year: 2017

This paper advances the design of CTC-based all-neural (or end-to-end) speech recognizers. We propose a novel symbol inventory, and a novel iterated-CTC method in which a second system is used to transform a noisy initial output into a cleaner version. We present a number of stabilization and initialization methods we have found useful in training these networks. We evaluate our system on the commonly used NIST 2000 conversational telephony test set, and significantly exceed the previously published performance of similar systems, both with and without the use of an external language model and decoding technology.
TaSNet: Time-Domain Audio Separation Network for Real-Time, Single-Channel Speech Separation
Venue: Conference ICASSP 2018
Year: 2018

Robust speech processing in multi-talker environments requires effective speech separation. Recent deep learning systems have made significant progress toward solving this problem, yet it remains challenging particularly in real-time, short latency applications. Most methods attempt to construct a mask for each source in time-frequency representation of the mixture signal which is not necessarily an optimal representation for speech separation. In addition, time-frequency decomposition results in inherent problems such as phase/magnitude decoupling and long time window which is required to achieve sufficient frequency resolution. We propose Time-domain Audio Separation Network (TasNet) to overcome these limitations. We directly model the signal in the time-domain using an encoder-decoder framework and perform the source separation on nonnegative encoder outputs. This method removes the frequency decomposition step and reduces the separation problem to estimation of source masks on encoder outputs which is then synthesized by the decoder. Our system outperforms the current state-of-the-art causal and noncausal speech separation algorithms, reduces the computational cost of speech separation, and significantly reduces the minimum required latency of the output. This makes TasNet suitable for applications where low-power, real-time implementation is desirable such as in hearable and telecommunication devices.
Semi-Recurrent Cnn-Based Vae-Gan for Sequential Data Generation
Venue: Conference ICASSP 2018
Year: 2018

A semi-recurrent hybrid VAE-GAN model for generating sequential data is introduced. In order to consider the spatial correlation of the data in each frame of the generated sequence, CNNs are utilized in the encoder, generator, and discriminator. The subsequent frames are sampled from the latent distributions obtained by encoding the previous frames. As a result, the dependencies between the frames are maintained. Two testing frameworks for synthesizing a sequence with any number of frames are also proposed. The promising experimental results on piano music generation indicates the potential of the proposed framework in modeling other sequential data such as video.
Improving End-of-Turn Detection in Spoken Dialogues by Detecting Speaker Intentions as a Secondary Task
Venue: Conference ICASSP 2018
Year: 2018

This work focuses on the use of acoustic cues for modeling turn-taking in dyadic spoken dialogues. Previous work has shown that speaker intentions (e.g., asking a question, uttering a backchannel, etc.) can influence turn-taking behavior and are good predictors of turn-transitions in spoken dialogues. However, speaker intentions are not readily available for use by automated systems at run-time; making it difficult to use this information to anticipate a turn-transition. To this end, we propose a multi-task neural approach for predicting turn- transitions and speaker intentions simultaneously. Our results show that adding the auxiliary task of speaker intention prediction improves the performance of turn-transition prediction in spoken dialogues, without relying on additional input features during run-time.
Can you Find a Face in a HEVC Bitstream?
Venue: Conference ICASSP 2018
Year: 2018

Finding faces in images is one of the most important tasks in computer vision, with applications in biometrics, surveillance, human-computer interaction, and other areas. In our earlier work, we demonstrated that it is possible to tell whether or not an image contains a face by only examining the HEVC syntax, without fully reconstructing the image. In the present work we move further in this direction by showing how to localize faces in HEVC-coded images, without full reconstruction. We also demonstrate the benefits that such approach can have in privacy-friendly face localization.
A Coupled Compressive Sensing Scheme for Unsourced Multiple Access
Venue: Conference ICASSP 2018
Year: 2018

This article introduces a novel paradigm for the unsourced multiple-access communication problem. This divide-and-conquer approach leverages recent advances in compressive sensing and forward error correction to produce a computationally efficient algorithm. Within the proposed framework, every active device first partitions its data into several sub-blocks, and subsequently adds redundancy using a systematic linear block code. Compressive sensing techniques are then employed to recover sub-blocks, and the original messages are obtained by connecting pieces together using a low-complexity tree-based algorithm. Numerical results suggest that the proposed scheme outperforms other existing practical coding schemes. Measured performance lies approximately $4.3$~dB away from the Polyanskiy achievability limit, which is obtained in the absence of complexity constraints.
Query-by-Example Spoken Term Detection Using Attention-Based Multi-Hop Networks
Venue: Conference ICASSP 2018
Year: 2018

Retrieving spoken content with spoken queries, or query-by- example spoken term detection (STD), is attractive because it makes possible the matching of signals directly on the acoustic level without transcribing them into text. Here, we propose an end-to-end query-by-example STD model based on an attention-based multi-hop network, whose input is a spoken query and an audio segment containing several utterances; the output states whether the audio segment includes the query. The model can be trained in either a supervised scenario using labeled data, or in an unsupervised fashion. In the supervised scenario, we find that the attention mechanism and multiple hops improve performance, and that the attention weights indicate the time span of the detected terms. In the unsupervised setting, the model mimics the behavior of the existing query-by-example STD system, yielding performance comparable to the existing system but with a lower search time complexity.
A Robust Machine Learning Method for Cell-Load Approximation in Wireless Networks
Venue: Conference ICASSP 2018
Year: 2018

We propose a learning algorithm for cell-load approximation in wireless networks. The proposed algorithm is robust in the sense that it is designed to cope with the uncertainty arising from a small number of training samples. This scenario is highly relevant in wireless networks where training has to be performed on short time scales because of a fast time-varying communication environment. The first part of this work studies the set of feasible rates and shows that this set is compact. We then prove that the mapping relating a feasible rate vector to the unique fixed point of the non-linear cell-load mapping is monotone and uniformly continuous. Utilizing these properties, we apply an approximation framework that achieves the best worst-case performance. Furthermore, the approximation preserves the monotonicity and continuity properties. Simulations show that the proposed method exhibits better robustness and accuracy for small training sets in comparison with standard approximation techniques for multivariate data.
Framework for Evaluation of Sound Event Detection in Web Videos
Venue: Conference ICASSP 2018
Year: 2018

The largest source of sound events is web videos. Most videos lack sound event labels at segment level, however, a significant number of them do respond to text queries, from a match found using metadata by search engines. In this paper we explore the extent to which a search query can be used as the true label for detection of sound events in videos. We present a framework for large-scale sound event recognition on web videos. The framework crawls videos using search queries corresponding to 78 sound event labels drawn from three datasets. The datasets are used to train three classifiers, and we obtain a prediction on 3.7 million web video segments. We evaluated performance using the search query as true label and compare it with human labeling. Both types of ground truth exhibited close performance, to within 10%, and similar performance trend with increasing number of evaluated segments. Hence, our experiments show potential for using search query as a preliminary true label for sound event recognition in web videos.
Spectral Feature Mapping with MIMIC Loss for Robust Speech Recognition
Venue: Conference ICASSP 2018
Year: 2018

For the task of speech enhancement, local learning objectives are agnostic to phonetic structures helpful for speech recognition. We propose to add a global criterion to ensure de-noised speech is useful for downstream tasks like ASR. We first train a spectral classifier on clean speech to predict senone labels. Then, the spectral classifier is joined with our speech enhancer as a noisy speech recognizer. This model is taught to imitate the output of the spectral classifier alone on clean speech. This \textit{mimic loss} is combined with the traditional local criterion to train the speech enhancer to produce de-noised speech. Feeding the de-noised speech to an off-the-shelf Kaldi training recipe for the CHiME-2 corpus shows significant improvements in WER.
Short Packet Structure for Ultra-Reliable Machine-Type Communication: Tradeoff between Detection and Decoding
Venue: Conference ICASSP 2018
Year: 2018

Machine-type communication requires rethinking of the structure of short packets due to the coding limitations and the significant role of the control information. In ultra-reliable low-latency communication (URLLC), it is crucial to optimally use the limited degrees of freedom (DoFs) to send data and control information. We consider a URLLC model for short packet transmission with acknowledgement (ACK). We compare the detection/decoding performance of two short packet structures: (1) time-multiplexed detection sequence and data; and (2) structure in which both packet detection and data decoding use all DoFs. Specifically, as an instance of the second structure we use superimposed sequences for detection and data. We derive the probabilities of false alarm and misdetection for an AWGN channel and numerically minimize the packet error probability (PER), showing that for delay-constrained data and ACK exchange, there is a tradeoff between the resources spent for detection and decoding. We show that the optimal PER for the superimposed structure is achieved for higher detection overhead. For this reason, the PER is also higher than in the preamble case. However, the superimposed structure is advantageous due to its flexibility to achieve optimal operation without the need to use multiple codebooks.
Statistical Speech Enhancement Based on Probabilistic Integration of Variational Autoencoder and Non-Negative Matrix Factorization
Venue: Conference ICASSP 2018
Year: 2018

This paper presents a statistical method of single-channel speech enhancement that uses a variational autoencoder (VAE) as a prior distribution on clean speech. A standard approach to speech enhancement is to train a deep neural network (DNN) to take noisy speech as input and output clean speech. Although this supervised approach requires a very large amount of pair data for training, it is not robust against unknown environments. Another approach is to use non-negative matrix factorization (NMF) based on basis spectra trained on clean speech in advance and those adapted to noise on the fly. This semi-supervised approach, however, causes considerable signal distortion in enhanced speech due to the unrealistic assumption that speech spectrograms are linear combinations of the basis spectra. Replacing the poor linear generative model of clean speech in NMF with a VAE---a powerful nonlinear deep generative model---trained on clean speech, we formulate a unified probabilistic generative model of noisy speech. Given noisy speech as observed data, we can sample clean speech from its posterior distribution. The proposed method outperformed the conventional DNN-based method in unseen noisy environments.
a Multi-Perspective Approach to Anomaly Detection for Self -Aware Embodied Agents
Venue: Conference ICASSP 2018
Year: 2018

This paper focuses on multi-sensor anomaly detection for moving cognitive agents using both external and private first-person visual observations. Both observation types are used to characterize agents' motion in a given environment. The proposed method generates locally uniform motion models by dividing a Gaussian process that approximates agents' displacements on the scene and provides a Shared Level (SL) self-awareness based on Environment Centered (EC) models. Such models are then used to train in a semi-unsupervised way a set of Generative Adversarial Networks (GANs) that produce an estimation of external and internal parameters of moving agents. Obtained results exemplify the feasibility of using multi-perspective data for predicting and analyzing trajectory information.
End-to-End Automatic Speech Translation of Audiobooks
Venue: Conference ICASSP 2018
Year: 2018

We investigate end-to-end speech-to-text translation on a corpus of audiobooks specifically augmented for this task. Previous works investigated the extreme case where source language transcription is not available during learning nor decoding, but we also study a midway case where source language transcription is available at training time only. In this case, a single model is trained to decode source speech into target text in a single pass. Experimental results show that it is possible to train compact and efficient end-to-end speech translation models in this setup. We also distribute the corpus and hope that our speech translation baseline on this corpus will be challenged in the future.
Maximum-A-Posteriori Signal Recovery with Prior Information: Applications to Compressive Sensing
Venue: Conference ICASSP 2018
Year: 2018

This paper studies the asymptotic performance of maximum-a-posteriori estimation in the presence of prior information. The problem arises in several applications such as recovery of signals with non-uniform sparsity pattern from underdetermined measurements. With prior information, the maximum-a-posteriori estimator might have asymmetric penalty. We consider a generic form of this estimator and study its performance via the replica method. Our analyses demonstrate an asymmetric form of the decoupling property in the large-system limit. Employing our results, we further investigate the performance of weighted zero-norm minimization for recovery of a non-uniform sparse signal. Our investigations illustrate that for a given distortion, the minimum number of required measurements can be significantly reduced by choosing weighting coefficients optimally.
Deep Feed-Forward Sequential Memory Networks for Speech Synthesis
Venue: Conference ICASSP 2018
Year: 2018

The Bidirectional LSTM (BLSTM) RNN based speech synthesis system is among the best parametric Text-to-Speech (TTS) systems in terms of the naturalness of generated speech, especially the naturalness in prosody. However, the model complexity and inference cost of BLSTM prevents its usage in many runtime applications. Meanwhile, Deep Feed-forward Sequential Memory Networks (DFSMN) has shown its consistent out-performance over BLSTM in both word error rate (WER) and the runtime computation cost in speech recognition tasks. Since speech synthesis also requires to model long-term dependencies compared to speech recognition, in this paper, we investigate the Deep-FSMN (DFSMN) in speech synthesis. Both objective and subjective experiments show that, compared with BLSTM TTS method, the DFSMN system can generate synthesized speech with comparable speech quality while drastically reduce model complexity and speech generation time.
Indian Buffet Process Deep Generative Models for Semi-Supervised Classification
Venue: Conference ICASSP 2018
Year: 2018

Deep generative models (DGMs) have brought about a major breakthrough, as well as renewed interest, in generative latent variable models. However, DGMs do not allow for performing data-driven inference of the number of latent features needed to represent the observed data. Traditional linear formulations address this issue by resorting to tools from the field of nonparametric statistics. Indeed, linear latent variable models imposed an Indian Buffet Process (IBP) prior have been extensively studied by the machine learning community; inference for such models can been performed either via exact sampling or via approximate variational techniques. Based on this inspiration, in this paper we examine whether similar ideas from the field of Bayesian nonparametrics can be utilized in the context of modern DGMs in order to address the latent variable dimensionality inference problem. To this end, we propose a novel DGM formulation, based on the imposition of an IBP prior. We devise an efficient Black-Box Variational inference algorithm for our model, and exhibit its efficacy in a number of semi-supervised classification experiments. In all cases, we use popular benchmark datasets, and compare to state-of-the-art DGMs.