OpenRaiser/Intern-Atlas huggingface dataset:
Information diffusion in interconnected heterogeneous networks
Venue: Conference ICASSP 2017
Year: 2017

In this paper, we are interested in modeling the diffusion of information in a multilayer network using thermodynamic diffusion approach. State of each agent is viewed as a topic mixture represented by a distribution over multiple topics. We have observed and learned diffusion-related thermodynamical patterns in the training data set, and we have used the estimated diffusion structure to predict the future states of the agents. A priori knowledge of a fraction of the state of all agents changes the problem to be a Kalman predictor problem that refines the predicted system state using the error in estimation of the agents. A real world Twitter data set is then used to evaluate and validate our information diffusion model.
Graph learning under sparsity priors
Venue: Conference ICASSP 2017
Year: 2017

Graph signals offer a very generic and natural representation for data that lives on networks or irregular structures. The actual data structure is however often unknown a priori but can sometimes be estimated from the knowledge of the application domain. If this is not possible, the data structure has to be inferred from the mere signal observations. This is exactly the problem that we address in this paper, under the assumption that the graph signals can be represented as a sparse linear combination of a few atoms of a structured graph dictionary. The dictionary is constructed on polynomials of the graph Laplacian, which can sparsely represent a general class of graph signals composed of localized patterns on the graph. We formulate a graph learning problem, whose solution provides an ideal fit between the signal observations and the sparse graph signal model. As the problem is non-convex, we propose to solve it by alternating between a signal sparse coding and a graph update step. We provide experimental results that outline the good graph recovery performance of our method, which generally compares favourably to other recent network inference algorithms.
Performance of time delay estimation in a cognitive radar
Venue: Conference ICASSP 2017
Year: 2017

A cognitive radar adapts the transmit waveform in response to changes in the radar and target environment. In this work, we analyze the recently proposed sub-Nyquist cognitive radar wherein the total transmit power in a multi-band cognitive waveform remains the same as its full-band conventional counterpart. For such a system, we derive lower bounds on the mean-squared-error (MSE) of a single-target time delay estimate. We formulate a procedure to select the optimal bands, and recommend distribution of the total power in different bands to enhance the accuracy of delay estimation. In particular, using Cramér-Rao bounds, we show that equi-width subbands in cognitive radar always have better delay estimation than the conventional radar. Further analysis using Ziv-Zakai bound reveals that cognitive radar performs well in low signal-to-noise (SNR) regions.
Low rank phase retrieval
Venue: Conference ICASSP 2017
Year: 2017

We develop two iterative algorithms for solving the low rank phase retrieval (LRPR) problem. LRPR refers to recovering a low-rank matrix $\X$ from magnitude-only (phaseless) measurements of random linear projections of its columns. Both methods consist of a spectral initialization step followed by an iterative algorithm to maximize the observed data likelihood. We obtain sample complexity bounds for our proposed initialization approach to provide a good approximation of the true $\X$. When the rank is low enough, these bounds are significantly lower than what existing single vector phase retrieval algorithms need. Via extensive experiments, we show that the same is also true for the proposed complete algorithms.
Globally optimal beamforming design for downlink CoMP transmission with limited backhaul capacity
Venue: Conference ICASSP 2017
Year: 2017

This paper considers a multicell downlink channel in which multiple base stations (BSs) cooperatively serve users by jointly precoding shared data transported from a central processor over limited-capacity backhaul links. We jointly design the beamformers and BS-user link selection so as to maximize the sum rate subject to user-specific signal-to-interference-noise (SINR) requirements, per-BS backhaul capacity and per-BS power constraints. As existing solutions for the considered problem are suboptimal and their optimality remains unknown due to the lack of globally optimal solutions, we characterized this gap by proposing a globally optimal algorithm for the problem of interest. Specifically, the proposed method is customized from a generic framework of a branch and bound algorithm applied to discrete monotonic optimization. We show that the proposed algorithm converges after a finite number of iterations, and can serve as a benchmark for existing suboptimal solutions and those that will be developed for similar contexts in the future. In this regard, we numerically compare the proposed optimal solution to a current state-of-the-art, which show that this suboptimal method only attains 70% to 90% of the optimal performance.
A Neural Network approach for mixing language models
Venue: Conference ICASSP 2017
Year: 2017

The performance of Neural Network (NN)-based language models is steadily improving due to the emergence of new architectures, which are able to learn different natural language characteristics. This paper presents a novel framework, which shows that a significant improvement can be achieved by combining different existing heterogeneous models in a single architecture. This is done through 1) a feature layer, which separately learns different NN-based models and 2) a mixture layer, which merges the resulting model features. In doing so, this architecture benefits from the learning capabilities of each model with no noticeable increase in the number of model parameters or the training time. Extensive experiments conducted on the Penn Treebank (PTB) and the Large Text Compression Benchmark (LTCB) corpus showed a significant reduction of the perplexity when compared to state-of-the-art feedforward as well as recurrent neural network architectures.
End-to-end visual speech recognition with LSTMS
Venue: Conference ICASSP 2017
Year: 2017

Traditional visual speech recognition systems consist of two stages, feature extraction and classification. Recently, several deep learning approaches have been presented which automatically extract features from the mouth images and aim to replace the feature extraction stage. However, research on joint learning of features and classification is very limited. In this work, we present an end-to-end visual speech recognition system based on Long-Short Memory (LSTM) networks. To the best of our knowledge, this is the first model which simultaneously learns to extract features directly from the pixels and perform classification and also achieves state-of-the-art performance in visual speech classification. The model consists of two streams which extract features directly from the mouth and difference images, respectively. The temporal dynamics in each stream are modelled by an LSTM and the fusion of the two streams takes place via a Bidirectional LSTM (BLSTM). An absolute improvement of 9.7% over the base line is reported on the OuluVS2 database, and 1.5% on the CUAVE database when compared with other methods which use a similar visual front-end.
A network of deep neural networks for Distant Speech Recognition
Venue: Conference ICASSP 2017
Year: 2017

Despite the remarkable progress recently made in distant speech recognition, state-of-the-art technology still suffers from a lack of robustness, especially when adverse acoustic conditions characterized by non-stationary noises and reverberation are met. A prominent limitation of current systems lies in the lack of matching and communication between the various technologies involved in the distant speech recognition process. The speech enhancement and speech recognition modules are, for instance, often trained independently. Moreover, the speech enhancement normally helps the speech recognizer, but the output of the latter is not commonly used, in turn, to improve the speech enhancement. To address both concerns, we propose a novel architecture based on a network of deep neural networks, where all the components are jointly trained and better cooperate with each other thanks to a full communication scheme between them. Experiments, conducted using different datasets, tasks and acoustic conditions, revealed that the proposed framework can overtake other competitive solutions, including recent joint training approaches.
Simultaneous low-rank component and graph estimation for high-dimensional graph signals: Application to brain imaging
Venue: Conference ICASSP 2017
Year: 2017

We propose an algorithm to uncover the intrinsic low-rank component of a high-dimensional, graph-smooth and grossly-corrupted dataset, under the situations that the underlying graph is unknown. Based on a model with a low-rank component plus a sparse perturbation, and an initial graph estimation, our proposed algorithm simultaneously learns the low-rank component and refines the graph. Our evaluations using synthetic and real brain imaging data in unsupervised and supervised classification tasks demonstrate encouraging performance.
How should we evaluate supervised hashing?
Venue: Conference ICASSP 2017
Year: 2017

Hashing produces compact representations for documents, to perform tasks like classification or retrieval based on these short codes. When hashing is supervised, the codes are trained using labels on the training data. This paper first shows that the evaluation protocols used in the literature for supervised hashing are not satisfactory: we show that a trivial solution that encodes the output of a classifier significantly outperforms existing supervised or semi-supervised methods, while using much shorter codes. We then propose two alternative protocols for supervised hashing: one based on retrieval on a disjoint set of classes, and another based on transfer learning to new classes. We provide two baseline methods for image-related tasks to assess the performance of (semi-)supervised hashing: without coding and with unsupervised codes. These baselines give a lower- and upper-bound on the performance of a supervised hashing scheme.
Multiple illumination phaseless super-resolution (MIPS) with applications to phaseless DoA estimation and diffraction imaging
Venue: Conference ICASSP 2017
Year: 2017

Phaseless super-resolution is the problem of recovering an unknown signal from measurements of the magnitudes of the low frequency Fourier transform of the signal. This problem arises in applications where measuring the phase, and making high-frequency measurements, are either too costly or altogether infeasible. The problem is especially challenging because it combines the difficult problems of phase retrieval and classical super-resolution
A scalable convolutional neural network for task-specified scenarios via knowledge distillation
Venue: Conference ICASSP 2017
Year: 2017

In this paper, we explore the redundancy in convolutional neural network, which scales with the complexity of vision tasks. Considering that many front-end visual systems are interested in only a limited range of visual targets, the removing of task-specified network redundancy can promote a wide range of potential applications. We propose a task-specified knowledge distillation algorithm to derive a simplified model with pre-set computation cost and minimized accuracy loss, which suits the resource constraint front-end systems well. Experiments on the MNIST and CIFAR10 datasets demonstrate the feasibility of the proposed approach as well as the existence of task-specified redundancy.
Fixed-point optimization of deep neural networks with adaptive step size retraining
Venue: Conference ICASSP 2017
Year: 2017

Fixed-point optimization of deep neural networks plays an important role in hardware based design and low-power implementations. Many deep neural networks show fairly good performance even with 2- or 3-bit precision when quantized weights are fine-tuned by retraining. We propose an improved fixedpoint optimization algorithm that estimates the quantization step size dynamically during the retraining. In addition, a gradual quantization scheme is also tested, which sequentially applies fixed-point optimizations from high- to low-precision. The experiments are conducted for feed-forward deep neural networks (FFDNNs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs).
Dual-Tree wavelet scattering network with parametric log transformation for object classification
Venue: Conference ICASSP 2017
Year: 2017

We introduce a ScatterNet that uses a parametric log transformation with Dual-Tree complex wavelets to extract translation invariant representations from a multi-resolution image. The parametric transformation aids the OLS pruning algorithm by converting the skewed distributions into relatively mean-symmetric distributions while the Dual-Tree wavelets improve the computational efficiency of the network. The proposed network is shown to outperform Mallat's ScatterNet on two image datasets, both for classification accuracy and computational efficiency. The advantages of the proposed network over other supervised and some unsupervised methods are also presented using experiments performed on different training dataset sizes.
Performance analysis for time-of-arrival estimation with oversampled low-complexity 1-bit a/d conversion
Venue: Conference ICASSP 2017
Year: 2017

Analog-to-digtial (A/D) conversion plays a crucial role when it comes to the design of energy-efficient and fast signal processing systems. As its complexity grows exponentially with the number of output bits, significant savings are possible when resorting to a minimum resolution of a single bit. However, then the nonlinear effect which is introduced by the A/D converter results in a pronounced performance loss, in particular for the case when the receiver is operated outside the low signal-to-noise ratio (SNR) regime. By trading the A/D resolution for a moderately faster sampling rate, we show that for time-of-arrival (TOA) estimation under any SNR level it is possible to obtain a low-complexity $1$-bit receive system which features a smaller performance degradation then the classical low SNR hard-limiting loss of $2/π$ ($-1.96$ dB). Key to this result is the employment of a lower bound for the Fisher information matrix which enables us to approximate the estimation performance for coarsely quantized receivers with correlated noise models in a pessimistic way.
Memory visualization for gated recurrent neural networks in speech recognition
Venue: Conference ICASSP 2017
Year: 2017

Recurrent neural networks (RNNs) have shown clear superiority in sequence modeling, particularly the ones with gated units, such as long short-term memory (LSTM) and gated recurrent unit (GRU). However, the dynamic properties behind the remarkable performance remain unclear in many applications, e.g., automatic speech recognition (ASR). This paper employs visualization techniques to study the behavior of LSTM and GRU when performing speech recognition tasks. Our experiments show some interesting patterns in the gated memory, and some of them have inspired simple yet effective modifications on the network structure. We report two of such modifications: (1) lazy cell update in LSTM, and (2) shortcut connections for residual learning. Both modifications lead to more comprehensible and powerful networks.
Joint Bayesian Gaussian Discriminant Analysis for speaker verification
Venue: Conference ICASSP 2017
Year: 2017

State-of-the-art i-vector based speaker verification relies on variants of Probabilistic Linear Discriminant Analysis (PLDA) for discriminant analysis. We are mainly motivated by the recent work of the joint Bayesian (JB) method, which is originally proposed for discriminant analysis in face verification. We apply JB to speaker verification and make three contributions beyond the original JB. 1) In contrast to the EM iterations with approximated statistics in the original JB, the EM iterations with exact statistics are employed and give better performance. 2) We propose to do simultaneous diagonalization (SD) of the within-class and between-class covariance matrices to achieve efficient testing, which has broader application scope than the SVD-based efficient testing method in the original JB. 3) We scrutinize similarities and differences between various Gaussian PLDAs and JB, complementing the previous analysis of comparing JB only with Prince-Elder PLDA. Extensive experiments are conducted on NIST SRE10 core condition 5, empirically validating the superiority of JB with faster convergence rate and 9-13% EER reduction compared with state-of-the-art PLDA.
Personalized acoustic modeling by weakly supervised multi-task deep learning using acoustic tokens discovered from unlabeled data
Venue: Conference ICASSP 2017
Year: 2017

It is well known that recognizers personalized to each user are much more effective than user-independent recognizers. With the popularity of smartphones today, although it is not difficult to collect a large set of audio data for each user, it is difficult to transcribe it. However, it is now possible to automatically discover acoustic tokens from unlabeled personal data in an unsupervised way. We therefore propose a multi-task deep learning framework called a phoneme-token deep neural network (PTDNN), jointly trained from unsupervised acoustic tokens discovered from unlabeled data and very limited transcribed data for personalized acoustic modeling. We term this scenario "weakly supervised". The underlying intuition is that the high degree of similarity between the HMM states of acoustic token models and phoneme models may help them learn from each other in this multi-task learning framework. Initial experiments performed over a personalized audio data set recorded from Facebook posts demonstrated that very good improvements can be achieved in both frame accuracy and word accuracy over popularly-considered baselines such as fDLR, speaker code and lightly supervised adaptation. This approach complements existing speaker adaptation approaches and can be used jointly with such techniques to yield improved results.
Adaptive DCTNet for audio signal classification
Venue: Conference ICASSP 2017
Year: 2017

In this paper, we investigate DCTNet for audio signal classification. Its output feature is related to Cohen's class of time-frequency distributions. We introduce the use of adaptive DCTNet (A-DCTNet) for audio signals feature extraction. The A-DCTNet applies the idea of constant-Q transform, with its center frequencies of filterbanks geometrically spaced. The A-DCTNet is adaptive to different acoustic scales, and it can better capture low frequency acoustic information that is sensitive to human audio perception than features such as Mel-frequency spectral coefficients (MFSC). We use features extracted by the A-DCTNet as input for classifiers. Experimental results show that the A-DCTNet and Recurrent Neural Networks (RNN) achieve state-of-the-art performance in bird song classification rate, and improve artist identification accuracy in music data. They demonstrate A-DCTNet's applicability to signal processing problems.
The microsoft 2016 conversational speech recognition system
Venue: Conference ICASSP 2017
Year: 2017

We describe Microsoft's conversational speech recognition system, in which we combine recent developments in neural-network-based acoustic and language modeling to advance the state of the art on the Switchboard recognition task. Inspired by machine learning ensemble techniques, the system uses a range of convolutional and recurrent neural networks. I-vector modeling and lattice-free MMI training provide significant gains for all acoustic model architectures. Language model rescoring with multiple forward and backward running RNNLMs, and word posterior-based system combination provide a 20% boost. The best single system uses a ResNet architecture acoustic model with RNNLM rescoring, and achieves a word error rate of 6.9% on the NIST 2000 Switchboard task. The combined system has an error rate of 6.2%, representing an improvement over previously reported results on this benchmark task.
End-to-end joint learning of natural language understanding and dialogue manager
Venue: Conference ICASSP 2017
Year: 2017

Natural language understanding and dialogue policy learning are both essential in conversational systems that predict the next system actions in response to a current user utterance. Conventional approaches aggregate separate models of natural language understanding (NLU) and system action prediction (SAP) as a pipeline that is sensitive to noisy outputs of error-prone NLU. To address the issues, we propose an end-to-end deep recurrent neural network with limited contextual dialogue memory by jointly training NLU and SAP on DSTC4 multi-domain human-human dialogues. Experiments show that our proposed model significantly outperforms the state-of-the-art pipeline models for both NLU and SAP, which indicates that our joint model is capable of mitigating the affects of noisy NLU outputs, and NLU model can be refined by error flows backpropagating from the extra supervised signals of system actions.
Revisiting the problem of audio-based hit song prediction using convolutional neural networks
Venue: Conference ICASSP 2017
Year: 2017

Being able to predict whether a song can be a hit has impor- tant applications in the music industry. Although it is true that the popularity of a song can be greatly affected by exter- nal factors such as social and commercial influences, to which degree audio features computed from musical signals (whom we regard as internal factors) can predict song popularity is an interesting research question on its own. Motivated by the recent success of deep learning techniques, we attempt to ex- tend previous work on hit song prediction by jointly learning the audio features and prediction models using deep learning. Specifically, we experiment with a convolutional neural net- work model that takes the primitive mel-spectrogram as the input for feature learning, a more advanced JYnet model that uses an external song dataset for supervised pre-training and auto-tagging, and the combination of these two models. We also consider the inception model to characterize audio infor- mation in different scales. Our experiments suggest that deep structures are indeed more accurate than shallow structures in predicting the popularity of either Chinese or Western Pop songs in Taiwan. We also use the tags predicted by JYnet to gain insights into the result of different models.
Interference reduction in music recordings combining Kernel Additive Modelling and Non-Negative Matrix Factorization
Venue: Conference ICASSP 2017
Year: 2017

In live and studio recordings unexpected sound events often lead to interferences in the signal. For non-stationary interferences, sound source separation techniques can be used to reduce the interference level in the recording. In this context, we present a novel approach combining the strengths of two algorithmic families: NMF and KAM. The recent KAM approach applies robust statistics on frames selected by a source-specific kernel to perform source separation. Based on semi-supervised NMF, we extend this approach in two ways. First, we locate the interference in the recording based on detected NMF activity. Second, we improve the kernel-based frame selection by incorporating an NMF-based estimate of the clean music signal. Further, we introduce a temporal context in the kernel, taking some musical structure into account. Our experiments show improved separation quality for our proposed method over a state-of-the-art approach for interference reduction.