pwc-archive/papers-with-abstracts huggingface dataset:
Consistent recovery threshold of hidden nearest neighbor graphs
Authors: Jian Ding, Yihong Wu, Jiaming Xu, Dana Yang
Date: 2019-11-18 00:00:00
Tasks: 2k

Motivated by applications such as discovering strong ties in social networks and assembling genome subsequences in biology, we study the problem of recovering a hidden $2k$-nearest neighbor (NN) graph in an $n$-vertex complete graph, whose edge weights are independent and distributed according to $P_n$ for edges in the hidden $2k$-NN graph and $Q_n$ otherwise. The special case of Bernoulli distributions corresponds to a variant of the Watts-Strogatz small-world graph. We focus on two types of asymptotic recovery guarantees as $n\to \infty$: (1) exact recovery: all edges are classified correctly with probability tending to one; (2) almost exact recovery: the expected number of misclassified edges is $o(nk)$. We show that the maximum likelihood estimator achieves (1) exact recovery for $2 \le k \le n^{o(1)}$ if $ \liminf \frac{2\alpha_n}{\log n}>1$; (2) almost exact recovery for $ 1 \le k \le o\left( \frac{\log n}{\log \log n} \right)$ if $\liminf \frac{kD(P_n||Q_n)}{\log n}>1$, where $\alpha_n \triangleq -2 \log \int \sqrt{d P_n d Q_n}$ is the R\'enyi divergence of order $\frac{1}{2}$ and $D(P_n||Q_n)$ is the Kullback-Leibler divergence. Under mild distributional assumptions, these conditions are shown to be information-theoretically necessary for any algorithm to succeed. A key challenge in the analysis is the enumeration of $2k$-NN graphs that differ from the hidden one by a given number of edges.
Low Complexity Autoencoder based End-to-End Learning of Coded Communications Systems
Authors: Nuwanthika Rajapaksha, Nandana Rajatheva, Matti Latva-aho
Date: 2019-11-19 00:00:00

End-to-end learning of a communications system using the deep learning-based autoencoder concept has drawn interest in recent research due to its simplicity, flexibility and its potential of adapting to complex channel models and practical system imperfections. In this paper, we have compared the bit error rate (BER) performance of autoencoder based systems and conventional channel coded systems with convolutional coding (CC), in order to understand the potential of deep learning-based systems as alternatives to conventional systems. From the simulations, autoencoder implementation was observed to have a better BER in 0-5 dB $E_{b}/N_{0}$ range than its equivalent half-rate convolutional coded BPSK with hard decision decoding, and to have only less than 1 dB gap at a BER of $10^{-5}$. Furthermore, we have also proposed a novel low complexity autoencoder architecture to implement end-to-end learning of coded systems in which we have shown better BER performance than the baseline implementation. The newly proposed low complexity autoencoder was capable of achieving a better BER performance than half-rate 16-QAM with hard decision decoding over the full 0-10 dB $E_{b}/N_{0}$ range and a better BER performance than the soft decision decoding in 0-4 dB $E_{b}/N_{0}$ range.
Low rank tensor completion with sparse regularization in a transformed domain
Authors: Ping-Ping Wang, Liang Li, Guang-Hui Cheng
Date: 2019-11-19 00:00:00

Tensor completion is a challenging problem with various applications. Many related models based on the low-rank prior of the tensor have been proposed. However, the low-rank prior may not be enough to recover the original tensor from the observed incomplete tensor. In this paper, we prose a tensor completion method by exploiting both the low-rank and sparse prior of tensor. Specifically, the tensor completion task can be formulated as a low-rank minimization problem with a sparse regularizer. The low-rank property is depicted by the tensor truncated nuclear norm based on tensor singular value decomposition (T-SVD) which is a better approximation of tensor tubal rank than tensor nuclear norm. While the sparse regularizer is imposed by a $\ell_{1}$-norm in a discrete cosine transformation (DCT) domain, which can better employ the local sparse property of completed data. To solve the optimization problem, we employ an alternating direction method of multipliers (ADMM) in which we only need to solve several subproblems which have closed-form solutions. Substantial experiments on real world images and videos show that the proposed method has better performances than the existing state-of-the-art methods.
Enhancing the Extraction of Interpretable Information for Ischemic Stroke Imaging from Deep Neural Networks
Authors: Erico Tjoa, Guo Heng, Lu Yuhao, Cuntai Guan
Date: 2019-11-19 00:00:00
Tasks: Lesion Segmentation

We implement a visual interpretability method Layer-wise Relevance Propagation (LRP) on top of 3D U-Net trained to perform lesion segmentation on the small dataset of multi-modal images provided by ISLES 2017 competition. We demonstrate that LRP modifications could provide more sensible visual explanations to an otherwise highly noise-skewed saliency map. We also link amplitude of modified signals to useful information content. High amplitude localized signals appear to constitute the noise that undermines the interpretability capacity of LRP. Furthermore, mathematical framework for possible analysis of function approximation is developed by analogy.
Distributed Microphone Speech Enhancement based on Deep Learning
Authors: Syu-Siang Wang, Yu-You Liang, Jeih-weih Hung, Yu Tsao, Hsin-Min Wang, Shih-Hau Fang
Date: 2019-11-19 00:00:00
Tasks: All, Deep Learning, Speech Enhancement

Speech-related applications deliver inferior performance in complex noise environments. Therefore, this study primarily addresses this problem by introducing speech-enhancement (SE) systems based on deep neural networks (DNNs) applied to a distributed microphone architecture, and then investigates the effectiveness of three different DNN-model structures. The first system constructs a DNN model for each microphone to enhance the recorded noisy speech signal, and the second system combines all the noisy recordings into a large feature structure that is then enhanced through a DNN model. As for the third system, a channel-dependent DNN is first used to enhance the corresponding noisy input, and all the channel-wise enhanced outputs are fed into a DNN fusion model to construct a nearly clean signal. All the three DNN SE systems are operated in the acoustic frequency domain of speech signals in a diffuse-noise field environment. Evaluation experiments were conducted on the Taiwan Mandarin Hearing in Noise Test (TMHINT) database, and the results indicate that all the three DNN-based SE systems provide the original noise-corrupted signals with improved speech quality and intelligibility, whereas the third system delivers the highest signal-to-noise ratio (SNR) improvement and optimal speech intelligibility.
Deep interval prediction model with gradient descend optimization method for short-term wind power prediction
Authors: Chaoshun Li, Geng Tang, Xiaoming Xue, Xinbiao Chen, Ruoheng Wang, Chu Zhang
Date: 2019-11-19 00:00:00
Tasks: Prediction

The application of wind power interval prediction for power systems attempts to give more comprehensive support to dispatchers and operators of the grid. Lower upper bound estimation (LUBE) method is widely applied in interval prediction. However, the existing LUBE approaches are trained by meta-heuristic optimization, which is either time-consuming or show poor effect when the LUBE model is complex. In this paper, a deep interval prediction method is designed in the framework of LUBE and an efficient gradient descend (GD) training approach is proposed to train the LUBE model. In this method, the long short-term memory is selected as a representative to show the modelling approach. The architecture of the proposed model consists of three parts, namely the long short-term memory module, the fully connected layers and the rank ordered module. Two loss functions are specially designed for implementing the GD training method based on the root mean square back propagation algorithm. To verify the performance of the proposed model, conventional LUBE models, as well as popular statistic interval prediction models are compared in numerical experiments. The results show that the proposed approach performs best in terms of effectiveness and efficiency with average 45% promotion in quality of prediction interval and 66% reduction of time consumptions compared to traditional LUBE models.
On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning
Authors: Aritra Dutta, El Houcine Bergou, Ahmed M. Abdelmoniem, Chen-Yu Ho, Atal Narayan Sahu, Marco Canini, Panos Kalnis
Date: 2019-11-19 00:00:00
Tasks: Model Compression, Quantization

Compressed communication, in the form of sparsification or quantization of stochastic gradients, is employed to reduce communication costs in distributed data-parallel training of deep neural networks. However, there exists a discrepancy between theory and practice: while theoretical analysis of most existing compression methods assumes compression is applied to the gradients of the entire model, many practical implementations operate individually on the gradients of each layer of the model. In this paper, we prove that layer-wise compression is, in theory, better, because the convergence rate is upper bounded by that of entire-model compression for a wide range of biased and unbiased compression methods. However, despite the theoretical bound, our experimental study of six well-known methods shows that convergence, in practice, may or may not be better, depending on the actual trained model and compression ratio. Our findings suggest that it would be advantageous for deep learning frameworks to include support for both layer-wise and entire-model compression.
Smoke Sky -- Exploring New Frontiers of Unmanned Aerial Systems for Wildland Fire Science and Applications
Authors: E. Natasha Stavros, Ali Agha, Allen Sirota, Marco Quadrelli, Kamak Ebadi, Kyongsik Yun
Date: 2019-11-12 00:00:00
Tasks: Management

Wildfire has had increasing impacts on society as the climate changes and the wildland urban interface grows. As such, there is a demand for innovative solutions to help manage fire. Managing wildfire can include proactive fire management such as prescribed burning within constrained areas or advancements for reactive fire management (e.g., fire suppression). Because of the growing societal impact, the JPL BlueSky program sought to assess the current state of fire management and technology and determine areas with high return on investment. To accomplish this, we met with the national interagency Unmanned Aerial System (UAS) Advisory Group (UASAG) and with leading technology transfer experts for fire science and management applications. We provide an overview of the current state as well as an analysis of the impact, maturity and feasibility of integrating different technologies that can be developed by JPL. Based on the findings, the highest return on investment technologies for fire management are first to develop single micro-aerial vehicle (MAV) autonomy, autonomous sensing over fire, and the associated data and information system for active fire local environment mapping. Once this is completed for a single MAV, expanding the work to include many in a swarm would require further investment of distributed MAV autonomy and MAV swarm mechanics, but could greatly expand the breadth of application over large fires. Important to investing in these technologies will be in developing collaborations with the key influencers and champions for using UAS technology in fire management.
Generative Audio Synthesis with a Parametric Model
Authors: Krishna Subramani, Alexandre D'Hooge, Preeti Rao
Date: 2019-11-15 00:00:00
Tasks: Audio Synthesis, model

Use a parametric representation of audio to train a generative model in the interest of obtaining more flexible control over the generated sound.
The Power of Factorization Mechanisms in Local and Central Differential Privacy
Authors: Alexander Edmonds, Aleksandar Nikolov, Jonathan Ullman
Date: 2019-11-19 00:00:00

We give new characterizations of the sample complexity of answering linear queries (statistical queries) in the local and central models of differential privacy: *In the non-interactive local model, we give the first approximate characterization of the sample complexity. Informally our bounds are tight to within polylogarithmic factors in the number of queries and desired accuracy. Our characterization extends to agnostic learning in the local model. *In the central model, we give a characterization of the sample complexity in the high-accuracy regime that is analogous to that of Nikolov, Talwar, and Zhang (STOC 2013), but is both quantitatively tighter and has a dramatically simpler proof. Our lower bounds apply equally to the empirical and population estimation problems. In both cases, our characterizations show that a particular factorization mechanism is approximately optimal, and the optimal sample complexity is bounded from above and below by well studied factorization norms of a matrix associated with the queries.
GMAN: A Graph Multi-Attention Network for Traffic Prediction
Authors: Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, Jianzhong Qi
Date: 2019-11-11 00:00:00
Tasks: Decoder, Image Dehazing, Prediction, Traffic Prediction

Long-term traffic prediction is highly challenging due to the complexity of traffic systems and the constantly changing nature of many impacting factors. In this paper, we focus on the spatio-temporal factors, and propose a graph multi-attention network (GMAN) to predict traffic conditions for time steps ahead at different locations on a road network graph. GMAN adapts an encoder-decoder architecture, where both the encoder and the decoder consist of multiple spatio-temporal attention blocks to model the impact of the spatio-temporal factors on traffic conditions. The encoder encodes the input traffic features and the decoder predicts the output sequence. Between the encoder and the decoder, a transform attention layer is applied to convert the encoded traffic features to generate the sequence representations of future time steps as the input of the decoder. The transform attention mechanism models the direct relationships between historical and future time steps that helps to alleviate the error propagation problem among prediction time steps. Experimental results on two real-world traffic prediction tasks (i.e., traffic volume prediction and traffic speed prediction) demonstrate the superiority of GMAN. In particular, in the 1 hour ahead prediction, GMAN outperforms state-of-the-art methods by up to 4% improvement in MAE measure. The source code is available at https://github.com/zhengchuanpan/GMAN.
Deep Learning for Epidemiological Predictions
Authors: Wu, Yuexin   Yang, Yiming   Nishiura, Hiroshi   Saitoh, Masaya
Date: 2018-07-21 00:00:00
Proceeding: The 41st International ACM SIGIR Conference 2018 7
Tasks: Deep Learning, Epidemiology, GPR, Multivariate Time Series Forecasting, Time Series, Time Series Analysis, Time Series Forecasting

Predicting new and urgent trends in epidemiological data is an important problem for public health, and has attracted increasing attention in the data mining and machine learning communities. The temporal nature of epidemiology data and the need for real-time prediction by the system makes the problem residing in the category of time-series forecasting or prediction. While traditional autoregressive (AR) methods and Gaussian Process Regression (GPR) have been actively studied for solving this problem, deep learning techniques have not been explored in this domain. In this paper, we develop a deep learning framework, for the first time, to predict epidemiology profiles in the time-series perspective. We adopt Recurrent Neural Networks (RNNs) to capture the long-term correlation in the data and Convolutional Neural Networks (CNNs) to fuse information from data of different sources. A residual structure is also applied to prevent overfitting issues in the training process. We compared our model with the most widely used AR models on USA and Japan datasets. Our approach provides consistently better results than these baseline methods
Universal Intelligence: A Definition of Machine Intelligence
Authors: Shane Legg, Marcus Hutter
Date: 2007-12-20 00:00:00

A fundamental problem in artificial intelligence is that nobody really knows what intelligence is. The problem is especially acute when we need to consider artificial systems which are significantly different to humans. In this paper we approach this problem in the following way: We take a number of well known informal definitions of human intelligence that have been given by experts, and extract their essential features. These are then mathematically formalised to produce a general measure of intelligence for arbitrary machines. We believe that this equation formally captures the concept of machine intelligence in the broadest reasonable sense. We then show how this formal definition is related to the theory of universal optimal learning agents. Finally, we survey the many other tests and definitions of intelligence that have been proposed for machines.
Semi-Orthogonal Low-Rank Matrix Factorization for Deep Neural Networks
Authors: Daniel Povey, Gaofeng Cheng, Yiming Wang, Ke Li, Hainan Xu, Mahsa Yarmohammadi, Sanjeev Khudanpur
Date: 2018-09-02 00:00:00
Proceeding: Interspeech 2018 2018 9
Tasks: speech-recognition, Speech Recognition

Time Delay Neural Networks (TDNNs), also known as onedimensional Convolutional Neural Networks (1-d CNNs), are an efficient and well-performing neural network architecture for speech recognition. We introduce a factored form of TDNNs (TDNN-F) which is structurally the same as a TDNN whose layers have been compressed via SVD, but is trained from a random start with one of the two factors of each matrix constrained to be semi-orthogonal. This gives substantial improvements over TDNNs and performs about as well as TDNN-LSTM hybrids.
The Rise of Guardians: Fact-checking URL Recommendation to Combat Fake News
Authors: Vo, Nguyen; Lee, Kyumin
Date: 2018-07-11 00:00:00
Proceeding: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018 2018 7
Tasks: Fact Checking, Misinformation, Recommendation Systems

A large body of research work and efforts have been focused on detecting fake news and building online fact-check systems in order to debunk fake news as soon as possible. Despite the existence of these systems, fake news is still wildly shared by online users. It indicates that these systems may not be fully utilized. After detecting fake news, what is the next step to stop people from sharing it? How can we improve the utilization of these fact-check systems? To fill this gap, in this paper, we (i) collect and analyze online users called guardians, who correct misinformation and fake news in online discussions by referring fact-checking URLs; and (ii) propose a novel fact-checking URL recommendation model to encourage the guardians to engage more in fact-checking activities. We found that the guardians usually took less than one day to reply to claims in online conversations and took another day to spread verified information to hundreds of millions of followers. Our proposed recommendation model outperformed four state-of-the-art models by 11%~33%.
Learning Cross-modal Context Graph for Visual Grounding
Authors: Yongfei Liu; Bo Wan; Xiaodan Zhu; Xuming He
Date: 2020-02-13 00:00:00
Proceeding: AAAI-2020 2020 2
Tasks: Graph Matching, Graph Neural Network, Language Modelling, Natural Language Visual Grounding, Phrase Grounding, Visual Grounding

Visual grounding is a ubiquitous building block in many vision-language tasks and yet remains challenging due to large variations in visual and linguistic features of grounding entities, strong context effect and the resulting semantic ambiguities. Prior works typically focus on learning representations of individual phrases with limited context information. To address their limitations, this paper proposes a language-guided graph representation to capture the global context of grounding entities and their relations, and develop a cross-modal graph matching strategy for the multiple-phrase visual grounding task. In particular, we introduce a modular graph neural network to compute context-aware representations of phrases and object proposals respectively via message propagation, followed by a graph-based matching module to generate globally consistent localization of grounding phrases. We train the entire graph neural network jointly in a two-stage strategy and evaluate it on the Flickr30K Entities benchmark. Extensive experiments show that our method outperforms the prior state of the arts by a sizable margin, evidencing the efficacy of our grounding framework. Code is available at https://github.com/youngfly11/LCMCG-PyTorch.
Dark energy constraints and correlations with systematics from CFHTLS weak lensing, SNLS supernovae Ia and WMAP5
Authors: M. Kilbinger, K. Benabed, J. Guy, P. Astier, I. Tereno, L. Fu, D. Wraith, J. Coupon, Y. Mellier, C. Balland, F. R. Bouchet, T. Hamana, D. Hardin, H. J. McCracken, R. Pain, N. Regnault, M. Schultheis, H. Yahagi
Date: 2008-10-28 00:00:00

We combine measurements of weak gravitational lensing from the CFHTLS-Wide survey, supernovae Ia from CFHT SNLS and CMB anisotropies from WMAP5 to obtain joint constraints on cosmological parameters, in particular, the dark energy equation of state parameter w. We assess the influence of systematics in the data on the results and look for possible correlations with cosmological parameters. We implement an MCMC algorithm to sample the parameter space of a flat CDM model with a dark-energy component of constant w. Systematics in the data are parametrised and included in the analysis. We determine the influence of photometric calibration of SNIa data on cosmological results by calculating the response of the distance modulus to photometric zero-point variations. The weak lensing data set is tested for anomalous field-to-field variations and a systematic shape measurement bias for high-z galaxies. Ignoring photometric uncertainties for SNLS biases cosmological parameters by at most 20% of the statistical errors, using supernovae only; the parameter uncertainties are underestimated by 10%. The weak lensing field-to-field variance pointings is 5%-15% higher than that predicted from N-body simulations. We find no bias of the lensing signal at high redshift, within the framework of a simple model. Assuming a systematic underestimation of the lensing signal at high redshift, the normalisation sigma_8 increases by up to 8%. Combining all three probes we obtain -0.10<1+w<0.06 at 68% confidence (-0.18<1+w<0.12 at 95%), including systematic errors. Systematics in the data increase the error bars by up to 35%; the best-fit values change by less than 0.15sigma. [Abridged]
On using 2D sequence-to-sequence models for speech recognition
Authors: Parnia Bahar, Albert Zeyer, Ralf Schlüter, Hermann Ney
Date: 2019-11-20 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Attention-based sequence-to-sequence models have shown promising results in automatic speech recognition. Using these architectures, one-dimensional input and output sequences are related by an attention approach, thereby replacing more explicit alignment processes, like in classical HMM-based modeling. In contrast, here we apply a novel two-dimensional long short-term memory (2DLSTM) architecture to directly model the input/output relation between audio/feature vector sequences and word sequences. The proposed model is an alternative model such that instead of using any type of attention components, we apply a 2DLSTM layer to assimilate the context from both input observations and output transcriptions. The experimental evaluation on the Switchboard 300h automatic speech recognition task shows word error rates for the 2DLSTM model that are competitive to end-to-end attention-based model.