pwc-archive/papers-with-abstracts huggingface dataset:
Text Infilling
Authors: Wanrong Zhu, Zhiting Hu, Eric Xing
Date: 2019-01-01 00:00:00
Tasks: Retrieval, Sentence, Text Generation, Text Infilling

Recent years have seen remarkable progress of text generation in different
contexts, such as the most common setting of generating text from scratch, and
the emerging paradigm of retrieval-and-rewriting. Text infilling, which fills
missing text portions of a sentence or paragraph, is also of numerous use in
real life, yet is under-explored. Previous work has focused on restricted
settings by either assuming single word per missing portion or limiting to a
single missing portion to the end of the text. This paper studies the general
task of text infilling, where the input text can have an arbitrary number of
portions to be filled, each of which may require an arbitrary unknown number of
tokens. We study various approaches for the task, including a self-attention
model with segment-aware position encoding and bidirectional context modeling.
We create extensive supervised data by masking out text with varying
strategies. Experiments show the self-attention model greatly outperforms
others, creating a strong baseline for future research.
Accelerated MM Algorithms for Ranking Scores Inference from Comparison Data
Authors: Milan Vojnovic, Se-Young Yun, Kaifang Zhou
Date: 2019-01-01 00:00:00
Tasks: Bayesian Inference

In this paper, we study a popular method for inference of the Bradley-Terry model parameters, namely the MM algorithm, for maximum likelihood estimation and maximum a posteriori probability estimation. This class of models includes the Bradley-Terry model of paired comparisons, the Rao-Kupper model of paired comparisons allowing for tie outcomes, the Luce choice model, and the Plackett-Luce ranking model. We establish tight characterizations of the convergence rate for the MM algorithm, and show that it is essentially equivalent to that of a gradient descent algorithm. For the maximum likelihood estimation, the convergence is shown to be linear with the rate crucially determined by the algebraic connectivity of the matrix of item pair co-occurrences in observed comparison data. For the Bayesian inference, the convergence rate is also shown to be linear, with the rate determined by a parameter of the prior distribution in a way that can make the convergence arbitrarily slow for small values of this parameter. We propose a simple modification of the classical MM algorithm that avoids the observed slow convergence issue and accelerates the convergence. The key component of the accelerated MM algorithm is a parameter rescaling performed at each iteration step that is carefully chosen based on theoretical analysis and characterisation of the convergence rate. Our experimental results, performed on both synthetic and real-world data, demonstrate the identified slow convergence issue of the classic MM algorithm, and show that significant efficiency gains can be obtained by our new proposed method.
Total Variation with Overlapping Group Sparsity and Lp Quasinorm for Infrared Image Deblurring under Salt-and-Pepper Noise
Authors: Xingguo Liu, Yinping Chen, Zhenming Peng, Juan Wu
Date: 2018-12-31 00:00:00
Tasks: Deblurring, Image Deblurring, Image Restoration

Because of the limitations of the infrared imaging principle and the
properties of infrared imaging systems, infrared images have some drawbacks,
including a lack of details, indistinct edges, and a large amount of
salt-andpepper noise. To improve the sparse characteristics of the image while
maintaining the image edges and weakening staircase artifacts, this paper
proposes a method that uses the Lp quasinorm instead of the L1 norm and for
infrared image deblurring with an overlapping group sparse total variation
method. The Lp quasinorm introduces another degree of freedom, better describes
image sparsity characteristics, and improves image restoration. Furthermore, we
adopt the accelerated alternating direction method of multipliers and fast
Fourier transform theory in the proposed method to improve the efficiency and
robustness of our algorithm. Experiments show that under different conditions
for blur and salt-and-pepper noise, the proposed method leads to excellent
performance in terms of objective evaluation and subjective visual results.
Adaptive Quantile Low-Rank Matrix Factorization
Authors: Shuang Xu, Chun-Xia Zhang, Jiangshe Zhang
Date: 2019-01-01 00:00:00

Low-rank matrix factorization (LRMF) has received much popularity owing to its successful applications in both computer vision and data mining. By assuming noise to come from a Gaussian, Laplace or mixture of Gaussian distributions, significant efforts have been made on optimizing the (weighted) $L_1$ or $L_2$-norm loss between an observed matrix and its bilinear factorization. However, the type of noise distribution is generally unknown in real applications and inappropriate assumptions will inevitably deteriorate the behavior of LRMF. On the other hand, real data are often corrupted by skew rather than symmetric noise. To tackle this problem, this paper presents a novel LRMF model called AQ-LRMF by modeling noise with a mixture of asymmetric Laplace distributions. An efficient algorithm based on the expectation-maximization (EM) algorithm is also offered to estimate the parameters involved in AQ-LRMF. The AQ-LRMF model possesses the advantage that it can approximate noise well no matter whether the real noise is symmetric or skew. The core idea of AQ-LRMF lies in solving a weighted $L_1$ problem with weights being learned from data. The experiments conducted on synthetic and real datasets show that AQ-LRMF outperforms several state-of-the-art techniques. Furthermore, AQ-LRMF also has the superiority over the other algorithms in terms of capturing local structural information contained in real images.
A Theoretical Analysis of Deep Q-Learning
Authors: Jianqing Fan, Zhaoran Wang, Yuchen Xie, Zhuoran Yang
Date: 2019-01-01 00:00:00
Tasks: Deep Reinforcement Learning, Q-Learning, Reinforcement Learning

Despite the great empirical success of deep reinforcement learning, its theoretical foundation is less well understood. In this work, we make the first attempt to theoretically understand the deep Q-network (DQN) algorithm (Mnih et al., 2015) from both algorithmic and statistical perspectives. In specific, we focus on a slight simplification of DQN that fully captures its key features. Under mild assumptions, we establish the algorithmic and statistical rates of convergence for the action-value functions of the iterative policy sequence obtained by DQN. In particular, the statistical error characterizes the bias and variance that arise from approximating the action-value function using deep neural network, while the algorithmic error converges to zero at a geometric rate. As a byproduct, our analysis provides justifications for the techniques of experience replay and target network, which are crucial to the empirical success of DQN. Furthermore, as a simple extension of DQN, we propose the Minimax-DQN algorithm for zero-sum Markov game with two players. Borrowing the analysis of DQN, we also quantify the difference between the policies obtained by Minimax-DQN and the Nash equilibrium of the Markov game in terms of both the algorithmic and statistical rates of convergence.
Fast and Globally Optimal Rigid Registration of 3D Point Sets by Transformation Decomposition
Authors: Xuechen Li, Yinlong Liu, Yiru Wang, Chen Wang, Manning Wang, Zhijian Song
Date: 2018-12-29 00:00:00
Tasks: Translation

The rigid registration of two 3D point sets is a fundamental problem in
computer vision. The current trend is to solve this problem globally using the
BnB optimization framework. However, the existing global methods are slow for
two main reasons: the computational complexity of BnB is exponential to the
problem dimensionality (which is six for 3D rigid registration), and the bound
evaluation used in BnB is inefficient. In this paper, we propose two techniques
to address these problems. First, we introduce the idea of translation
invariant vectors, which allows us to decompose the search of a 6D rigid
transformation into a search of 3D rotation followed by a search of 3D
translation, each of which is solved by a separate BnB algorithm. This
transformation decomposition reduces the problem dimensionality of BnB
algorithms and substantially improves its efficiency. Then, we propose a new
data structure, named 3D Integral Volume, to accelerate the bound evaluation in
both BnB algorithms. By combining these two techniques, we implement an
efficient algorithm for rigid registration of 3D point sets. Extensive
experiments on both synthetic and real data show that the proposed algorithm is
three orders of magnitude faster than the existing state-of-the-art global
methods.
Realizing data features by deep nets
Authors: Zheng-Chu Guo, Lei Shi, Shao-Bo Lin
Date: 2019-01-01 00:00:00

This paper considers the power of deep neural networks (deep nets for short)
in realizing data features. Based on refined covering number estimates, we find
that, to realize some complex data features, deep nets can improve the
performances of shallow neural networks (shallow nets for short) without
requiring additional capacity costs. This verifies the advantage of deep nets
in realizing complex features. On the other hand, to realize some simple data
feature like the smoothness, we prove that, up to a logarithmic factor, the
approximation rate of deep nets is asymptotically identical to that of shallow
nets, provided that the depth is fixed. This exhibits a limitation of deep nets
in realizing simple features.
MaD: Mapping and debugging framework for implementing deep neural network onto a neuromorphic chip with crossbar array of synapses
Authors: Roshan Gopalakrishnan, Ashish Jith Sreejith Kumar, Yansong Chua
Date: 2019-01-01 00:00:00

Neuromorphic systems or dedicated hardware for neuromorphic computing is
getting popular with the advancement in research on different device materials
for synapses, especially in crossbar architecture and also algorithms specific
or compatible to neuromorphic hardware. Hence, an automated mapping of any deep
neural network onto the neuromorphic chip with crossbar array of synapses and
an efficient debugging framework is very essential. Here, mapping is defined as
the deployment of a section of deep neural network layer onto a neuromorphic
core and the generation of connection lists among population of neurons to
specify the connectivity between various neuromorphic cores on the neuromorphic
chip. Debugging is the verification of computations performed on the
neuromorphic chip during inferencing. Together the framework becomes Mapping
and Debugging (MaD) framework. MaD framework is quite general in usage as it is
a Python wrapper which can be integrated with almost every simulator tools for
neuromorphic chips. This paper illustrates the MaD framework in detail,
considering some optimizations while mapping onto a single neuromorphic core. A
classification task on MNIST and CIFAR-10 datasets are considered for test case
implementation of MaD framework.
Variational Self-attention Model for Sentence Representation
Authors: Qiang Zhang, Shangsong Liang, Emine Yilmaz
Date: 2018-12-30 00:00:00
Tasks: model, Sentence, Stance Detection, Variational Inference

This paper proposes a variational self-attention model (VSAM) that employs variational inference to derive self-attention. We model the self-attention vector as random variables by imposing a probabilistic distribution. The self-attention mechanism summarizes source information as an attention vector by weighted sum, where the weights are a learned probabilistic distribution. Compared with conventional deterministic counterpart, the stochastic units incorporated by VSAM allow multi-modal attention distributions. Furthermore, by marginalizing over the latent variables, VSAM is more robust against overfitting. Experiments on the stance detection task demonstrate the superiority of our method.
FPGA-based Accelerators of Deep Learning Networks for Learning and Classification: A Review
Authors: Ahmad Shawahna, Sadiq M. Sait, Aiman El-Maleh
Date: 2019-01-01 00:00:00
Tasks: CPU, Deep Learning, General Classification

Due to recent advances in digital technologies, and availability of credible
data, an area of artificial intelligence, deep learning, has emerged, and has
demonstrated its ability and effectiveness in solving complex learning problems
not possible before. In particular, convolution neural networks (CNNs) have
demonstrated their effectiveness in image detection and recognition
applications. However, they require intensive CPU operations and memory
bandwidth that make general CPUs fail to achieve desired performance levels.
Consequently, hardware accelerators that use application specific integrated
circuits (ASICs), field programmable gate arrays (FPGAs), and graphic
processing units (GPUs) have been employed to improve the throughput of CNNs.
More precisely, FPGAs have been recently adopted for accelerating the
implementation of deep learning networks due to their ability to maximize
parallelism as well as due to their energy efficiency. In this paper, we review
recent existing techniques for accelerating deep learning networks on FPGAs. We
highlight the key features employed by the various techniques for improving the
acceleration performance. In addition, we provide recommendations for enhancing
the utilization of FPGAs for CNNs acceleration. The techniques investigated in
this paper represent the recent trends in FPGA-based accelerators of deep
learning networks. Thus, this review is expected to direct the future advances
on efficient hardware accelerators and to be useful for deep learning
researchers.
Gated-Dilated Networks for Lung Nodule Classification in CT scans
Authors: Mundher Al-Shabi, Hwee Kuan Lee, Maxine Tan
Date: 2019-01-01 00:00:00
Tasks: Classification, Computed Tomography (CT), General Classification, Lung Nodule Classification

Different types of Convolutional Neural Networks (CNNs) have been applied to detect cancerous lung nodules from computed tomography (CT) scans. However, the size of a nodule is very diverse and can range anywhere between 3 and 30 millimeters. The high variation of nodule sizes makes classifying them a difficult and challenging task. In this study, we propose a novel CNN architecture called Gated-Dilated (GD) networks to classify nodules as malignant or benign. Unlike previous studies, the GD network uses multiple dilated convolutions instead of max-poolings to capture the scale variations. Moreover, the GD network has a Context-Aware sub-network that analyzes the input features and guides the features to a suitable dilated convolution. We evaluated the proposed network on more than 1,000 CT scans from the LIDC-LDRI dataset. Our proposed network outperforms state-of-the-art baseline models including Multi-Crop, Resnet, and Densenet, with an AUC of >0.95. Compared to the baseline models, the GD network improves the classification accuracies of mid-range sized nodules. Furthermore, we observe a relationship between the size of the nodule and the attention signal generated by the Context-Aware sub-network, which validates our new network architecture.
An Active Learning Framework for Efficient Robust Policy Search
Authors: Sai Kiran Narayanaswami, Nandan Sudarsanam, Balaraman Ravindran
Date: 2019-01-01 00:00:00
Tasks: Active Learning, continuous-control, Continuous Control, Multi-Task Learning

Robust Policy Search is the problem of learning policies that do not degrade in performance when subject to unseen environment model parameters. It is particularly relevant for transferring policies learned in a simulation environment to the real world. Several existing approaches involve sampling large batches of trajectories which reflect the differences in various possible environments, and then selecting some subset of these to learn robust policies, such as the ones that result in the worst performance. We propose an active learning based framework, EffAcTS, to selectively choose model parameters for this purpose so as to collect only as much data as necessary to select such a subset. We apply this framework using Linear Bandits, and experimentally validate the gains in sample efficiency and the performance of our approach on standard continuous control tasks. We also present a Multi-Task Learning perspective to the problem of Robust Policy Search, and draw connections from our proposed framework to existing work on Multi-Task Learning.
Closing the gap towards end-to-end autonomous vehicle system
Authors: Yonatan Glassner, Liran Gispan, Ariel Ayash, Tal Furman Shohet
Date: 2019-01-01 00:00:00
Tasks: Autonomous Vehicles

Designing a driving policy for autonomous vehicles is a difficult task.
Recent studies suggested an end-toend (E2E) training of a policy to predict car
actuators directly from raw sensory inputs. It is appealing due to the ease of
labeled data collection and since handcrafted features are avoided. Explicit
drawbacks such as interpretability, safety enforcement and learning efficiency
limit the practical application of the approach. In this paper, we amend the
basic E2E architecture to address these shortcomings, while retaining the power
of end-to-end learning. A key element in our proposed architecture is
formulation of the learning problem as learning of trajectory. We also apply a
Gaussian mixture model loss to contend with multi-modal data, and adopt a
finance risk measure, conditional value at risk, to emphasize rare events. We
analyze the effect of each concept and present driving performance in a highway
scenario in the TORCS simulator. Video is available in this link:
https://www.youtube.com/watch?v=1JYNBZNOe_4
Morphological Network: How Far Can We Go with Morphological Neurons?
Authors: Ranjan Mondal, Sanchayan Santra, Soumendu Sundar Mukherjee, Bhabatosh Chanda
Date: 2019-01-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Image Classification, Image Dehazing, regression, Representation Learning, Satellite Image Classification

Morphological neurons, that is morphological operators such as dilation and erosion with learnable structuring elements, have intrigued researchers for quite some time because of the power these operators bring to the table despite their simplicity. These operators are known to be powerful nonlinear tools, but for a given problem coming up with a sequence of operations and their structuring element is a non-trivial task. So, the existing works have mainly focused on this part of the problem without delving deep into their applicability as generic operators. A few works have tried to utilize morphological neurons as a part of classification (and regression) networks when the input is a feature vector. However, these methods mainly focus on a specific problem, without going into generic theoretical analysis. In this work, we have theoretically analyzed morphological neurons and have shown that these are far more powerful than previously anticipated. Our proposed morphological block, containing dilation and erosion followed by their linear combination, represents a sum of hinge functions. Existing works show that hinge functions perform quite well in classification and regression problems. Two morphological blocks can even approximate any continuous function. However, to facilitate the theoretical analysis that we have done in this paper, we have restricted ourselves to the 1D version of the operators, where the structuring element operates on the whole input. Experimental evaluations also indicate the effectiveness of networks built with morphological neurons, over similarly structured neural networks.
DECT-MULTRA: Dual-Energy CT Image Decomposition With Learned Mixed Material Models and Efficient Clustering
Authors: Zhipeng Li, Saiprasad Ravishankar, Yong Long, Jeffrey A. Fessler
Date: 2019-01-01 00:00:00
Tasks: Clustering

Dual energy computed tomography (DECT) imaging plays an important role in advanced imaging applications due to its material decomposition capability. Image-domain decomposition operates directly on CT images using linear matrix inversion, but the decomposed material images can be severely degraded by noise and artifacts. This paper proposes a new method dubbed DECT-MULTRA for image-domain DECT material decomposition that combines conventional penalized weighted-least squares (PWLS) estimation with regularization based on a mixed union of learned transforms (MULTRA) model. Our proposed approach pre-learns a union of common-material sparsifying transforms from patches extracted from all the basis materials, and a union of cross-material sparsifying transforms from multi-material patches. The common-material transforms capture the common properties among different material images, while the cross-material transforms capture the cross-dependencies. The proposed PWLS formulation is optimized efficiently by alternating between an image update step and a sparse coding and clustering step, with both of these steps having closed-form solutions. The effectiveness of our method is validated with both XCAT phantom and clinical head data. The results demonstrate that our proposed method provides superior material image quality and decomposition accuracy compared to other competing methods.
A Hardware Friendly Unsupervised Memristive Neural Network with Weight Sharing Mechanism
Authors: Zhiri Tang, Ruohua Zhu, Peng Lin, Jin He, Hao Wang, Qijun Huang, Sheng Chang, Qiming Ma
Date: 2019-01-01 00:00:00

Memristive neural networks (MNNs), which use memristors as neurons or
synapses, have become a hot research topic recently. However, most memristors
are not compatible with mainstream integrated circuit technology and their
stabilities in large-scale are not very well so far. In this paper, a hardware
friendly MNN circuit is introduced, in which the memristive characteristics are
implemented by digital integrated circuit. Through this method, spike timing
dependent plasticity (STDP) and unsupervised learning are realized. A weight
sharing mechanism is proposed to bridge the gap of network scale and hardware
resource. Experiment results show the hardware resource is significantly saved
with it, maintaining good recognition accuracy and high speed. Moreover, the
tendency of resource increase is slower than the expansion of network scale,
which infers our method's potential on large scale neuromorphic network's
realization.
Training with the Invisibles: Obfuscating Images to Share Safely for Learning Visual Recognition Models
Authors: Tae-hoon Kim, Dongmin Kang, Kari Pulli, Jonghyun Choi
Date: 2019-01-01 00:00:00
Tasks: Attribute, Facial Landmark Detection, General Classification, image-classification, Image Classification

High-performance visual recognition systems generally require a large collection of labeled images to train. The expensive data curation can be an obstacle for improving recognition performance. Sharing more data allows training for better models. But personal and private information in the data prevent such sharing. To promote sharing visual data for learning a recognition model, we propose to obfuscate the images so that humans are not able to recognize their detailed contents, while machines can still utilize them to train new models. We validate our approach by comprehensive experiments on three challenging visual recognition tasks; image classification, attribute classification, and facial landmark detection on several datasets including SVHN, CIFAR10, Pascal VOC 2012, CelebA, and MTFL. Our method successfully obfuscates the images from humans recognition, but a machine model trained with them performs within about 1% margin (up to 0.48%) of the performance of a model trained with the original, non-obfuscated data.