pwc-archive/papers-with-abstracts huggingface dataset:
Functionality-Oriented Convolutional Filter Pruning
Authors: Zhuwei Qin, Fuxun Yu, ChenChen Liu, Xiang Chen
Date: 2018-10-12 00:00:00
Proceeding: ICLR 2019 5

The sophisticated structure of Convolutional Neural Network (CNN) allows for outstanding performance, but at the cost of intensive computation. As significant redundancies inevitably present in such a structure, many works have been proposed to prune the convolutional filters for computation cost reduction. Although extremely effective, most works are based only on quantitative characteristics of the convolutional filters, and highly overlook the qualitative interpretation of individual filter's specific functionality. In this work, we interpreted the functionality and redundancy of the convolutional filters from different perspectives, and proposed a functionality-oriented filter pruning method. With extensive experiment results, we proved the convolutional filters' qualitative significance regardless of magnitude, demonstrated significant neural network redundancy due to repetitive filter functions, and analyzed the filter functionality defection under inappropriate retraining process. Such an interpretable pruning approach not only offers outstanding computation cost optimization over previous filter pruning methods, but also interprets filter pruning process.
Deep-Waveform: A Learned OFDM Receiver Based on Deep Complex-valued Convolutional Networks
Authors: Zhongyuan Zhao, Mehmet C. Vuran, Fujuan Guo, Stephen D. Scott
Date: 2018-10-16 00:00:00

The (inverse) discrete Fourier transform (DFT/IDFT) is often perceived as essential to orthogonal frequency-division multiplexing (OFDM) systems. In this paper, a deep complex-valued convolutional network (DCCN) is developed to recover bits from time-domain OFDM signals without relying on any explicit DFT/IDFT. The DCCN can exploit the cyclic prefix (CP) of OFDM waveform for increased SNR by replacing DFT with a learned linear transform, and has the advantage of combining CP-exploitation, channel estimation, and intersymbol interference (ISI) mitigation, with a complexity of $\mathcal{O}(N^2)$. Numerical tests show that the DCCN receiver can outperform the legacy channel estimators based on ideal and approximate linear minimum mean square error (LMMSE) estimation and a conventional CP-enhanced technique in Rayleigh fading channels with various delay spreads and mobility. The proposed approach benefits from the expressive nature of complex-valued neural networks, which, however, currently lack support from popular deep learning platforms. In response, guidelines of exact and approximate implementations of a complex-valued convolutional layer are provided for the design and analysis of convolutional networks for wireless PHY. Furthermore, a suite of novel training techniques are developed to improve the convergence and generalizability of the trained model in fading channels. This work demonstrates the capability of deep neural networks in processing OFDM waveforms and the results suggest that the FFT processor in OFDM receivers can be replaced by a hardware AI accelerator.
Convolutional Collaborative Filter Network for Video Based Recommendation Systems
Authors: Cheng-Kang Hsieh, Miguel Campo, Abhinav Taliyan, Matt Nickens, Mitkumar Pandya, JJ Espinoza
Date: 2018-10-18 00:00:00
Tasks: Recommendation Systems

This analysis explores the temporal sequencing of objects in a movie trailer.
Temporal sequencing of objects in a movie trailer (e.g., a long shot of an
object vs intermittent short shots) can convey information about the type of
movie, plot of the movie, role of the main characters, and the filmmakers
cinematographic choices. When combined with historical customer data,
sequencing analysis can be used to improve predictions of customer behavior.
E.g., a customer buys tickets to a new movie and maybe the customer has seen
movies in the past that contained similar sequences. To explore object
sequencing in movie trailers, we propose a video convolutional network to
capture actions and scenes that are predictive of customers' preferences. The
model learns the specific nature of sequences for different types of objects
(e.g., cars vs faces), and the role of sequences in predicting customer future
behavior. We show how such a temporal-aware model outperforms simple feature
pooling methods proposed in our previous works and, importantly, demonstrate
the additional model explain-ability allowed by such a model.
Thermodynamics and Feature Extraction by Machine Learning
Authors: Shotaro Shiba Funai, Dimitrios Giataganas
Date: 2018-10-18 00:00:00
Tasks: BIG-bench Machine Learning

Machine learning methods are powerful in distinguishing different phases of
matter in an automated way and provide a new perspective on the study of
physical phenomena. We train a Restricted Boltzmann Machine (RBM) on data
constructed with spin configurations sampled from the Ising Hamiltonian at
different values of temperature and external magnetic field using Monte Carlo
methods. From the trained machine we obtain the flow of iterative
reconstruction of spin state configurations to faithfully reproduce the
observables of the physical system. We find that the flow of the trained RBM
approaches the spin configurations of the maximal possible specific heat which
resemble the near criticality region of the Ising model. In the special case of
the vanishing magnetic field the trained RBM converges to the critical point of
the Renormalization Group (RG) flow of the lattice model. Our results suggest
an alternative explanation of how the machine identifies the physical phase
transitions, by recognizing certain properties of the configuration like the
maximization of the specific heat, instead of associating directly the
recognition procedure with the RG flow and its fixed points. Then from the
reconstructed data we deduce the critical exponent associated to the
magnetization to find satisfactory agreement with the actual physical value. We
assume no prior knowledge about the criticality of the system and its
Hamiltonian.
Gradient Agreement as an Optimization Objective for Meta-Learning
Authors: Amir Erfan Eshratifar, David Eigen, Massoud Pedram
Date: 2018-10-18 00:00:00
Tasks: Meta-Learning

This paper presents a novel optimization method for maximizing generalization
over tasks in meta-learning. The goal of meta-learning is to learn a model for
an agent adapting rapidly when presented with previously unseen tasks. Tasks
are sampled from a specific distribution which is assumed to be similar for
both seen and unseen tasks. We focus on a family of meta-learning methods
learning initial parameters of a base model which can be fine-tuned quickly on
a new task, by few gradient steps (MAML). Our approach is based on pushing the
parameters of the model to a direction in which tasks have more agreement upon.
If the gradients of a task agree with the parameters update vector, then their
inner product will be a large positive value. As a result, given a batch of
tasks to be optimized for, we associate a positive (negative) weight to the
loss function of a task, if the inner product between its gradients and the
average of the gradients of all tasks in the batch is a positive (negative)
value. Therefore, the degree of the contribution of a task to the parameter
updates is controlled by introducing a set of weights on the loss function of
the tasks. Our method can be easily integrated with the current meta-learning
algorithms for neural networks. Our experiments demonstrate that it yields
models with better generalization compared to MAML and Reptile.
Testing Matrix Rank, Optimally
Authors: Maria-Florina Balcan, Yi Li, David P. Woodruff, Hongyang Zhang
Date: 2018-10-18 00:00:00

We show that for the problem of testing if a matrix $A \in F^{n \times n}$
has rank at most $d$, or requires changing an $\epsilon$-fraction of entries to
have rank at most $d$, there is a non-adaptive query algorithm making
$\widetilde{O}(d^2/\epsilon)$ queries. Our algorithm works for any field $F$.
This improves upon the previous $O(d^2/\epsilon^2)$ bound (SODA'03), and
bypasses an $\Omega(d^2/\epsilon^2)$ lower bound of (KDD'14) which holds if the
algorithm is required to read a submatrix. Our algorithm is the first such
algorithm which does not read a submatrix, and instead reads a carefully
selected non-adaptive pattern of entries in rows and columns of $A$. We
complement our algorithm with a matching query complexity lower bound for
non-adaptive testers over any field. We also give tight bounds of
$\widetilde{\Theta}(d^2)$ queries in the sensing model for which query access
comes in the form of $\langle X_i, A\rangle:=tr(X_i^\top A)$; perhaps
surprisingly these bounds do not depend on $\epsilon$.
  We next develop a novel property testing framework for testing numerical
properties of a real-valued matrix $A$ more generally, which includes the
stable rank, Schatten-$p$ norms, and SVD entropy. Specifically, we propose a
bounded entry model, where $A$ is required to have entries bounded by $1$ in
absolute value. We give upper and lower bounds for a wide range of problems in
this model, and discuss connections to the sensing model above.
Exploiting High-Level Semantics for No-Reference Image Quality Assessment of Realistic Blur Images
Authors: Dingquan Li, Tingting Jiang, Ming Jiang
Date: 2018-10-18 00:00:00
Tasks: Image Quality Assessment, No-Reference Image Quality Assessment

To guarantee a satisfying Quality of Experience (QoE) for consumers, it is
required to measure image quality efficiently and reliably. The neglect of the
high-level semantic information may result in predicting a clear blue sky as
bad quality, which is inconsistent with human perception. Therefore, in this
paper, we tackle this problem by exploiting the high-level semantics and
propose a novel no-reference image quality assessment method for realistic blur
images. Firstly, the whole image is divided into multiple overlapping patches.
Secondly, each patch is represented by the high-level feature extracted from
the pre-trained deep convolutional neural network model. Thirdly, three
different kinds of statistical structures are adopted to aggregate the
information from different patches, which mainly contain some common statistics
(i.e., the mean\&standard deviation, quantiles and moments). Finally, the
aggregated features are fed into a linear regression model to predict the image
quality. Experiments show that, compared with low-level features, high-level
features indeed play a more critical role in resolving the aforementioned
challenging problem for quality estimation. Besides, the proposed method
significantly outperforms the state-of-the-art methods on two realistic blur
image databases and achieves comparable performance on two synthetic blur image
databases.
A Unified Approach to Translate Classical Bandit Algorithms to the Structured Bandit Setting
Authors: Samarth Gupta, Shreyas Chaudhari, Subhojyoti Mukherjee, Gauri Joshi, Osman Yağan
Date: 2018-10-18 00:00:00
Tasks: Thompson Sampling

We consider a finite-armed structured bandit problem in which mean rewards of different arms are known functions of a common hidden parameter $\theta^*$. Since we do not place any restrictions of these functions, the problem setting subsumes several previously studied frameworks that assume linear or invertible reward functions. We propose a novel approach to gradually estimate the hidden $\theta^*$ and use the estimate together with the mean reward functions to substantially reduce exploration of sub-optimal arms. This approach enables us to fundamentally generalize any classic bandit algorithm including UCB and Thompson Sampling to the structured bandit setting. We prove via regret analysis that our proposed UCB-C algorithm (structured bandit versions of UCB) pulls only a subset of the sub-optimal arms $O(\log T)$ times while the other sub-optimal arms (referred to as non-competitive arms) are pulled $O(1)$ times. As a result, in cases where all sub-optimal arms are non-competitive, which can happen in many practical scenarios, the proposed algorithms achieve bounded regret. We also conduct simulations on the Movielens recommendations dataset to demonstrate the improvement of the proposed algorithms over existing structured bandit algorithms.
Fast deep reinforcement learning using online adjustments from the past
Authors: Steven Hansen, Pablo Sprechmann, Alexander Pritzel, André Barreto, Charles Blundell
Date: 2018-10-18 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Atari Games, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Retrieval

We propose Ephemeral Value Adjusments (EVA): a means of allowing deep
reinforcement learning agents to rapidly adapt to experience in their replay
buffer. EVA shifts the value predicted by a neural network with an estimate of
the value function found by planning over experience tuples from the replay
buffer near the current state. EVA combines a number of recent ideas around
combining episodic memory-like structures into reinforcement learning agents:
slot-based storage, content-based retrieval, and memory-based planning. We show
that EVAis performant on a demonstration task and Atari games.
Planning in Stochastic Environments with Goal Uncertainty
Authors: Sandhya Saisubramanian, Kyle Hollins Wray, Luis Pineda, Shlomo Zilberstein
Date: 2018-10-18 00:00:00
Tasks: Decision Making

We present the Goal Uncertain Stochastic Shortest Path (GUSSP) problem -- a general framework to model path planning and decision making in stochastic environments with goal uncertainty. The framework extends the stochastic shortest path (SSP) model to dynamic environments in which it is impossible to determine the exact goal states ahead of plan execution. GUSSPs introduce flexibility in goal specification by allowing a belief over possible goal configurations. The unique observations at potential goals helps the agent identify the true goal during plan execution. The partial observability is restricted to goals, facilitating the reduction to an SSP with a modified state space. We formally define a GUSSP and discuss its theoretical properties. We then propose an admissible heuristic that reduces the planning time using FLARES -- a start-of-the-art probabilistic planner. We also propose a determinization approach for solving this class of problems. Finally, we present empirical results on a search and rescue mobile robot and three other problem domains in simulation.
Contextual Topic Modeling For Dialog Systems
Authors: Chandra Khatri, Rahul Goel, Behnam Hedayatnia, Angeliki Metanillou, Anushree Venkatesh, Raefer Gabriel, Arindam Mandal
Date: 2018-10-18 00:00:00
Tasks: Chatbot, Classification, General Classification, Topic Classification, Topic Models

Accurate prediction of conversation topics can be a valuable signal for
creating coherent and engaging dialog systems. In this work, we focus on
context-aware topic classification methods for identifying topics in free-form
human-chatbot dialogs. We extend previous work on neural topic classification
and unsupervised topic keyword detection by incorporating conversational
context and dialog act features. On annotated data, we show that incorporating
context and dialog acts leads to relative gains in topic classification
accuracy by 35% and on unsupervised keyword detection recall by 11% for
conversational interactions where topics frequently span multiple utterances.
We show that topical metrics such as topical depth is highly correlated with
dialog evaluation metrics such as coherence and engagement implying that
conversational topic models can predict user satisfaction. Our work for
detecting conversation topics and keywords can be used to guide chatbots
towards coherent dialog.
Private Machine Learning in TensorFlow using Secure Computation
Authors: Morten Dahl, Jason Mancuso, Yann Dupis, Ben Decoste, Morgan Giraud, Ian Livingstone, Justin Patriquin, Gavin Uhma
Date: 2018-10-18 00:00:00
Tasks: BIG-bench Machine Learning

We present a framework for experimenting with secure multi-party computation
directly in TensorFlow. By doing so we benefit from several properties valuable
to both researchers and practitioners, including tight integration with
ordinary machine learning processes, existing optimizations for distributed
computation in TensorFlow, high-level abstractions for expressing complex
algorithms and protocols, and an expanded set of familiar tooling. We give an
open source implementation of a state-of-the-art protocol and report on
concrete benchmarks using typical models from private machine learning.
KTAN: Knowledge Transfer Adversarial Network
Authors: Peiye Liu, Wu Liu, Huadong Ma, Tao Mei, Mingoo Seok
Date: 2018-10-18 00:00:00
Tasks: image-classification, Image Classification, Knowledge Distillation, object-detection, Object Detection, Transfer Learning

To reduce the large computation and storage cost of a deep convolutional
neural network, the knowledge distillation based methods have pioneered to
transfer the generalization ability of a large (teacher) deep network to a
light-weight (student) network. However, these methods mostly focus on
transferring the probability distribution of the softmax layer in a teacher
network and thus neglect the intermediate representations. In this paper, we
propose a knowledge transfer adversarial network to better train a student
network. Our technique holistically considers both intermediate representations
and probability distributions of a teacher network. To transfer the knowledge
of intermediate representations, we set high-level teacher feature maps as a
target, toward which the student feature maps are trained. Specifically, we
arrange a Teacher-to-Student layer for enabling our framework suitable for
various student structures. The intermediate representation helps the student
network better understand the transferred generalization as compared to the
probability distribution only. Furthermore, we infuse an adversarial learning
process by employing a discriminator network, which can fully exploit the
spatial correlation of feature maps in training a student network. The
experimental results demonstrate that the proposed method can significantly
improve the performance of a student network on both image classification and
object detection tasks.
Approximate Dynamic Programming for Planning a Ride-Sharing System using Autonomous Fleets of Electric Vehicles
Authors: Lina Al-Kanj, Juliana Nascimento, Warren B. Powell
Date: 2018-10-18 00:00:00
Tasks: Autonomous Vehicles

Within a decade, almost every major auto company, along with fleet operators
such as Uber, have announced plans to put autonomous vehicles on the road. At
the same time, electric vehicles are quickly emerging as a next-generation
technology that is cost effective, in addition to offering the benefits of
reducing the carbon footprint. The combination of a centrally managed fleet of
driverless vehicles, along with the operating characteristics of electric
vehicles, is creating a transformative new technology that offers significant
cost savings with high service levels. This problem involves a dispatch problem
for assigning riders to cars, a surge pricing problem for deciding on the price
per trip and a planning problem for deciding on the fleet size. We use
approximate dynamic programming to develop high-quality operational dispatch
strategies to determine which car is best for a particular trip, when a car
should be recharged, and when it should be re-positioned to a different zone
which offers a higher density of trips. We prove that the value functions are
monotone in the battery and time dimensions and use hierarchical aggregation to
get better estimates of the value functions with a small number of
observations. Then, surge pricing is discussed using an adaptive learning
approach to decide on the price for each trip. Finally, we discuss the fleet
size problem which depends on the previous two problems.
Adversarial TableQA: Attention Supervision for Question Answering on Tables
Authors: Minseok Cho, Reinald Kim Amplayo, Seung-won Hwang, Jonghyuck Park
Date: 2018-10-18 00:00:00
Tasks: Question Answering

The task of answering a question given a text passage has shown great
developments on model performance thanks to community efforts in building
useful datasets. Recently, there have been doubts whether such rapid progress
has been based on truly understanding language. The same question has not been
asked in the table question answering (TableQA) task, where we are tasked to
answer a query given a table. We show that existing efforts, of using "answers"
for both evaluation and supervision for TableQA, show deteriorating
performances in adversarial settings of perturbations that do not affect the
answer. This insight naturally motivates to develop new models that understand
question and table more precisely. For this goal, we propose Neural Operator
(NeOp), a multi-layer sequential network with attention supervision to answer
the query given a table. NeOp uses multiple Selective Recurrent Units (SelRUs)
to further help the interpretability of the answers of the model. Experiments
show that the use of operand information to train the model significantly
improves the performance and interpretability of TableQA models. NeOp
outperforms all the previous models by a big margin.
Salience Biased Loss for Object Detection in Aerial Images
Authors: Peng Sun, Guang Chen, Guerdan Luke, Yi Shang
Date: 2018-10-18 00:00:00
Tasks: Object, object-detection, Object Detection, Object Detection In Aerial Images

Object detection in remote sensing, especially in aerial images, remains a
challenging problem due to low image resolution, complex backgrounds, and
variation of scale and angles of objects in images. In current implementations,
multi-scale based and angle-based networks have been proposed and generate
promising results with aerial image detection. In this paper, we propose a
novel loss function, called Salience Biased Loss (SBL), for deep neural
networks, which uses salience information of the input image to achieve
improved performance for object detection. Our novel loss function treats
training examples differently based on input complexity in order to avoid the
over-contribution of easy cases in the training process. In our experiments,
RetinaNet was trained with SBL to generate an one-stage detector,
SBL-RetinaNet. SBL-RetinaNet is applied to the largest existing public aerial
image dataset, DOTA. Experimental results show our proposed loss function with
the RetinaNet architecture outperformed other state-of-art object detection
models by at least 4.31 mAP, and RetinaNet by 2.26 mAP with the same inference
speed of RetinaNet.
First-order and second-order variants of the gradient descent in a unified framework
Authors: Thomas Pierrot, Nicolas Perrin, Olivier Sigaud
Date: 2018-10-18 00:00:00
Tasks: BIG-bench Machine Learning

In this paper, we provide an overview of first-order and second-order variants of the gradient descent method that are commonly used in machine learning. We propose a general framework in which 6 of these variants can be interpreted as different instances of the same approach. They are the vanilla gradient descent, the classical and generalized Gauss-Newton methods, the natural gradient descent method, the gradient covariance matrix approach, and Newton's method. Besides interpreting these methods within a single framework, we explain their specificities and show under which conditions some of them coincide.