pwc-archive/papers-with-abstracts huggingface dataset:
Getting deep recommenders fit: Bloom embeddings for sparse binary input/output networks
Authors: Joan Serrà, Alexandros Karatzoglou
Date: 2017-06-13 00:00:00

Recommendation algorithms that incorporate techniques from deep learning are
becoming increasingly popular. Due to the structure of the data coming from
recommendation domains (i.e., one-hot-encoded vectors of item preferences),
these algorithms tend to have large input and output dimensionalities that
dominate their overall size. This makes them difficult to train, due to the
limited memory of graphical processing units, and difficult to deploy on mobile
devices with limited hardware. To address these difficulties, we propose Bloom
embeddings, a compression technique that can be applied to the input and output
of neural network models dealing with sparse high-dimensional binary-coded
instances. Bloom embeddings are computationally efficient, and do not seriously
compromise the accuracy of the model up to 1/5 compression ratios. In some
cases, they even improve over the original accuracy, with relative increases up
to 12%. We evaluate Bloom embeddings on 7 data sets and compare it against 4
alternative methods, obtaining favorable results. We also discuss a number of
further advantages of Bloom embeddings, such as 'on-the-fly' constant-time
operation, zero or marginal space requirements, training time speedups, or the
fact that they do not require any change to the core model architecture or
training configuration.
Synthesizing Imperative Programs from Examples Guided by Static Analysis
Authors: Sunbeom So, Hakjoo Oh
Date: 2017-02-21 00:00:00
Tasks: Program Synthesis

We present a novel algorithm that synthesizes imperative programs for
introductory programming courses. Given a set of input-output examples and a
partial program, our algorithm generates a complete program that is consistent
with every example. Our key idea is to combine enumerative program synthesis
and static analysis, which aggressively prunes out a large search space while
guaranteeing to find, if any, a correct solution. We have implemented our
algorithm in a tool, called SIMPL, and evaluated it on 30 problems used in
introductory programming courses. The results show that SIMPL is able to solve
the benchmark problems in 6.6 seconds on average.
Accelerated Dual Learning by Homotopic Initialization
Authors: Hadi Daneshmand, Hamed Hassani, Thomas Hofmann
Date: 2017-06-13 00:00:00

Gradient descent and coordinate descent are well understood in terms of their
asymptotic behavior, but less so in a transient regime often used for
approximations in machine learning. We investigate how proper initialization
can have a profound effect on finding near-optimal solutions quickly. We show
that a certain property of a data set, namely the boundedness of the
correlations between eigenfeatures and the response variable, can lead to
faster initial progress than expected by commonplace analysis. Convex
optimization problems can tacitly benefit from that, but this automatism does
not apply to their dual formulation. We analyze this phenomenon and devise
provably good initialization strategies for dual optimization as well as
heuristics for the non-convex case, relevant for deep learning. We find our
predictions and methods to be experimentally well-supported.
Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processes
Authors: Eduardo C. Garrido-Merchán, Daniel Hernández-Lobato
Date: 2017-06-12 00:00:00
Tasks: Bayesian Optimization, Gaussian Processes

Bayesian optimization (BO) methods are useful for optimizing functions that
are expensive to evaluate, lack an analytical expression and whose evaluations
can be contaminated by noise. These methods rely on a probabilistic model of
the objective function, typically a Gaussian process (GP), upon which an
acquisition function is built. This function guides the optimization process
and measures the expected utility of performing an evaluation of the objective
at a new point. GPs assume continous input variables. When this is not the
case, such as when some of the input variables take integer values, one has to
introduce extra approximations. A common approach is to round the suggested
variable value to the closest integer before doing the evaluation of the
objective. We show that this can lead to problems in the optimization process
and describe a more principled approach to account for input variables that are
integer-valued. We illustrate in both synthetic and a real experiments the
utility of our approach, which significantly improves the results of standard
BO methods on problems involving integer-valued variables.
Guarantees for Greedy Maximization of Non-submodular Functions with Applications
Authors: Andrew An Bian, Joachim M. Buhmann, Andreas Krause, Sebastian Tschiatschek
Date: 2017-03-06 00:00:00
Proceeding: ICML 2017 8
Tasks: Experimental Design

We investigate the performance of the standard Greedy algorithm for cardinality constrained maximization of non-submodular nondecreasing set functions. While there are strong theoretical guarantees on the performance of Greedy for maximizing submodular functions, there are few guarantees for non-submodular ones. However, Greedy enjoys strong empirical performance for many important non-submodular functions, e.g., the Bayesian A-optimality objective in experimental design. We prove theoretical guarantees supporting the empirical performance. Our guarantees are characterized by a combination of the (generalized) curvature $\alpha$ and the submodularity ratio $\gamma$. In particular, we prove that Greedy enjoys a tight approximation guarantee of $\frac{1}{\alpha}(1- e^{-\gamma\alpha})$ for cardinality constrained maximization. In addition, we bound the submodularity ratio and curvature for several important real-world objectives, including the Bayesian A-optimality objective, the determinantal function of a square submatrix and certain linear programs with combinatorial constraints. We experimentally validate our theoretical findings for both synthetic and real-world applications.
Long-Term Video Interpolation with Bidirectional Predictive Network
Authors: Xiongtao Chen, Wenmin Wang, Jinzhuo Wang, Weimian Li, Baoyang Chen
Date: 2017-06-13 00:00:00

This paper considers the challenging task of long-term video interpolation.
Unlike most existing methods that only generate few intermediate frames between
existing adjacent ones, we attempt to speculate or imagine the procedure of an
episode and further generate multiple frames between two non-consecutive frames
in videos. In this paper, we present a novel deep architecture called
bidirectional predictive network (BiPN) that predicts intermediate frames from
two opposite directions. The bidirectional architecture allows the model to
learn scene transformation with time as well as generate longer video
sequences. Besides, our model can be extended to predict multiple possible
procedures by sampling different noise vectors. A joint loss composed of clues
in image and feature spaces and adversarial loss is designed to train our
model. We demonstrate the advantages of BiPN on two benchmarks Moving 2D Shapes
and UCF101 and report competitive results to recent approaches.
A Supervised Approach to Extractive Summarisation of Scientific Papers
Authors: Ed Collins, Isabelle Augenstein, Sebastian Riedel
Date: 2017-06-13 00:00:00
Proceeding: CONLL 2017 8
Tasks: Sentence

Automatic summarisation is a popular approach to reduce a document to its
main arguments. Recent research in the area has focused on neural approaches to
summarisation, which can be very data-hungry. However, few large datasets exist
and none for the traditionally popular domain of scientific publications, which
opens up challenging research avenues centered on encoding large, complex
documents. In this paper, we introduce a new dataset for summarisation of
computer science publications by exploiting a large resource of author provided
summaries and show straightforward ways of extending it further. We develop
models on the dataset making use of both neural sentence encoding and
traditionally used summarisation features and show that models which encode
sentences as well as their local and global context perform best, significantly
outperforming well-established baseline methods.
Recommendations for Marketing Campaigns in Telecommunication Business based on the footprint analysis
Authors: J. Sidorova, L. Skold, O. Rosander, L. Lundberg
Date: 2017-06-13 00:00:00
Tasks: Combinatorial Optimization, Marketing

A major investment made by a telecom operator goes into the infrastructure
and its maintenance, while business revenues are proportional to how big and
good the customer base is. We present a data-driven analytic strategy based on
combinatorial optimization and analysis of historical data. The data cover
historical mobility of the users in one region of Sweden during a week.
Applying the proposed method to the case study, we have identified the optimal
proportion of geo-demographic segments in the customer base, developed a
functionality to assess the potential of a planned marketing campaign, and
explored the problem of an optimal number and types of the geo-demographic
segments to target through marketing campaigns. With the help of fuzzy logic,
the conclusions of data analysis are automatically translated into
comprehensible recommendations in a natural language.
Fuzzy Recommendations in Marketing Campaigns
Authors: S. Podapati, L. Lundberg, L. Skold, O. Rosander, J. Sidorova
Date: 2017-06-13 00:00:00
Tasks: Marketing

The population in Sweden is growing rapidly due to immigration. In this
light, the issue of infrastructure upgrades to provide telecommunication
services is of importance. New antennas can be installed at hot spots of user
demand, which will require an investment, and/or the clientele expansion can be
carried out in a planned manner to promote the exploitation of the
infrastructure in the less loaded geographical zones. In this paper, we explore
the second alternative. Informally speaking, the term Infrastructure-Stressing
describes a user who stays in the zones of high demand, which are prone to
produce service failures, if further loaded. We have studied the
Infrastructure-Stressing population in the light of their correlation with
geo-demographic segments. This is motivated by the fact that specific
geo-demographic segments can be targeted via marketing campaigns. Fuzzy logic
is applied to create an interface between big data, numeric methods for
processing big data and a manager.
Exact Learning from an Honest Teacher That Answers Membership Queries
Authors: Nader H. Bshouty
Date: 2017-06-13 00:00:00

Given a teacher that holds a function $f:X\to R$ from some class of functions
$C$. The teacher can receive from the learner an element~$d$ in the domain $X$
(a query) and returns the value of the function in $d$, $f(d)\in R$. The
learner goal is to find $f$ with a minimum number of queries, optimal time
complexity, and optimal resources.
  In this survey, we present some of the results known from the literature,
different techniques used, some new problems, and open problems.
Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflections
Authors: Zakaria Mhammedi, Andrew Hellicar, Ashfaqur Rahman, James Bailey
Date: 2016-12-01 00:00:00
Proceeding: ICML 2017 8

The problem of learning long-term dependencies in sequences using Recurrent
Neural Networks (RNNs) is still a major challenge. Recent methods have been
suggested to solve this problem by constraining the transition matrix to be
unitary during training which ensures that its norm is equal to one and
prevents exploding gradients. These methods either have limited expressiveness
or scale poorly with the size of the network when compared with the simple RNN
case, especially when using stochastic gradient descent with a small mini-batch
size. Our contributions are as follows; we first show that constraining the
transition matrix to be unitary is a special case of an orthogonal constraint.
Then we present a new parametrisation of the transition matrix which allows
efficient training of an RNN while ensuring that the matrix is always
orthogonal. Our results show that the orthogonal constraint on the transition
matrix applied through our parametrisation gives similar benefits to the
unitary constraint, without the time complexity limitations.
Traffic-Aware Transmission Mode Selection in D2D-enabled Cellular Networks with Token System
Authors: Yiling Yuan, Tao Yang, Hui Feng, Bo Hu, Jianqiu Zhang, Bin Wang, Qiyong Lu
Date: 2017-03-02 00:00:00

We consider a D2D-enabled cellular network where user equipments (UEs) owned
by rational users are incentivized to form D2D pairs using tokens. They
exchange tokens electronically to "buy" and "sell" D2D services. Meanwhile the
devices have the ability to choose the transmission mode, i.e. receiving data
via cellular links or D2D links. Thus taking the different benefits brought by
diverse traffic types as a prior, the UEs can utilize their tokens more
efficiently via transmission mode selection. In this paper, the optimal
transmission mode selection strategy as well as token collection policy are
investigated to maximize the long-term utility in the dynamic network
environment. The optimal policy is proved to be a threshold strategy, and the
thresholds have a monotonicity property. Numerical simulations verify our
observations and the gain from transmission mode selection is observed.
SEP-Nets: Small and Effective Pattern Networks
Authors: Zhe Li, Xiaoyu Wang, Xutao Lv, Tianbao Yang
Date: 2017-06-13 00:00:00
Tasks: Binarization, Quantization

While going deeper has been witnessed to improve the performance of
convolutional neural networks (CNN), going smaller for CNN has received
increasing attention recently due to its attractiveness for mobile/embedded
applications. It remains an active and important topic how to design a small
network while retaining the performance of large and deep CNNs (e.g., Inception
Nets, ResNets). Albeit there are already intensive studies on compressing the
size of CNNs, the considerable drop of performance is still a key concern in
many designs. This paper addresses this concern with several new contributions.
First, we propose a simple yet powerful method for compressing the size of deep
CNNs based on parameter binarization. The striking difference from most
previous work on parameter binarization/quantization lies at different
treatments of $1\times 1$ convolutions and $k\times k$ convolutions ($k>1$),
where we only binarize $k\times k$ convolutions into binary patterns. The
resulting networks are referred to as pattern networks. By doing this, we show
that previous deep CNNs such as GoogLeNet and Inception-type Nets can be
compressed dramatically with marginal drop in performance. Second, in light of
the different functionalities of $1\times 1$ (data projection/transformation)
and $k\times k$ convolutions (pattern extraction), we propose a new block
structure codenamed the pattern residual block that adds transformed feature
maps generated by $1\times 1$ convolutions to the pattern feature maps
generated by $k\times k$ convolutions, based on which we design a small network
with $\sim 1$ million parameters. Combining with our parameter binarization, we
achieve better performance on ImageNet than using similar sized networks
including recently released Google MobileNets.
Deep Control - a simple automatic gain control for memory efficient and high performance training of deep convolutional neural networks
Authors: Brendan Ruff
Date: 2017-06-13 00:00:00

Training a deep convolutional neural net typically starts with a random
initialisation of all filters in all layers which severely reduces the forward
signal and back-propagated error and leads to slow and sub-optimal training.
Techniques that counter that focus on either increasing the signal or
increasing the gradients adaptively but the model behaves very differently at
the beginning of training compared to later when stable pathways through the
net have been established. To compound this problem the effective minibatch
size varies greatly between layers at different depths and between individual
filters as activation sparsity typically increases with depth leading to a
reduction in effective learning rate since gradients may superpose rather than
add and this further compounds the covariate shift problem as deeper neurons
are less able to adapt to upstream shift.
  Proposed here is a method of automatic gain control of the signal built into
each convolutional neuron that achieves equivalent or superior performance than
batch normalisation and is compatible with single sample or minibatch gradient
descent. The same model is used both for training and inference.
  The technique comprises a scaled per sample map mean subtraction from the raw
convolutional filter output followed by scaling of the difference.
A New Probabilistic Algorithm for Approximate Model Counting
Authors: Cunjing Ge, Feifei Ma, Tian Liu, Jian Zhang
Date: 2017-06-13 00:00:00
Tasks: model

Constrained counting is important in domains ranging from artificial
intelligence to software analysis. There are already a few approaches for
counting models over various types of constraints. Recently, hashing-based
approaches achieve both theoretical guarantees and scalability, but still rely
on solution enumeration. In this paper, a new probabilistic polynomial time
approximate model counter is proposed, which is also a hashing-based universal
framework, but with only satisfiability queries. A variant with a dynamic
stopping criterion is also presented. Empirical evaluation over benchmarks on
propositional logic formulas and SMT(BV) formulas shows that the approach is
promising.
Point Linking Network for Object Detection
Authors: Xinggang Wang, Kaibing Chen, Zilong Huang, Cong Yao, Wenyu Liu
Date: 2017-06-12 00:00:00
Tasks: Object, object-detection, Object Detection

Object detection is a core problem in computer vision. With the development
of deep ConvNets, the performance of object detectors has been dramatically
improved. The deep ConvNets based object detectors mainly focus on regressing
the coordinates of bounding box, e.g., Faster-R-CNN, YOLO and SSD. Different
from these methods that considering bounding box as a whole, we propose a novel
object bounding box representation using points and links and implemented using
deep ConvNets, termed as Point Linking Network (PLN). Specifically, we regress
the corner/center points of bounding-box and their links using a fully
convolutional network; then we map the corner points and their links back to
multiple bounding boxes; finally an object detection result is obtained by
fusing the multiple bounding boxes. PLN is naturally robust to object occlusion
and flexible to object scale variation and aspect ratio variation. In the
experiments, PLN with the Inception-v2 model achieves state-of-the-art
single-model and single-scale results on the PASCAL VOC 2007, the PASCAL VOC
2012 and the COCO detection benchmarks without bells and whistles. The source
code will be released.
CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos
Authors: Zheng Shou, Jonathan Chan, Alireza Zareian, Kazuyuki Miyazawa, Shih-Fu Chang
Date: 2017-03-04 00:00:00
Proceeding: CVPR 2017 7
Tasks: Action Localization, GPU, Temporal Action Localization

Temporal action localization is an important yet challenging problem. Given a
long, untrimmed video consisting of multiple action instances and complex
background contents, we need not only to recognize their action categories, but
also to localize the start time and end time of each instance. Many
state-of-the-art systems use segment-level classifiers to select and rank
proposal segments of pre-determined boundaries. However, a desirable model
should move beyond segment-level and make dense predictions at a fine
granularity in time to determine precise temporal boundaries. To this end, we
design a novel Convolutional-De-Convolutional (CDC) network that places CDC
filters on top of 3D ConvNets, which have been shown to be effective for
abstracting action semantics but reduce the temporal length of the input data.
The proposed CDC filter performs the required temporal upsampling and spatial
downsampling operations simultaneously to predict actions at the frame-level
granularity. It is unique in jointly modeling action semantics in space-time
and fine-grained temporal dynamics. We train the CDC network in an end-to-end
manner efficiently. Our model not only achieves superior performance in
detecting actions in every frame, but also significantly boosts the precision
of localizing temporal boundaries. Finally, the CDC network demonstrates a very
high efficiency with the ability to process 500 frames per second on a single
GPU server. We will update the camera-ready version and publish the source
codes online soon.
Fully Character-Level Neural Machine Translation without Explicit Segmentation
Authors: Jason Lee, Kyunghyun Cho, Thomas Hofmann
Date: 2016-10-10 00:00:00
Proceeding: TACL 2017 1
Tasks: de-en, Machine Translation, NMT, Translation

Most existing machine translation systems operate at the level of words,
relying on explicit segmentation to extract tokens. We introduce a neural
machine translation (NMT) model that maps a source character sequence to a
target character sequence without any segmentation. We employ a character-level
convolutional network with max-pooling at the encoder to reduce the length of
source representation, allowing the model to be trained at a speed comparable
to subword-level models while capturing local regularities. Our
character-to-character model outperforms a recently proposed baseline with a
subword-level encoder on WMT'15 DE-EN and CS-EN, and gives comparable
performance on FI-EN and RU-EN. We then demonstrate that it is possible to
share a single character-level encoder across multiple languages by training a
model on a many-to-one translation task. In this multilingual setting, the
character-level encoder significantly outperforms the subword-level encoder on
all the language pairs. We observe that on CS-EN, FI-EN and RU-EN, the quality
of the multilingual character-level translation even surpasses the models
specifically trained on that language pair alone, both in terms of BLEU score
and human judgment.
Video-based Person Re-identification with Accumulative Motion Context
Authors: Hao Liu, Zequn Jie, Karlekar Jayashree, Meibin Qi, Jianguo Jiang, Shuicheng Yan, Jiashi Feng
Date: 2017-01-01 00:00:00
Tasks: Person Re-Identification, Video-Based Person Re-Identification

Video based person re-identification plays a central role in realistic
security and video surveillance. In this paper we propose a novel Accumulative
Motion Context (AMOC) network for addressing this important problem, which
effectively exploits the long-range motion context for robustly identifying the
same person under challenging conditions. Given a video sequence of the same or
different persons, the proposed AMOC network jointly learns appearance
representation and motion context from a collection of adjacent frames using a
two-stream convolutional architecture. Then AMOC accumulates clues from motion
context by recurrent aggregation, allowing effective information flow among
adjacent frames and capturing dynamic gist of the persons. The architecture of
AMOC is end-to-end trainable and thus motion context can be adapted to
complement appearance clues under unfavorable conditions (e.g. occlusions).
Extensive experiments are conduced on three public benchmark datasets, i.e.,
the iLIDS-VID, PRID-2011 and MARS datasets, to investigate the performance of
AMOC. The experimental results demonstrate that the proposed AMOC network
outperforms state-of-the-arts for video-based re-identification significantly
and confirm the advantage of exploiting long-range motion context for video
based person re-identification, validating our motivation evidently.