pwc-archive/papers-with-abstracts huggingface dataset:
Transferring Physical Motion Between Domains for Neural Inertial Tracking
Authors: Changhao Chen, Yishu Miao, Chris Xiaoxuan Lu, Phil Blunsom, Andrew Markham, Niki Trigoni
Date: 2018-10-04 00:00:00
Tasks: Domain Adaptation

Inertial information processing plays a pivotal role in ego-motion awareness
for mobile agents, as inertial measurements are entirely egocentric and not
environment dependent. However, they are affected greatly by changes in sensor
placement/orientation or motion dynamics, and it is infeasible to collect
labelled data from every domain. To overcome the challenges of domain
adaptation on long sensory sequences, we propose a novel framework that
extracts domain-invariant features of raw sequences from arbitrary domains, and
transforms to new domains without any paired data. Through the experiments, we
demonstrate that it is able to efficiently and effectively convert the raw
sequence from a new unlabelled target domain into an accurate inertial
trajectory, benefiting from the physical motion knowledge transferred from the
labelled source domain. We also conduct real-world experiments to show our
framework can reconstruct physically meaningful trajectories from raw IMU
measurements obtained with a standard mobile phone in various attachments.
Unsupervised Adversarial Visual Level Domain Adaptation for Learning Video Object Detectors from Images
Authors: Avisek Lahiri, Charan Reddy, Prabir Kumar Biswas
Date: 2018-10-04 00:00:00
Tasks: Domain Adaptation, Image-to-Image Translation, Object, object-detection, Object Detection, Translation, Video Understanding

Deep learning based object detectors require thousands of diversified
bounding box and class annotated examples. Though image object detectors have
shown rapid progress in recent years with the release of multiple large-scale
static image datasets, object detection on videos still remains an open problem
due to scarcity of annotated video frames. Having a robust video object
detector is an essential component for video understanding and curating
large-scale automated annotations in videos. Domain difference between images
and videos makes the transferability of image object detectors to videos
sub-optimal. The most common solution is to use weakly supervised annotations
where a video frame has to be tagged for presence/absence of object categories.
This still takes up manual effort. In this paper we take a step forward by
adapting the concept of unsupervised adversarial image-to-image translation to
perturb static high quality images to be visually indistinguishable from a set
of video frames. We assume the presence of a fully annotated static image
dataset and an unannotated video dataset. Object detector is trained on
adversarially transformed image dataset using the annotations of the original
dataset. Experiments on Youtube-Objects and Youtube-Objects-Subset datasets
with two contemporary baseline object detectors reveal that such unsupervised
pixel level domain adaptation boosts the generalization performance on video
frames compared to direct application of original image object detector. Also,
we achieve competitive performance compared to recent baselines of weakly
supervised methods. This paper can be seen as an application of image
translation for cross domain object detection.
Finding Solutions to Generative Adversarial Privacy
Authors: Dae Hyun Kim, Taeyoung Kong, Seungbin Jeong
Date: 2018-10-04 00:00:00

We present heuristics for solving the maximin problem induced by the
generative adversarial privacy setting for linear and convolutional neural
network (CNN) adversaries. In the linear adversary setting, we present a greedy
algorithm for approximating the optimal solution for the privatizer, which
performs better as the number of instances increases. We also provide an
analysis of the algorithm to show that it not only removes the features most
correlated with the private label first, but also preserves the prediction
accuracy of public labels that are sufficiently independent of the features
that are relevant to the private label. In the CNN adversary setting, we
present a method of hiding selected information from the adversary while
preserving the others through alternately optimizing the goals of the
privatizer and the adversary using neural network backpropagation. We
experimentally show that our method succeeds on a fixed adversary.
Towards Fast and Energy-Efficient Binarized Neural Network Inference on FPGA
Authors: Cheng Fu, Shilin Zhu, Hao Su, Ching-En Lee, Jishen Zhao
Date: 2018-10-04 00:00:00

Binarized Neural Network (BNN) removes bitwidth redundancy in classical CNN
by using a single bit (-1/+1) for network parameters and intermediate
representations, which has greatly reduced the off-chip data transfer and
storage overhead. However, a large amount of computation redundancy still
exists in BNN inference. By analyzing local properties of images and the
learned BNN kernel weights, we observe an average of $\sim$78% input similarity
and $\sim$59% weight similarity among weight kernels, measured by our proposed
metric in common network architectures. Thus there does exist redundancy that
can be exploited to further reduce the amount of on-chip computations.
  Motivated by the observation, in this paper, we proposed two types of fast
and energy-efficient architectures for BNN inference. We also provide analysis
and insights to pick the better strategy of these two for different datasets
and network models. By reusing the results from previous computation, much
cycles for data buffer access and computations can be skipped. By experiments,
we demonstrate that 80% of the computation and 40% of the buffer access can be
skipped by exploiting BNN similarity. Thus, our design can achieve 17%
reduction in total power consumption, 54% reduction in on-chip power
consumption and 2.4$\times$ maximum speedup, compared to the baseline without
applying our reuse technique. Our design also shows 1.9$\times$ more
area-efficiency compared to state-of-the-art BNN inference design. We believe
our deployment of BNN on FPGA leads to a promising future of running deep
learning models on mobile devices.
Weakly-Convex Concave Min-Max Optimization: Provable Algorithms and Applications in Machine Learning
Authors: Hassan Rafique, Mingrui Liu, Qihang Lin, Tianbao Yang
Date: 2018-10-04 00:00:00
Tasks: BIG-bench Machine Learning

Min-max problems have broad applications in machine learning, including learning with non-decomposable loss and learning with robustness to data distribution. Convex-concave min-max problem is an active topic of research with efficient algorithms and sound theoretical foundations developed. However, it remains a challenge to design provably efficient algorithms for non-convex min-max problems with or without smoothness. In this paper, we study a family of non-convex min-max problems, whose objective function is weakly convex in the variables of minimization and is concave in the variables of maximization. We propose a proximally guided stochastic subgradient method and a proximally guided stochastic variance-reduced method for the non-smooth and smooth instances, respectively, in this family of problems. We analyze the time complexities of the proposed methods for finding a nearly stationary point of the outer minimization problem corresponding to the min-max problem.
Gradient Descent Provably Optimizes Over-parameterized Neural Networks
Authors: Simon S. Du, Xiyu Zhai, Barnabas Poczos, Aarti Singh
Date: 2018-10-04 00:00:00
Proceeding: ICLR 2019 5

One of the mysteries in the success of neural networks is randomly
initialized first order methods like gradient descent can achieve zero training
loss even though the objective function is non-convex and non-smooth. This
paper demystifies this surprising phenomenon for two-layer fully connected ReLU
activated neural networks. For an $m$ hidden node shallow neural network with
ReLU activation and $n$ training data, we show as long as $m$ is large enough
and no two inputs are parallel, randomly initialized gradient descent converges
to a globally optimal solution at a linear convergence rate for the quadratic
loss function.
  Our analysis relies on the following observation: over-parameterization and
random initialization jointly restrict every weight vector to be close to its
initialization for all iterations, which allows us to exploit a strong
convexity-like property to show that gradient descent converges at a global
linear rate to the global optimum. We believe these insights are also useful in
analyzing deep models and other first order methods.
Classical linear logic, cobordisms and categorical semantics of categorial grammars
Authors: Sergey Slavnov
Date: 2018-10-04 00:00:00

We propose a categorial grammar based on classical multiplicative linear
logic.
  This can be seen as an extension of abstract categorial grammars (ACG) and is
at least as expressive. However, constituents of {\it linear logic grammars
(LLG)} are not abstract ${\lambda}$-terms, but simply tuples of words with
labeled endpoints, we call them {\it multiwords}. At least, this gives a
concrete and intuitive representation of ACG.
  A key observation is that the class of multiwords has a fundamental algebraic
structure. Namely, multiwords can be organized in a category, very similar to
the category of topological cobordisms. This category is symmetric monoidal
closed and compact closed and thus is a model of linear $\lambda$-calculus and
classical linear logic. We think that this category is interesting on its own
right. In particular, it might provide categorical representation for other
formalisms.
  On the other hand, many models of language semantics are based on commutative
logic or, more generally, on symmetric monoidal closed categories. But the
category of {\it word cobordisms} is a category of language elements, which is
itself symmetric monoidal closed and independent of any grammar. Thus, it might
prove useful in understanding language semantics as well.
Gradient descent aligns the layers of deep linear networks
Authors: Ziwei Ji, Matus Telgarsky
Date: 2018-10-04 00:00:00
Proceeding: ICLR 2019 5

This paper establishes risk convergence and asymptotic weight matrix
alignment --- a form of implicit regularization --- of gradient flow and
gradient descent when applied to deep linear networks on linearly separable
data. In more detail, for gradient flow applied to strictly decreasing loss
functions (with similar results for gradient descent with particular decreasing
step sizes): (i) the risk converges to 0; (ii) the normalized i-th weight
matrix asymptotically equals its rank-1 approximation $u_iv_i^{\top}$; (iii)
these rank-1 matrices are aligned across layers, meaning
$|v_{i+1}^{\top}u_i|\to1$. In the case of the logistic loss (binary cross
entropy), more can be said: the linear function induced by the network --- the
product of its weight matrices --- converges to the same direction as the
maximum margin solution. This last property was identified in prior work, but
only under assumptions on gradient descent which here are implied by the
alignment phenomenon.
Robust Estimation and Generative Adversarial Nets
Authors: Chao Gao, jiyi LIU, Yuan YAO, Weizhi Zhu
Date: 2018-10-04 00:00:00

Robust estimation under Huber's $\epsilon$-contamination model has become an
important topic in statistics and theoretical computer science. Statistically
optimal procedures such as Tukey's median and other estimators based on depth
functions are impractical because of their computational intractability. In
this paper, we establish an intriguing connection between $f$-GANs and various
depth functions through the lens of $f$-Learning. Similar to the derivation of
$f$-GANs, we show that these depth functions that lead to statistically optimal
robust estimators can all be viewed as variational lower bounds of the total
variation distance in the framework of $f$-Learning. This connection opens the
door of computing robust estimators using tools developed for training GANs. In
particular, we show in both theory and experiments that some appropriate
structures of discriminator networks with hidden layers in GANs lead to
statistically optimal robust location estimators for both Gaussian distribution
and general elliptical distributions where first moment may not exist.
Polar Feature Based Deep Architectures for Automatic Modulation Classification Considering Channel Fading
Authors: Chieh-Fang Teng, Ching-Chun Liao, Chun-Hsiang Chen, An-Yeu Wu
Date: 2018-10-04 00:00:00
Tasks: Deep Learning, General Classification

To develop intelligent receivers, automatic modulation classification (AMC)
plays an important role for better spectrum utilization. The emerging deep
learning (DL) technique has received much attention in AMC due to its superior
performance in classifying data with deep structure. In this work, a novel
polar-based deep learning architecture with channel compensation network (CCN)
is proposed. Our test results show that learning features from polar domain
(r-theta) can improve recognition accuracy by 5% and reduce training overhead
by 48%. Besides, the proposed CCN is also robust to channel fading, such as
amplitude and phase offsets, and can improve the recognition accuracy by 14%
under practical channel environments.
Detecting DGA domains with recurrent neural networks and side information
Authors: Ryan R. Curtin, Andrew B. Gardner, Slawomir Grzonkowski, Alexey Kleymenov, Alejandro Mosquera
Date: 2018-10-04 00:00:00
Tasks: Blocking, Malware Detection

Modern malware typically makes use of a domain generation algorithm (DGA) to avoid command and control domains or IPs being seized or sinkholed. This means that an infected system may attempt to access many domains in an attempt to contact the command and control server. Therefore, the automatic detection of DGA domains is an important task, both for the sake of blocking malicious domains and identifying compromised hosts. However, many DGAs use English wordlists to generate plausibly clean-looking domain names; this makes automatic detection difficult. In this work, we devise a notion of difficulty for DGA families called the smashword score; this measures how much a DGA family looks like English words. We find that this measure accurately reflects how much a DGA family's domains look like they are made from natural English words. We then describe our new modeling approach, which is a combination of a novel recurrent neural network architecture with domain registration side information. Our experiments show the model is capable of effectively identifying domains generated by difficult DGA families. Our experiments also show that our model outperforms existing approaches, and is able to reliably detect difficult DGA families such as matsnu, suppobox, rovnix, and others. The model's performance compared to the state of the art is best for DGA families that resemble English words. We believe that this model could either be used in a standalone DGA domain detector---such as an endpoint security application---or alternately the model could be used as a part of a larger malware detection system.
Convergence of the Expectation-Maximization Algorithm Through Discrete-Time Lyapunov Stability Theory
Authors: Orlando Romero, Sarthak Chatterjee, Sérgio Pequito
Date: 2018-10-04 00:00:00
Tasks: Clustering, Density Estimation

In this paper, we propose a dynamical systems perspective of the
Expectation-Maximization (EM) algorithm. More precisely, we can analyze the EM
algorithm as a nonlinear state-space dynamical system. The EM algorithm is
widely adopted for data clustering and density estimation in statistics,
control systems, and machine learning. This algorithm belongs to a large class
of iterative algorithms known as proximal point methods. In particular, we
re-interpret limit points of the EM algorithm and other local maximizers of the
likelihood function it seeks to optimize as equilibria in its dynamical system
representation. Furthermore, we propose to assess its convergence as asymptotic
stability in the sense of Lyapunov. As a consequence, we proceed by leveraging
recent results regarding discrete-time Lyapunov stability theory in order to
establish asymptotic stability (and thus, convergence) in the dynamical system
representation of the EM algorithm.
Transfer Incremental Learning using Data Augmentation
Authors: Ghouthi Boukli Hacene, Vincent Gripon, Nicolas Farrugia, Matthieu Arzel, Michel Jezequel
Date: 2018-10-04 00:00:00
Tasks: Data Augmentation, Incremental Learning

Deep learning-based methods have reached state of the art performances,
relying on large quantity of available data and computational power. Such
methods still remain highly inappropriate when facing a major open machine
learning problem, which consists of learning incrementally new classes and
examples over time. Combining the outstanding performances of Deep Neural
Networks (DNNs) with the flexibility of incremental learning techniques is a
promising venue of research. In this contribution, we introduce Transfer
Incremental Learning using Data Augmentation (TILDA). TILDA is based on
pre-trained DNNs as feature extractor, robust selection of feature vectors in
subspaces using a nearest-class-mean based technique, majority votes and data
augmentation at both the training and the prediction stages. Experiments on
challenging vision datasets demonstrate the ability of the proposed method for
low complexity incremental learning, while achieving significantly better
accuracy than existing incremental counterparts.
Seq2Slate: Re-ranking and Slate Optimization with RNNs
Authors: Irwan Bello, Sayali Kulkarni, Sagar Jain, Craig Boutilier, Ed Chi, Elad Eban, Xiyang Luo, Alan Mackey, Ofer Meshi
Date: 2018-10-04 00:00:00
Proceeding: ICLR 2019 5
Tasks: Information Retrieval, Re-Ranking, Retrieval

Ranking is a central task in machine learning and information retrieval. In
this task, it is especially important to present the user with a slate of items
that is appealing as a whole. This in turn requires taking into account
interactions between items, since intuitively, placing an item on the slate
affects the decision of which other items should be placed alongside it. In
this work, we propose a sequence-to-sequence model for ranking called
seq2slate. At each step, the model predicts the next `best' item to place on
the slate given the items already selected. The sequential nature of the model
allows complex dependencies between the items to be captured directly in a
flexible and scalable way. We show how to learn the model end-to-end from weak
supervision in the form of easily obtained click-through data. We further
demonstrate the usefulness of our approach in experiments on standard ranking
benchmarks as well as in a real-world recommendation system.
Domain Specific Approximation for Object Detection
Authors: Ting-Wu Chin, Chia-Lin Yu, Matthew Halpern, Hasan Genc, Shiao-Li Tsao, Vijay Janapa Reddi
Date: 2018-10-04 00:00:00
Tasks: Object, object-detection, Object Detection

There is growing interest in object detection in advanced driver assistance
systems and autonomous robots and vehicles. To enable such innovative systems,
we need faster object detection. In this work, we investigate the trade-off
between accuracy and speed with domain-specific approximations, i.e.
category-aware image size scaling and proposals scaling, for two
state-of-the-art deep learning-based object detection meta-architectures. We
study the effectiveness of applying approximation both statically and
dynamically to understand the potential and the applicability of them. By
conducting experiments on the ImageNet VID dataset, we show that
domain-specific approximation has great potential to improve the speed of the
system without deteriorating the accuracy of object detectors, i.e. up to 7.5x
speedup for dynamic domain-specific approximation. To this end, we present our
insights toward harvesting domain-specific approximation as well as devise a
proof-of-concept runtime, AutoFocus, that exploits dynamic domain-specific
approximation.
From Soft Classifiers to Hard Decisions: How fair can we be?
Authors: Ran Canetti, Aloni Cohen, Nishanth Dikkala, Govind Ramnarayan, Sarah Scheffler, Adam Smith
Date: 2018-10-03 00:00:00
Tasks: Decision Making, Fairness

A popular methodology for building binary decision-making classifiers in the
presence of imperfect information is to first construct a non-binary "scoring"
classifier that is calibrated over all protected groups, and then to
post-process this score to obtain a binary decision. We study the feasibility
of achieving various fairness properties by post-processing calibrated scores,
and then show that deferring post-processors allow for more fairness conditions
to hold on the final decision. Specifically, we show:
  1. There does not exist a general way to post-process a calibrated classifier
to equalize protected groups' positive or negative predictive value (PPV or
NPV). For certain "nice" calibrated classifiers, either PPV or NPV can be
equalized when the post-processor uses different thresholds across protected
groups, though there exist distributions of calibrated scores for which the two
measures cannot be both equalized. When the post-processing consists of a
single global threshold across all groups, natural fairness properties, such as
equalizing PPV in a nontrivial way, do not hold even for "nice" classifiers.
  2. When the post-processing is allowed to `defer' on some decisions (that is,
to avoid making a decision by handing off some examples to a separate process),
then for the non-deferred decisions, the resulting classifier can be made to
equalize PPV, NPV, false positive rate (FPR) and false negative rate (FNR)
across the protected groups. This suggests a way to partially evade the
impossibility results of Chouldechova and Kleinberg et al., which preclude
equalizing all of these measures simultaneously. We also present different
deferring strategies and show how they affect the fairness properties of the
overall system.
  We evaluate our post-processing techniques using the COMPAS data set from
2016.
Image and Encoded Text Fusion for Multi-Modal Classification
Authors: Ignazio Gallo, Alessandro Calefati, Shah Nawaz, Muhammad Kamran Janjua
Date: 2018-10-03 00:00:00
Tasks: Classification, General Classification, Multi-modal Classification

Multi-modal approaches employ data from multiple input streams such as
textual and visual domains. Deep neural networks have been successfully
employed for these approaches. In this paper, we present a novel multi-modal
approach that fuses images and text descriptions to improve multi-modal
classification performance in real-world scenarios. The proposed approach
embeds an encoded text onto an image to obtain an information-enriched image.
To learn feature representations of resulting images, standard Convolutional
Neural Networks (CNNs) are employed for the classification task. We demonstrate
how a CNN based pipeline can be used to learn representations of the novel
fusion approach. We compare our approach with individual sources on two
large-scale multi-modal classification datasets while obtaining encouraging
results. Furthermore, we evaluate our approach against two famous multi-modal
strategies namely early fusion and late fusion.