pwc-archive/papers-with-abstracts huggingface dataset:
Accelerated Optimization in the PDE Framework: Formulations for the Active Contour Case
Authors: Anthony Yezzi, Ganesh Sundaramoorthi
Date: 2017-11-27 00:00:00
Tasks: parameter estimation

Following the seminal work of Nesterov, accelerated optimization methods have
been used to powerfully boost the performance of first-order, gradient-based
parameter estimation in scenarios where second-order optimization strategies
are either inapplicable or impractical. Not only does accelerated gradient
descent converge considerably faster than traditional gradient descent, but it
also performs a more robust local search of the parameter space by initially
overshooting and then oscillating back as it settles into a final
configuration, thereby selecting only local minimizers with a basis of
attraction large enough to contain the initial overshoot. This behavior has
made accelerated and stochastic gradient search methods particularly popular
within the machine learning community. In their recent PNAS 2016 paper,
Wibisono, Wilson, and Jordan demonstrate how a broad class of accelerated
schemes can be cast in a variational framework formulated around the Bregman
divergence, leading to continuum limit ODE's. We show how their formulation may
be further extended to infinite dimension manifolds (starting here with the
geometric space of curves and surfaces) by substituting the Bregman divergence
with inner products on the tangent space and explicitly introducing a
distributed mass model which evolves in conjunction with the object of interest
during the optimization process. The co-evolving mass model, which is
introduced purely for the sake of endowing the optimization with helpful
dynamics, also links the resulting class of accelerated PDE based optimization
schemes to fluid dynamical formulations of optimal mass transport.
Homotopy Parametric Simplex Method for Sparse Learning
Authors: Haotian Pang, Robert Vanderbei, Han Liu, Tuo Zhao
Date: 2017-04-04 00:00:00
Tasks: regression, Sparse Learning

High dimensional sparse learning has imposed a great computational challenge
to large scale data analysis. In this paper, we are interested in a broad class
of sparse learning approaches formulated as linear programs parametrized by a
{\em regularization factor}, and solve them by the parametric simplex method
(PSM). Our parametric simplex method offers significant advantages over other
competing methods: (1) PSM naturally obtains the complete solution path for all
values of the regularization parameter; (2) PSM provides a high precision dual
certificate stopping criterion; (3) PSM yields sparse solutions through very
few iterations, and the solution sparsity significantly reduces the
computational cost per iteration. Particularly, we demonstrate the superiority
of PSM over various sparse learning approaches, including Dantzig selector for
sparse linear regression, LAD-Lasso for sparse robust linear regression, CLIME
for sparse precision matrix estimation, sparse differential network estimation,
and sparse Linear Programming Discriminant (LPD) analysis. We then provide
sufficient conditions under which PSM always outputs sparse solutions such that
its computational performance can be significantly boosted. Thorough numerical
experiments are provided to demonstrate the outstanding performance of the PSM
method.
Differentiable Learning of Logical Rules for Knowledge Base Reasoning
Authors: Fan Yang, Zhilin Yang, William W. Cohen
Date: 2017-02-27 00:00:00
Proceeding: NeurIPS 2017 12

We study the problem of learning probabilistic first-order logical rules for
knowledge base reasoning. This learning problem is difficult because it
requires learning the parameters in a continuous space as well as the structure
in a discrete space. We propose a framework, Neural Logic Programming, that
combines the parameter and structure learning of first-order logical rules in
an end-to-end differentiable model. This approach is inspired by a
recently-developed differentiable logic called TensorLog, where inference tasks
can be compiled into sequences of differentiable operations. We design a neural
controller system that learns to compose these operations. Empirically, our
method outperforms prior work on multiple knowledge base benchmark datasets,
including Freebase and WikiMovies.
Learning multiple visual domains with residual adapters
Authors: Sylvestre-Alvise Rebuffi, Hakan Bilen, Andrea Vedaldi
Date: 2017-05-22 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Continual Learning

There is a growing interest in learning data representations that work well
for many different types of problems and data. In this paper, we look in
particular at the task of learning a single visual representation that can be
successfully utilized in the analysis of very different types of images, from
dog breeds to stop signs and digits. Inspired by recent work on learning
networks that predict the parameters of another, we develop a tunable deep
network architecture that, by means of adapter residual modules, can be steered
on the fly to diverse visual domains. Our method achieves a high degree of
parameter sharing while maintaining or even improving the accuracy of
domain-specific representations. We also introduce the Visual Decathlon
Challenge, a benchmark that evaluates the ability of representations to capture
simultaneously ten very different visual domains and measures their ability to
recognize well uniformly.
Time-Contrastive Learning Based DNN Bottleneck Features for Text-Dependent Speaker Verification
Authors: Achintya Kr. Sarkar, Zheng-Hua Tan
Date: 2017-04-06 00:00:00
Tasks: Contrastive Learning, Speaker Verification, Text-Dependent Speaker Verification

In this paper, we present a time-contrastive learning (TCL) based bottleneck (BN)feature extraction method for speech signals with an application to text-dependent (TD) speaker verification (SV). It is well-known that speech signals exhibit quasi-stationary behavior in and only in a short interval, and the TCL method aims to exploit this temporal structure. More specifically, it trains deep neural networks (DNNs) to discriminate temporal events obtained by uniformly segmenting speech signals, in contrast to existing DNN based BN feature extraction methods that train DNNs using labeled data to discriminate speakers or pass-phrases or phones or a combination of them. In the context of speaker verification, speech data of fixed pass-phrases are used for TCL-BN training, while the pass-phrases used for TCL-BN training are excluded from being used for SV, so that the learned features can be considered generic. The method is evaluated on the RedDots Challenge 2016 database. Experimental results show that TCL-BN is superior to the existing speaker and pass-phrase discriminant BN features and the Mel-frequency cepstral coefficient feature for text-dependent speaker verification.
Separating Self-Expression and Visual Content in Hashtag Supervision
Authors: Andreas Veit, Maximilian Nickel, Serge Belongie, Laurens van der Maaten
Date: 2017-11-27 00:00:00
Proceeding: CVPR 2018 6
Tasks: Retrieval

The variety, abundance, and structured nature of hashtags make them an
interesting data source for training vision models. For instance, hashtags have
the potential to significantly reduce the problem of manual supervision and
annotation when learning vision models for a large number of concepts. However,
a key challenge when learning from hashtags is that they are inherently
subjective because they are provided by users as a form of self-expression. As
a consequence, hashtags may have synonyms (different hashtags referring to the
same visual content) and may be ambiguous (the same hashtag referring to
different visual content). These challenges limit the effectiveness of
approaches that simply treat hashtags as image-label pairs. This paper presents
an approach that extends upon modeling simple image-label pairs by modeling the
joint distribution of images, hashtags, and users. We demonstrate the efficacy
of such approaches in image tagging and retrieval experiments, and show how the
joint model can be used to perform user-conditional retrieval and tagging.
Lexical-semantic resources: yet powerful resources for automatic personality classification
Authors: Xuan-Son Vu, Lucie Flekova, Lili Jiang, Iryna Gurevych
Date: 2017-11-27 00:00:00
Proceeding: GWC 2018 1
Tasks: Classification, General Classification, Word Sense Disambiguation

In this paper, we aim to reveal the impact of lexical-semantic resources,
used in particular for word sense disambiguation and sense-level semantic
categorization, on automatic personality classification task. While stylistic
features (e.g., part-of-speech counts) have been shown their power in this
task, the impact of semantics beyond targeted word lists is relatively
unexplored. We propose and extract three types of lexical-semantic features,
which capture high-level concepts and emotions, overcoming the lexical gap of
word n-grams. Our experimental results are comparable to state-of-the-art
methods, while no personality-specific resources are required.
Distilling a Neural Network Into a Soft Decision Tree
Authors: Nicholas Frosst, Geoffrey Hinton
Date: 2017-11-27 00:00:00
Tasks: General Classification

Deep neural networks have proved to be a very effective way to perform
classification tasks. They excel when the input data is high dimensional, the
relationship between the input and the output is complicated, and the number of
labeled training examples is large. But it is hard to explain why a learned
network makes a particular classification decision on a particular test case.
This is due to their reliance on distributed hierarchical representations. If
we could take the knowledge acquired by the neural net and express the same
knowledge in a model that relies on hierarchical decisions instead, explaining
a particular decision would be much easier. We describe a way of using a
trained neural net to create a type of soft decision tree that generalizes
better than one learned directly from the training data.
Data Dependent Kernel Approximation using Pseudo Random Fourier Features
Authors: Bharath Bhushan Damodaran, Nicolas Courty, Philippe-Henri Gosselin
Date: 2017-11-27 00:00:00

Kernel methods are powerful and flexible approach to solve many problems in
machine learning. Due to the pairwise evaluations in kernel methods, the
complexity of kernel computation grows as the data size increases; thus the
applicability of kernel methods is limited for large scale datasets. Random
Fourier Features (RFF) has been proposed to scale the kernel method for solving
large scale datasets by approximating kernel function using randomized Fourier
features. While this method proved very popular, still it exists shortcomings
to be effectively used. As RFF samples the randomized features from a
distribution independent of training data, it requires sufficient large number
of feature expansions to have similar performances to kernelized classifiers,
and this is proportional to the number samples in the dataset. Thus, reducing
the number of feature dimensions is necessary to effectively scale to large
datasets. In this paper, we propose a kernel approximation method in a data
dependent way, coined as Pseudo Random Fourier Features (PRFF) for reducing the
number of feature dimensions and also to improve the prediction performance.
The proposed approach is evaluated on classification and regression problems
and compared with the RFF, orthogonal random features and Nystr{\"o}m approach
GazeGAN - Unpaired Adversarial Image Generation for Gaze Estimation
Authors: Matan Sela, Pingmei Xu, Junfeng He, Vidhya Navalpakkam, Dmitry Lagun
Date: 2017-11-27 00:00:00
Tasks: Gaze Estimation, Image Generation, Medical Diagnosis

Recent research has demonstrated the ability to estimate gaze on mobile
devices by performing inference on the image from the phone's front-facing
camera, and without requiring specialized hardware. While this offers wide
potential applications such as in human-computer interaction, medical diagnosis
and accessibility (e.g., hands free gaze as input for patients with motor
disorders), current methods are limited as they rely on collecting data from
real users, which is a tedious and expensive process that is hard to scale
across devices. There have been some attempts to synthesize eye region data
using 3D models that can simulate various head poses and camera settings,
however these lack in realism.
  In this paper, we improve upon a recently suggested method, and propose a
generative adversarial framework to generate a large dataset of high resolution
colorful images with high diversity (e.g., in subjects, head pose, camera
settings) and realism, while simultaneously preserving the accuracy of gaze
labels. The proposed approach operates on extended regions of the eye, and even
completes missing parts of the image. Using this rich synthesized dataset, and
without using any additional training data from real users, we demonstrate
improvements over state-of-the-art for estimating 2D gaze position on mobile
devices. We further demonstrate cross-device generalization of model
performance, as well as improved robustness to diverse head pose, blur and
distance.
Table-to-text Generation by Structure-aware Seq2seq Learning
Authors: Tianyu Liu, Kexiang Wang, Lei Sha, Baobao Chang, Zhifang Sui
Date: 2017-11-27 00:00:00
Tasks: Table-to-Text Generation, Text Generation

Table-to-text generation aims to generate a description for a factual table
which can be viewed as a set of field-value records. To encode both the content
and the structure of a table, we propose a novel structure-aware seq2seq
architecture which consists of field-gating encoder and description generator
with dual attention. In the encoding phase, we update the cell memory of the
LSTM unit by a field gate and its corresponding field value in order to
incorporate field information into table representation. In the decoding phase,
dual attention mechanism which contains word level attention and field level
attention is proposed to model the semantic relevance between the generated
description and the table. We conduct experiments on the \texttt{WIKIBIO}
dataset which contains over 700k biographies and corresponding infoboxes from
Wikipedia. The attention visualizations and case studies show that our model is
capable of generating coherent and informative descriptions based on the
comprehensive understanding of both the content and the structure of a table.
Automatic evaluations also show our model outperforms the baselines by a great
margin. Code for this work is available on
https://github.com/tyliupku/wiki2bio.
Language Bootstrapping: Learning Word Meanings From Perception-Action Association
Authors: Giampiero Salvi, Luis Montesano, Alexandre Bernardino, José Santos-Victor
Date: 2017-11-27 00:00:00
Tasks: Language Acquisition, speech-recognition, Speech Recognition

We address the problem of bootstrapping language acquisition for an
artificial system similarly to what is observed in experiments with human
infants. Our method works by associating meanings to words in manipulation
tasks, as a robot interacts with objects and listens to verbal descriptions of
the interactions. The model is based on an affordance network, i.e., a mapping
between robot actions, robot perceptions, and the perceived effects of these
actions upon objects. We extend the affordance model to incorporate spoken
words, which allows us to ground the verbal symbols to the execution of actions
and the perception of the environment. The model takes verbal descriptions of a
task as the input and uses temporal co-occurrence to create links between
speech utterances and the involved objects, actions, and effects. We show that
the robot is able form useful word-to-meaning associations, even without
considering grammatical structure in the learning process and in the presence
of recognition errors. These word-to-meaning associations are embedded in the
robot's own understanding of its actions. Thus, they can be directly used to
instruct the robot to perform tasks and also allow to incorporate context in
the speech recognition task. We believe that the encouraging results with our
approach may afford robots with a capacity to acquire language descriptors in
their operation's environment as well as to shed some light as to how this
challenging process develops with human infants.
Deep Supervised Discrete Hashing
Authors: Qi Li, Zhenan Sun, Ran He, Tieniu Tan
Date: 2017-05-31 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Deep Hashing, General Classification, Image Retrieval, Retrieval

With the rapid growth of image and video data on the web, hashing has been
extensively studied for image or video search in recent years. Benefit from
recent advances in deep learning, deep hashing methods have achieved promising
results for image retrieval. However, there are some limitations of previous
deep hashing methods (e.g., the semantic information is not fully exploited).
In this paper, we develop a deep supervised discrete hashing algorithm based on
the assumption that the learned binary codes should be ideal for
classification. Both the pairwise label information and the classification
information are used to learn the hash codes within one stream framework. We
constrain the outputs of the last layer to be binary codes directly, which is
rarely investigated in deep hashing algorithm. Because of the discrete nature
of hash codes, an alternating minimization method is used to optimize the
objective function. Experimental results have shown that our method outperforms
current state-of-the-art methods on benchmark datasets.
MMD GAN: Towards Deeper Understanding of Moment Matching Network
Authors: Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, Barnabás Póczos
Date: 2017-05-24 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Computational Efficiency, Generative Adversarial Network

Generative moment matching network (GMMN) is a deep generative model that
differs from Generative Adversarial Network (GAN) by replacing the
discriminator in GAN with a two-sample test based on kernel maximum mean
discrepancy (MMD). Although some theoretical guarantees of MMD have been
studied, the empirical performance of GMMN is still not as competitive as that
of GAN on challenging and large benchmark datasets. The computational
efficiency of GMMN is also less desirable in comparison with GAN, partially due
to its requirement for a rather large batch size during the training. In this
paper, we propose to improve both the model expressiveness of GMMN and its
computational efficiency by introducing adversarial kernel learning techniques,
as the replacement of a fixed Gaussian kernel in the original GMMN. The new
approach combines the key ideas in both GMMN and GAN, hence we name it MMD GAN.
The new distance measure in MMD GAN is a meaningful loss that enjoys the
advantage of weak topology and can be optimized via gradient descent with
relatively small batch sizes. In our evaluation on multiple benchmark datasets,
including MNIST, CIFAR- 10, CelebA and LSUN, the performance of MMD-GAN
significantly outperforms GMMN, and is competitive with other representative
GAN works.
Production Ready Chatbots: Generate if not Retrieve
Authors: Aniruddha Tammewar, Monik Pamecha, Chirag Jain, Apurva Nagvenkar, Krupal Modi
Date: 2017-11-27 00:00:00
Tasks: Retrieval, Scheduling

In this paper, we present a hybrid model that combines a neural
conversational model and a rule-based graph dialogue system that assists users
in scheduling reminders through a chat conversation. The graph based system has
high precision and provides a grammatically accurate response but has a low
recall. The neural conversation model can cater to a variety of requests, as it
generates the responses word by word as opposed to using canned responses. The
hybrid system shows significant improvements over the existing baseline system
of rule based approach and caters to complex queries with a domain-restricted
neural model. Restricting the conversation topic and combination of graph based
retrieval system with a neural generative model makes the final system robust
enough for a real world application.
Improving OCR Accuracy on Early Printed Books by utilizing Cross Fold Training and Voting
Authors: Christian Reul, Uwe Springmann, Christoph Wick, Frank Puppe
Date: 2017-11-27 00:00:00
Tasks: Optical Character Recognition (OCR)

In this paper we introduce a method that significantly reduces the character
error rates for OCR text obtained from OCRopus models trained on early printed
books. The method uses a combination of cross fold training and confidence
based voting. After allocating the available ground truth in different subsets
several training processes are performed, each resulting in a specific OCR
model. The OCR text generated by these models then gets voted to determine the
final output by taking the recognized characters, their alternatives, and the
confidence values assigned to each character into consideration. Experiments on
seven early printed books show that the proposed method outperforms the
standard approach considerably by reducing the amount of errors by up to 50%
and more.
DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess
Authors: Eli David, Nathan S. Netanyahu, Lior Wolf
Date: 2017-11-27 00:00:00
Tasks: feature selection, Game of Chess, Position

We present an end-to-end learning method for chess, relying on deep neural
networks. Without any a priori knowledge, in particular without any knowledge
regarding the rules of chess, a deep neural network is trained using a
combination of unsupervised pretraining and supervised training. The
unsupervised training extracts high level features from a given position, and
the supervised training learns to compare two chess positions and select the
more favorable one. The training relies entirely on datasets of several million
chess games, and no further domain specific knowledge is incorporated.
  The experiments show that the resulting neural network (referred to as
DeepChess) is on a par with state-of-the-art chess playing programs, which have
been developed through many years of manual feature selection and tuning.
DeepChess is the first end-to-end machine learning-based method that results in
a grandmaster-level chess playing performance.
DeepAPT: Nation-State APT Attribution Using End-to-End Deep Neural Networks
Authors: Ishai Rosenberg, Guillaume Sicard, Eli David
Date: 2017-11-27 00:00:00
Tasks: Authorship Attribution

In recent years numerous advanced malware, aka advanced persistent threats
(APT) are allegedly developed by nation-states. The task of attributing an APT
to a specific nation-state is extremely challenging for several reasons. Each
nation-state has usually more than a single cyber unit that develops such
advanced malware, rendering traditional authorship attribution algorithms
useless. Furthermore, those APTs use state-of-the-art evasion techniques,
making feature extraction challenging. Finally, the dataset of such available
APTs is extremely small.
  In this paper we describe how deep neural networks (DNN) could be
successfully employed for nation-state APT attribution. We use sandbox reports
(recording the behavior of the APT when run dynamically) as raw input for the
neural network, allowing the DNN to learn high level feature abstractions of
the APTs itself. Using a test set of 1,000 Chinese and Russian developed APTs,
we achieved an accuracy rate of 94.6%.