pwc-archive/papers-with-abstracts huggingface dataset:
GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders
Authors: Martin Simonovsky, Nikos Komodakis
Date: 2018-02-09 00:00:00
Proceeding: ICLR 2018 1
Tasks: Decoder, Graph Embedding

Deep learning on graphs has become a popular research topic with many
applications. However, past work has concentrated on learning graph embedding
tasks, which is in contrast with advances in generative models for images and
text. Is it possible to transfer this progress to the domain of graphs? We
propose to sidestep hurdles associated with linearization of such discrete
structures by having a decoder output a probabilistic fully-connected graph of
a predefined maximum size directly at once. Our method is formulated as a
variational autoencoder. We evaluate on the challenging task of molecule
generation.
From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots
Authors: Heung-Yeung Shum, Xiaodong He, Di Li
Date: 2018-01-06 00:00:00
Tasks: Chatbot

Conversational systems have come a long way since their inception in the
1960s. After decades of research and development, we've seen progress from
Eliza and Parry in the 60's and 70's, to task-completion systems as in the
DARPA Communicator program in the 2000s, to intelligent personal assistants
such as Siri in the 2010s, to today's social chatbots like XiaoIce. Social
chatbots' appeal lies not only in their ability to respond to users' diverse
requests, but also in being able to establish an emotional connection with
users. The latter is done by satisfying users' need for communication,
affection, as well as social belonging. To further the advancement and adoption
of social chatbots, their design must focus on user engagement and take both
intellectual quotient (IQ) and emotional quotient (EQ) into account. Users
should want to engage with a social chatbot; as such, we define the success
metric for social chatbots as conversation-turns per session (CPS). Using
XiaoIce as an illustrative example, we discuss key technologies in building
social chatbots from core chat to visual awareness to skills. We also show how
XiaoIce can dynamically recognize emotion and engage the user throughout long
conversations with appropriate interpersonal responses. As we become the first
generation of humans ever living with AI, we have a responsibility to design
social chatbots to be both useful and empathetic, so they will become
ubiquitous and help society as a whole.
Communication-Computation Efficient Gradient Coding
Authors: Min Ye, Emmanuel Abbe
Date: 2018-02-09 00:00:00
Proceeding: ICML 2018 7

This paper develops coding techniques to reduce the running time of
distributed learning tasks. It characterizes the fundamental tradeoff to
compute gradients (and more generally vector summations) in terms of three
parameters: computation load, straggler tolerance and communication cost. It
further gives an explicit coding scheme that achieves the optimal tradeoff
based on recursive polynomial constructions, coding both across data subsets
and vector components. As a result, the proposed scheme allows to minimize the
running time for gradient computations. Implementations are made on Amazon EC2
clusters using Python with mpi4py package. Results show that the proposed
scheme maintains the same generalization error while reducing the running time
by $32\%$ compared to uncoded schemes and $23\%$ compared to prior coded
schemes focusing only on stragglers (Tandon et al., ICML 2017).
Certified Robustness to Adversarial Examples with Differential Privacy
Authors: Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, Suman Jana
Date: 2018-02-09 00:00:00

Adversarial examples that fool machine learning models, particularly deep neural networks, have been a topic of intense research interest, with attacks and defenses being developed in a tight back-and-forth. Most past defenses are best effort and have been shown to be vulnerable to sophisticated attacks. Recently a set of certified defenses have been introduced, which provide guarantees of robustness to norm-bounded attacks, but they either do not scale to large datasets or are limited in the types of models they can support. This paper presents the first certified defense that both scales to large networks and datasets (such as Google's Inception network for ImageNet) and applies broadly to arbitrary model types. Our defense, called PixelDP, is based on a novel connection between robustness against adversarial examples and differential privacy, a cryptographically-inspired formalism, that provides a rigorous, generic, and flexible foundation for defense.
Metric Learning via Maximizing the Lipschitz Margin Ratio
Authors: Mingzhi Dong, Xiaochen Yang, Yang Wu, Jing-Hao Xue
Date: 2018-02-09 00:00:00
Tasks: Metric Learning

In this paper, we propose the Lipschitz margin ratio and a new metric
learning framework for classification through maximizing the ratio. This
framework enables the integration of both the inter-class margin and the
intra-class dispersion, as well as the enhancement of the generalization
ability of a classifier. To introduce the Lipschitz margin ratio and its
associated learning bound, we elaborate the relationship between metric
learning and Lipschitz functions, as well as the representability and
learnability of the Lipschitz functions. After proposing the new metric
learning framework based on the introduced Lipschitz margin ratio, we also
prove that some well known metric learning algorithms can be shown as special
cases of the proposed framework. In addition, we illustrate the framework by
implementing it for learning the squared Mahalanobis metric, and by
demonstrating its encouraging results on eight popular datasets of machine
learning.
Learning Local Metrics and Influential Regions for Classification
Authors: Mingzhi Dong, Yujiang Wang, Xiaochen Yang, Jing-Hao Xue
Date: 2018-02-09 00:00:00
Tasks: Classification, General Classification, Metric Learning

The performance of distance-based classifiers heavily depends on the
underlying distance metric, so it is valuable to learn a suitable metric from
the data. To address the problem of multimodality, it is desirable to learn
local metrics. In this short paper, we define a new intuitive distance with
local metrics and influential regions, and subsequently propose a novel local
metric learning method for distance-based classification. Our key intuition is
to partition the metric space into influential regions and a background region,
and then regulate the effectiveness of each local metric to be within the
related influential regions. We learn local metrics and influential regions to
reduce the empirical hinge loss, and regularize the parameters on the basis of
a resultant learning bound. Encouraging experimental results are obtained from
various public and popular data sets.
Seeing Through Noise: Visually Driven Speaker Separation and Enhancement
Authors: Aviv Gabbay, Ariel Ephrat, Tavi Halperin, Shmuel Peleg
Date: 2017-08-22 00:00:00
Tasks: Speaker Separation

Isolating the voice of a specific person while filtering out other voices or
background noises is challenging when video is shot in noisy environments. We
propose audio-visual methods to isolate the voice of a single speaker and
eliminate unrelated sounds. First, face motions captured in the video are used
to estimate the speaker's voice, by passing the silent video frames through a
video-to-speech neural network-based model. Then the speech predictions are
applied as a filter on the noisy input audio. This approach avoids using
mixtures of sounds in the learning process, as the number of such possible
mixtures is huge, and would inevitably bias the trained model. We evaluate our
method on two audio-visual datasets, GRID and TCD-TIMIT, and show that our
method attains significant SDR and PESQ improvements over the raw
video-to-speech predictions, and a well-known audio-only method.
Estimating the Spectral Density of Large Implicit Matrices
Authors: Ryan P. Adams, Jeffrey Pennington, Matthew J. Johnson, Jamie Smith, Yaniv Ovadia, Brian Patton, James Saunderson
Date: 2018-02-09 00:00:00

Many important problems are characterized by the eigenvalues of a large
matrix. For example, the difficulty of many optimization problems, such as
those arising from the fitting of large models in statistics and machine
learning, can be investigated via the spectrum of the Hessian of the empirical
loss function. Network data can be understood via the eigenstructure of a graph
Laplacian matrix using spectral graph theory. Quantum simulations and other
many-body problems are often characterized via the eigenvalues of the solution
space, as are various dynamic systems. However, naive eigenvalue estimation is
computationally expensive even when the matrix can be represented; in many of
these situations the matrix is so large as to only be available implicitly via
products with vectors. Even worse, one may only have noisy estimates of such
matrix vector products. In this work, we combine several different techniques
for randomized estimation and show that it is possible to construct unbiased
estimators to answer a broad class of questions about the spectra of such
implicit matrices, even in the presence of noise. We validate these methods on
large-scale problems in which graph theory and random matrix theory provide
ground truth.
Fibres of Failure: Classifying errors in predictive processes
Authors: Leo Carlsson, Gunnar Carlsson, Mikael Vejdemo-Johansson
Date: 2018-02-09 00:00:00
Tasks: General Classification, Topological Data Analysis

We describe Fibres of Failure (FiFa), a method to classify failure modes of
predictive processes using the Mapper algorithm from Topological Data Analysis.
  Our method uses Mapper to build a graph model of input data stratified by
prediction error.
  Groupings found in high-error regions of the Mapper model then provide
distinct failure modes of the predictive process.
  We demonstrate FiFa on misclassifications of MNIST images with added noise,
and demonstrate two ways to use the failure mode classification: either to
produce a correction layer that adjusts predictions by similarity to the
failure modes; or to inspect members of the failure modes to illustrate and
investigate what characterizes each failure mode.
UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
Authors: Leland McInnes, John Healy, James Melville
Date: 2018-02-09 00:00:00
Tasks: BIG-bench Machine Learning, Dimensionality Reduction

UMAP (Uniform Manifold Approximation and Projection) is a novel manifold learning technique for dimension reduction. UMAP is constructed from a theoretical framework based in Riemannian geometry and algebraic topology. The result is a practical scalable algorithm that applies to real world data. The UMAP algorithm is competitive with t-SNE for visualization quality, and arguably preserves more of the global structure with superior run time performance. Furthermore, UMAP has no computational restrictions on embedding dimension, making it viable as a general purpose dimension reduction technique for machine learning.
Predicting University Students' Academic Success and Major using Random Forests
Authors: Cédric Beaulac, Jeffrey S. Rosenthal
Date: 2018-02-09 00:00:00

In this article, a large data set containing every course taken by every
undergraduate student in a major university in Canada over 10 years is
analysed. Modern machine learning algorithms can use large data sets to build
useful tools for the data provider, in this case, the university. In this
article, two classifiers are constructed using random forests. To begin, the
first two semesters of courses completed by a student are used to predict if
they will obtain an undergraduate degree. Secondly, for the students that
completed a program, their major is predicted using once again the first few
courses they have registered to. A classification tree is an intuitive and
powerful classifier and building a random forest of trees improves this
classifier. Random forests also allow for reliable variable importance
measurements. These measures explain what variables are useful to the
classifiers and can be used to better understand what is statistically related
to the students' situation. The results are two accurate classifiers and a
variable importance analysis that provides useful information to university
administrations.
Narrow Artificial Intelligence with Machine Learning for Real-Time Estimation of a Mobile Agents Location Using Hidden Markov Models
Authors: Cédric Beaulac, Fabrice Larribe
Date: 2018-02-09 00:00:00
Tasks: Position

We propose to use a supervised machine learning technique to track the
location of a mobile agent in real time. Hidden Markov Models are used to build
artificial intelligence that estimates the unknown position of a mobile target
moving in a defined environment. This narrow artificial intelligence performs
two distinct tasks. First, it provides real-time estimation of the mobile
agent's position using the forward algorithm. Second, it uses the Baum-Welch
algorithm as a statistical learning tool to gain knowledge of the mobile
target. Finally, an experimental environment is proposed, namely a video game
that we use to test our artificial intelligence. We present statistical and
graphical results to illustrate the efficiency of our method.
Make the Minority Great Again: First-Order Regret Bound for Contextual Bandits
Authors: Zeyuan Allen-Zhu, Sébastien Bubeck, Yuanzhi Li
Date: 2018-02-09 00:00:00
Proceeding: ICML 2018 7
Tasks: Multi-Armed Bandits

Regret bounds in online learning compare the player's performance to $L^*$,
the optimal performance in hindsight with a fixed strategy. Typically such
bounds scale with the square root of the time horizon $T$. The more refined
concept of first-order regret bound replaces this with a scaling $\sqrt{L^*}$,
which may be much smaller than $\sqrt{T}$. It is well known that minor variants
of standard algorithms satisfy first-order regret bounds in the full
information and multi-armed bandit settings. In a COLT 2017 open problem,
Agarwal, Krishnamurthy, Langford, Luo, and Schapire raised the issue that
existing techniques do not seem sufficient to obtain first-order regret bounds
for the contextual bandit problem. In the present paper, we resolve this open
problem by presenting a new strategy based on augmenting the policy space.
A Sinkhorn-Newton method for entropic optimal transport
Authors: Christoph Brauer, Christian Clason, Dirk Lorenz, Benedikt Wirth
Date: 2017-10-18 00:00:00

We consider the entropic regularization of discretized optimal transport and
propose to solve its optimality conditions via a logarithmic Newton iteration.
We show a quadratic convergence rate and validate numerically that the method
compares favorably with the more commonly used Sinkhorn--Knopp algorithm for
small regularization strength. We further investigate numerically the
robustness of the proposed method with respect to parameters such as the mesh
size of the discretization.
ATPboost: Learning Premise Selection in Binary Setting with ATP Feedback
Authors: Bartosz Piotrowski, Josef Urban
Date: 2018-02-09 00:00:00
Tasks: Binary Classification, General Classification

ATPboost is a system for solving sets of large-theory problems by
interleaving ATP runs with state-of-the-art machine learning of premise
selection from the proofs. Unlike many previous approaches that use multi-label
setting, the learning is implemented as binary classification that estimates
the pairwise-relevance of (theorem, premise) pairs. ATPboost uses for this the
XGBoost gradient boosting algorithm, which is fast and has state-of-the-art
performance on many tasks. Learning in the binary setting however requires
negative examples, which is nontrivial due to many alternative proofs. We
discuss and implement several solutions in the context of the ATP/ML feedback
loop, and show that ATPboost with such methods significantly outperforms the
k-nearest neighbors multilabel classifier.
Prototypical Recurrent Unit
Authors: Dingkun Long, Richong Zhang, Yongyi Mao
Date: 2016-11-20 00:00:00
Tasks: Memorization

Despite the great successes of deep learning, the effectiveness of deep
neural networks has not been understood at any theoretical depth. This work is
motivated by the thrust of developing a deeper understanding of recurrent
neural networks, particularly LSTM/GRU-like networks. As the highly complex
structure of the recurrent unit in LSTM and GRU networks makes them difficult
to analyze, our methodology in this research theme is to construct an
alternative recurrent unit that is as simple as possible and yet also captures
the key components of LSTM/GRU recurrent units. Such a unit can then be used
for the study of recurrent networks and its structural simplicity may allow
easier analysis. Towards that goal, we take a system-theoretic perspective to
design a new recurrent unit, which we call the prototypical recurrent unit
(PRU). Not only having minimal complexity, PRU is demonstrated experimentally
to have comparable performance to GRU and LSTM unit. This establishes PRU
networks as a prototype for future study of LSTM/GRU-like recurrent networks.
This paper also studies the memorization abilities of LSTM, GRU and PRU
networks, motivated by the folk belief that such networks possess long-term
memory. For this purpose, we design a simple and controllable task, called
``memorization problem'', where the networks are trained to memorize certain
targeted information. We show that the memorization performance of all three
networks depends on the amount of targeted information, the amount of
``interfering" information, and the state space dimension of the recurrent
unit. Experiments are also performed for another controllable task, the adding
problem, and similar conclusions are obtained.
Multi-dimensional imaging data recovery via minimizing the partial sum of tubal nuclear norm
Authors: Tai-Xiang Jiang, Ting-Zhu Huang, Xi-Le Zhao, Liang-Jian Deng
Date: 2017-12-15 00:00:00

In this paper, we investigate tensor recovery problems within the tensor singular value decomposition (t-SVD) framework. We propose the partial sum of the tubal nuclear norm (PSTNN) of a tensor. The PSTNN is a surrogate of the tensor tubal multi-rank. We build two PSTNN-based minimization models for two typical tensor recovery problems, i.e., the tensor completion and the tensor principal component analysis. We give two algorithms based on the alternating direction method of multipliers (ADMM) to solve proposed PSTNN-based tensor recovery models. Experimental results on the synthetic data and real-world data reveal the superior of the proposed PSTNN.
Theoretical Analysis of Sparse Subspace Clustering with Missing Entries
Authors: Manolis C. Tsakiris, Rene Vidal
Date: 2018-01-01 00:00:00
Proceeding: ICML 2018 7
Tasks: Clustering

Sparse Subspace Clustering (SSC) is a popular unsupervised machine learning
method for clustering data lying close to an unknown union of low-dimensional
linear subspaces; a problem with numerous applications in pattern recognition
and computer vision. Even though the behavior of SSC for complete data is by
now well-understood, little is known about its theoretical properties when
applied to data with missing entries. In this paper we give theoretical
guarantees for SSC with incomplete data, and analytically establish that
projecting the zero-filled data onto the observation pattern of the point being
expressed leads to a substantial improvement in performance. The main insight
that stems from our analysis is that even though the projection induces
additional missing entries, this is counterbalanced by the fact that the
projected and zero-filled data are in effect incomplete points associated with
the union of the corresponding projected subspaces, with respect to which the
point being expressed is complete. The significance of this phenomenon
potentially extends to the entire class of self-expressive methods.
Predicting Customer Churn: Extreme Gradient Boosting with Temporal Data
Authors: Bryan Gregory
Date: 2018-02-09 00:00:00
Tasks: Feature Engineering, Time Series, Time Series Analysis

Accurately predicting customer churn using large scale time-series data is a
common problem facing many business domains. The creation of model features
across various time windows for training and testing can be particularly
challenging due to temporal issues common to time-series data. In this paper,
we will explore the application of extreme gradient boosting (XGBoost) on a
customer dataset with a wide-variety of temporal features in order to create a
highly-accurate customer churn model. In particular, we describe an effective
method for handling temporally sensitive feature engineering. The proposed
model was submitted in the WSDM Cup 2018 Churn Challenge and achieved
first-place out of 575 teams.
Deep Learning for Malicious Flow Detection
Authors: Yun-Chun Chen, Yu-Jhe Li, Aragorn Tseng, TsungNan Lin
Date: 2018-02-09 00:00:00
Tasks: Deep Learning, Zero-Shot Learning

Cyber security has grown up to be a hot issue in recent years. How to
identify potential malware becomes a challenging task. To tackle this
challenge, we adopt deep learning approaches and perform flow detection on real
data. However, real data often encounters an issue of imbalanced data
distribution which will lead to a gradient dilution issue. When training a
neural network, this problem will not only result in a bias toward the majority
class but show the inability to learn from the minority classes. In this paper,
we propose an end-to-end trainable Tree-Shaped Deep Neural Network (TSDNN)
which classifies the data in a layer-wise manner. To better learn from the
minority classes, we propose a Quantity Dependent Backpropagation (QDBP)
algorithm which incorporates the knowledge of the disparity between classes. We
evaluate our method on an imbalanced data set. Experimental result demonstrates
that our approach outperforms the state-of-the-art methods and justifies that
the proposed method is able to overcome the difficulty of imbalanced learning.
We also conduct a partial flow experiment which shows the feasibility of
real-time detection and a zero-shot learning experiment which justifies the
generalization capability of deep learning in cyber security.
Detecting Zones and Threat on 3D Body for Security in Airports using Deep Machine Learning
Authors: Abel Ag Rb Guimaraes, Ghassem Tofighi
Date: 2018-02-02 00:00:00
Tasks: BIG-bench Machine Learning

In this research, it was used a segmentation and classification method to
identify threat recognition in human scanner images of airport security. The
Department of Homeland Security's (DHS) in USA has a higher false alarm,
produced from theirs algorithms using today's scanners at the airports. To
repair this problem they started a new competition at Kaggle site asking the
science community to improve their detection with new algorithms. The dataset
used in this research comes from DHS at
https://www.kaggle.com/c/passenger-screening-algorithm-challenge/data According
to DHS: "This dataset contains a large number of body scans acquired by a new
generation of millimeter wave scanner called the High Definition-Advanced
Imaging Technology (HD-AIT) system. They are comprised of volunteers wearing
different clothing types (from light summer clothes to heavy winter clothes),
different body mass indices, different genders, different numbers of threats,
and different types of threats". Using Python as a principal language, the
preprocessed of the dataset images extracted features from 200 bodies using:
intensity, intensity differences and local neighbourhood to detect, to produce
segmentation regions and label those regions to be used as a truth in a
training and test dataset. The regions are subsequently give to a CNN deep
learning classifier to predict 17 classes (that represents the body zones):
zone1, zone2, ... zone17 and zones with threat in a total of 34 zones. The
analysis showed the results of the classifier an accuracy of 98.2863% and a
loss of 0.091319, as well as an average of 100% for recall and precision.