pwc-archive/papers-with-abstracts huggingface dataset:
Exploiting random projections and sparsity with random forests and gradient boosting methods -- Application to multi-label and multi-output learning, random forest model compression and leveraging input sparsity
Authors: Arnaud Joly
Date: 2017-04-26 00:00:00
Tasks: Model Compression

Within machine learning, the supervised learning field aims at modeling the
input-output relationship of a system, from past observations of its behavior.
Decision trees characterize the input-output relationship through a series of
nested $if-then-else$ questions, the testing nodes, leading to a set of
predictions, the leaf nodes. Several of such trees are often combined together
for state-of-the-art performance: random forest ensembles average the
predictions of randomized decision trees trained independently in parallel,
while tree boosting ensembles train decision trees sequentially to refine the
predictions made by the previous ones.
  The emergence of new applications requires scalable supervised learning
algorithms in terms of computational power and memory space with respect to the
number of inputs, outputs, and observations without sacrificing accuracy. In
this thesis, we identify three main areas where decision tree methods could be
improved for which we provide and evaluate original algorithmic solutions: (i)
learning over high dimensional output spaces, (ii) learning with large sample
datasets and stringent memory constraints at prediction time and (iii) learning
over high dimensional sparse input spaces.
Riemannian Optimization for Skip-Gram Negative Sampling
Authors: Alexander Fonarev, Oleksii Hrinchuk, Gleb Gusev, Pavel Serdyukov, Ivan Oseledets
Date: 2017-04-26 00:00:00
Proceeding: ACL 2017 7
Tasks: Riemannian optimization

Skip-Gram Negative Sampling (SGNS) word embedding model, well known by its
implementation in "word2vec" software, is usually optimized by stochastic
gradient descent. However, the optimization of SGNS objective can be viewed as
a problem of searching for a good matrix with the low-rank constraint. The most
standard way to solve this type of problems is to apply Riemannian optimization
framework to optimize the SGNS objective over the manifold of required low-rank
matrices. In this paper, we propose an algorithm that optimizes SGNS objective
using Riemannian optimization and demonstrates its superiority over popular
competitors, such as the original method to train SGNS and SVD over SPPMI
matrix.
Automatic Viseme Vocabulary Construction to Enhance Continuous Lip-reading
Authors: Adriana Fernandez-Lopez, Federico M. Sukno
Date: 2017-04-26 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Clustering, Lip Reading, speech-recognition, Speech Recognition

Speech is the most common communication method between humans and involves
the perception of both auditory and visual channels. Automatic speech
recognition focuses on interpreting the audio signals, but it has been
demonstrated that video can provide information that is complementary to the
audio. Thus, the study of automatic lip-reading is important and is still an
open problem. One of the key challenges is the definition of the visual
elementary units (the visemes) and their vocabulary. Many researchers have
analyzed the importance of the phoneme to viseme mapping and have proposed
viseme vocabularies with lengths between 11 and 15 visemes. These viseme
vocabularies have usually been manually defined by their linguistic properties
and in some cases using decision trees or clustering techniques. In this work,
we focus on the automatic construction of an optimal viseme vocabulary based on
the association of phonemes with similar appearance. To this end, we construct
an automatic system that uses local appearance descriptors to extract the main
characteristics of the mouth region and HMMs to model the statistic relations
of both viseme and phoneme sequences. To compare the performance of the system
different descriptors (PCA, DCT and SIFT) are analyzed. We test our system in a
Spanish corpus of continuous speech. Our results indicate that we are able to
recognize approximately 58% of the visemes, 47% of the phonemes and 23% of the
words in a continuous speech scenario and that the optimal viseme vocabulary
for Spanish is composed by 20 visemes.
Airway segmentation from 3D chest CT volumes based on volume of interest using gradient vector flow
Authors: Qier Meng, Takayuki Kitasaka, Masahiro Oda, Junji Ueno, Kensaku MORI
Date: 2017-04-26 00:00:00
Tasks: Segmentation

Some lung diseases are related to bronchial airway structures and morphology.
Although airway segmentation from chest CT volumes is an important task in the
computer-aided diagnosis and surgery assistance systems for the chest, complete
3-D airway structure segmentation is a quite challenging task due to its
complex tree-like structure. In this paper, we propose a new airway
segmentation method from 3D chest CT volumes based on volume of interests (VOI)
using gradient vector flow (GVF). This method segments the bronchial regions by
applying the cavity enhancement filter (CEF) to trace the bronchial tree
structure from the trachea. It uses the CEF in the VOI to segment each branch.
And a tube-likeness function based on GVF and the GVF magnitude map in each VOI
are utilized to assist predicting the positions and directions of child
branches. By calculating the tube-likeness function based on GVF and the GVF
magnitude map, the airway-like candidate structures are identified and their
centrelines are extracted. Based on the extracted centrelines, we can detect
the branch points of the bifurcations and directions of the airway branches in
the next level. At the same time, a leakage detection is performed to avoid the
leakage by analysing the pixel information and the shape information of airway
candidate regions extracted in the VOI. Finally, we unify all of the extracted
bronchial regions to form an integrated airway tree. Preliminary experiments
using four cases of chest CT volumes demonstrated that the proposed method can
extract more bronchial branches in comparison with other methods.
Towards Estimating the Upper Bound of Visual-Speech Recognition: The Visual Lip-Reading Feasibility Database
Authors: Adriana Fernandez-Lopez, Oriol Martinez, Federico M. Sukno
Date: 2017-04-26 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Lip Reading, speech-recognition, Speech Recognition, Visual Speech Recognition

Speech is the most used communication method between humans and it involves
the perception of auditory and visual channels. Automatic speech recognition
focuses on interpreting the audio signals, although the video can provide
information that is complementary to the audio. Exploiting the visual
information, however, has proven challenging. On one hand, researchers have
reported that the mapping between phonemes and visemes (visual units) is
one-to-many because there are phonemes which are visually similar and
indistinguishable between them. On the other hand, it is known that some people
are very good lip-readers (e.g: deaf people). We study the limit of visual only
speech recognition in controlled conditions. With this goal, we designed a new
database in which the speakers are aware of being read and aim to facilitate
lip-reading. In the literature, there are discrepancies on whether
hearing-impaired people are better lip-readers than normal-hearing people.
Then, we analyze if there are differences between the lip-reading abilities of
9 hearing-impaired and 15 normal-hearing people. Finally, human abilities are
compared with the performance of a visual automatic speech recognition system.
In our tests, hearing-impaired participants outperformed the normal-hearing
participants but without reaching statistical significance. Human observers
were able to decode 44% of the spoken message. In contrast, the visual only
automatic system achieved 20% of word recognition rate. However, if we repeat
the comparison in terms of phonemes both obtained very similar recognition
rates, just above 50%. This suggests that the gap between human lip-reading and
automatic speech-reading might be more related to the use of context than to
the ability to interpret mouth appearance.
Deep Text Classification Can be Fooled
Authors: Bin Liang, Hongcheng Li, Miaoqiang Su, Pan Bian, Xirong Li, Wenchang Shi
Date: 2017-04-26 00:00:00
Tasks: Classification, General Classification, text-classification, Text Classification

In this paper, we present an effective method to craft text adversarial
samples, revealing one important yet underestimated fact that DNN-based text
classifiers are also prone to adversarial sample attack. Specifically,
confronted with different adversarial scenarios, the text items that are
important for classification are identified by computing the cost gradients of
the input (white-box attack) or generating a series of occluded test samples
(black-box attack). Based on these items, we design three perturbation
strategies, namely insertion, modification, and removal, to generate
adversarial samples. The experiment results show that the adversarial samples
generated by our method can successfully fool both state-of-the-art
character-level and word-level DNN-based text classifiers. The adversarial
samples can be perturbed to any desirable classes without compromising their
utilities. At the same time, the introduced perturbation is difficult to be
perceived.
Relief R-CNN : Utilizing Convolutional Features for Fast Object Detection
Authors: Guiying Li, Junlong Liu, Chunhui Jiang, Liangpeng Zhang, Minlong Lin, Ke Tang
Date: 2016-01-25 00:00:00
Tasks: object-detection, Object Detection, Real-Time Object Detection, Region Proposal

R-CNN style methods are sorts of the state-of-the-art object detection
methods, which consist of region proposal generation and deep CNN
classification. However, the proposal generation phase in this paradigm is
usually time consuming, which would slow down the whole detection time in
testing. This paper suggests that the value discrepancies among features in
deep convolutional feature maps contain plenty of useful spatial information,
and proposes a simple approach to extract the information for fast region
proposal generation in testing. The proposed method, namely Relief R-CNN
(R2-CNN), adopts a novel region proposal generator in a trained R-CNN style
model. The new generator directly generates proposals from convolutional
features by some simple rules, thus resulting in a much faster proposal
generation speed and a lower demand of computation resources. Empirical studies
show that R2-CNN could achieve the fastest detection speed with comparable
accuracy among all the compared algorithms in testing.
Other Topics You May Also Agree or Disagree: Modeling Inter-Topic Preferences using Tweets and Matrix Factorization
Authors: Akira Sasaki, Kazuaki Hanawa, Naoaki Okazaki, Kentaro Inui
Date: 2017-04-26 00:00:00
Proceeding: ACL 2017 7
Tasks: Stance Detection

We present in this paper our approach for modeling inter-topic preferences of
Twitter users: for example, those who agree with the Trans-Pacific Partnership
(TPP) also agree with free trade. This kind of knowledge is useful not only for
stance detection across multiple topics but also for various real-world
applications including public opinion surveys, electoral predictions, electoral
campaigns, and online debates. In order to extract users' preferences on
Twitter, we design linguistic patterns in which people agree and disagree about
specific topics (e.g., "A is completely wrong"). By applying these linguistic
patterns to a collection of tweets, we extract statements agreeing and
disagreeing with various topics. Inspired by previous work on item
recommendation, we formalize the task of modeling inter-topic preferences as
matrix factorization: representing users' preferences as a user-topic matrix
and mapping both users and topics onto a latent feature space that abstracts
the preferences. Our experimental results demonstrate both that our proposed
approach is useful in predicting missing preferences of users and that the
latent vector representations of topics successfully encode inter-topic
preferences.
Failures of Gradient-Based Deep Learning
Authors: Shai Shalev-Shwartz, Ohad Shamir, Shaked Shammah
Date: 2017-03-23 00:00:00
Proceeding: ICML 2017 8
Tasks: Deep Learning

In recent years, Deep Learning has become the go-to solution for a broad
range of applications, often outperforming state-of-the-art. However, it is
important, for both theoreticians and practitioners, to gain a deeper
understanding of the difficulties and limitations associated with common
approaches and algorithms. We describe four types of simple problems, for which
the gradient-based algorithms commonly used in deep learning either fail or
suffer from significant difficulties. We illustrate the failures through
practical experiments, and provide theoretical insights explaining their
source, and how they might be remedied.
A Flexible Framework for Hypothesis Testing in High-dimensions
Authors: Adel Javanmard, Jason D. Lee
Date: 2017-04-26 00:00:00
Tasks: regression, Two-sample testing, valid, Vocal Bursts Intensity Prediction

Hypothesis testing in the linear regression model is a fundamental statistical problem. We consider linear regression in the high-dimensional regime where the number of parameters exceeds the number of samples ($p> n$). In order to make informative inference, we assume that the model is approximately sparse, that is the effect of covariates on the response can be well approximated by conditioning on a relatively small number of covariates whose identities are unknown. We develop a framework for testing very general hypotheses regarding the model parameters. Our framework encompasses testing whether the parameter lies in a convex cone, testing the signal strength, and testing arbitrary functionals of the parameter. We show that the proposed procedure controls the type I error, and also analyze the power of the procedure. Our numerical experiments confirm our theoretical findings and demonstrate that we control false positive rate (type I error) near the nominal level, and have high power. By duality between hypotheses testing and confidence intervals, the proposed framework can be used to obtain valid confidence intervals for various functionals of the model parameters. For linear functionals, the length of confidence intervals is shown to be minimax rate optimal.
Anisotropic twicing for single particle reconstruction using autocorrelation analysis
Authors: Tejal Bhamre, Teng Zhang, Amit Singer
Date: 2017-04-26 00:00:00

The missing phase problem in X-ray crystallography is commonly solved using
the technique of molecular replacement, which borrows phases from a previously
solved homologous structure, and appends them to the measured Fourier
magnitudes of the diffraction patterns of the unknown structure. More recently,
molecular replacement has been proposed for solving the missing orthogonal
matrices problem arising in Kam's autocorrelation analysis for single particle
reconstruction using X-ray free electron lasers and cryo-EM. In classical
molecular replacement, it is common to estimate the magnitudes of the unknown
structure as twice the measured magnitudes minus the magnitudes of the
homologous structure, a procedure known as `twicing'. Mathematically, this is
equivalent to finding an unbiased estimator for a complex-valued scalar. We
generalize this scheme for the case of estimating real or complex valued
matrices arising in single particle autocorrelation analysis. We name this
approach "Anisotropic Twicing" because unlike the scalar case, the unbiased
estimator is not obtained by a simple magnitude isotropic correction. We
compare the performance of the least squares, twicing and anisotropic twicing
estimators on synthetic and experimental datasets. We demonstrate 3D homology
modeling in cryo-EM directly from experimental data without iterative
refinement or class averaging, for the first time.
Linear Convergence of Accelerated Stochastic Gradient Descent for Nonconvex Nonsmooth Optimization
Authors: Feihu Huang, Songcan Chen
Date: 2017-04-26 00:00:00

In this paper, we study the stochastic gradient descent (SGD) method for the
nonconvex nonsmooth optimization, and propose an accelerated SGD method by
combining the variance reduction technique with Nesterov's extrapolation
technique. Moreover, based on the local error bound condition, we establish the
linear convergence of our method to obtain a stationary point of the nonconvex
optimization. In particular, we prove that not only the sequence generated
linearly converges to a stationary point of the problem, but also the
corresponding sequence of objective values is linearly convergent. Finally,
some numerical experiments demonstrate the effectiveness of our method. To the
best of our knowledge, it is first proved that the accelerated SGD method
converges linearly to the local minimum of the nonconvex optimization.
Structured Production System (extended abstract)
Authors: Yi Zhou
Date: 2017-04-26 00:00:00

In this extended abstract, we propose Structured Production Systems (SPS),
which extend traditional production systems with well-formed syntactic
structures. Due to the richness of structures, structured production systems
significantly enhance the expressive power as well as the flexibility of
production systems, for instance, to handle uncertainty. We show that different
rule application strategies can be reduced into the basic one by utilizing
structures. Also, many fundamental approaches in computer science, including
automata, grammar and logic, can be captured by structured production systems.
Multidimensional Binary Search for Contextual Decision-Making
Authors: Ilan Lobel, Renato Paes Leme, Adrian Vladu
Date: 2016-11-02 00:00:00
Tasks: Decision Making

We consider a multidimensional search problem that is motivated by questions
in contextual decision-making, such as dynamic pricing and personalized
medicine. Nature selects a state from a $d$-dimensional unit ball and then
generates a sequence of $d$-dimensional directions. We are given access to the
directions, but not access to the state. After receiving a direction, we have
to guess the value of the dot product between the state and the direction. Our
goal is to minimize the number of times when our guess is more than $\epsilon$
away from the true answer. We construct a polynomial time algorithm that we
call Projected Volume achieving regret $O(d\log(d/\epsilon))$, which is optimal
up to a $\log d$ factor. The algorithm combines a volume cutting strategy with
a new geometric technique that we call cylindrification.
Supervised Adversarial Networks for Image Saliency Detection
Authors: Hengyue Pan, Hui Jiang
Date: 2017-04-24 00:00:00
Tasks: Generative Adversarial Network, Image Generation, Saliency Detection

In the past few years, Generative Adversarial Network (GAN) became a
prevalent research topic. By defining two convolutional neural networks
(G-Network and D-Network) and introducing an adversarial procedure between them
during the training process, GAN has ability to generate good quality images
that look like natural images from a random vector. Besides image generation,
GAN may have potential to deal with wide range of real world problems. In this
paper, we follow the basic idea of GAN and propose a novel model for image
saliency detection, which is called Supervised Adversarial Networks (SAN).
Specifically, SAN also trains two models simultaneously: the G-Network takes
natural images as inputs and generates corresponding saliency maps (synthetic
saliency maps), and the D-Network is trained to determine whether one sample is
a synthetic saliency map or ground-truth saliency map. However, different from
GAN, the proposed method uses fully supervised learning to learn both G-Network
and D-Network by applying class labels of the training set. Moreover, a novel
kind of layer call conv-comparison layer is introduced into the D-Network to
further improve the saliency performance by forcing the high-level feature of
synthetic saliency maps and ground-truthes as similar as possible. Experimental
results on Pascal VOC 2012 database show that the SAN model can generate high
quality saliency maps for many complicate natural images.
Deterministic and Probabilistic Conditions for Finite Completability of Low-rank Multi-View Data
Authors: Morteza Ashraphijuo, Xiaodong Wang, Vaneet Aggarwal
Date: 2017-01-03 00:00:00
Tasks: Matrix Completion

We consider the multi-view data completion problem, i.e., to complete a
matrix $\mathbf{U}=[\mathbf{U}_1|\mathbf{U}_2]$ where the ranks of
$\mathbf{U},\mathbf{U}_1$, and $\mathbf{U}_2$ are given. In particular, we
investigate the fundamental conditions on the sampling pattern, i.e., locations
of the sampled entries for finite completability of such a multi-view data
given the corresponding rank constraints. In contrast with the existing
analysis on Grassmannian manifold for a single-view matrix, i.e., conventional
matrix completion, we propose a geometric analysis on the manifold structure
for multi-view data to incorporate more than one rank constraint. We provide a
deterministic necessary and sufficient condition on the sampling pattern for
finite completability. We also give a probabilistic condition in terms of the
number of samples per column that guarantees finite completability with high
probability. Finally, using the developed tools, we derive the deterministic
and probabilistic guarantees for unique completability.
An ensemble-based online learning algorithm for streaming data
Authors: Tien Thanh Nguyen, Thi Thu Thuy Nguyen, Xuan Cuong Pham, Alan Wee-Chung Liew, James C. Bezdek
Date: 2017-04-26 00:00:00
Tasks: BIG-bench Machine Learning

In this study, we introduce an ensemble-based approach for online machine
learning. The ensemble of base classifiers in our approach is obtained by
learning Naive Bayes classifiers on different training sets which are generated
by projecting the original training set to lower dimensional space. We propose
a mechanism to learn sequences of data using data chunks paradigm. The
experiments conducted on a number of UCI datasets and one synthetic dataset
demonstrate that the proposed approach performs significantly better than some
well-known online learning algorithms.
Attend to You: Personalized Image Captioning with Context Sequence Memory Networks
Authors: Cesc Chunseong Park, Byeongchang Kim, Gunhee Kim
Date: 2017-04-21 00:00:00
Proceeding: CVPR 2017 7
Tasks: Descriptive, Image Captioning, Sentence

We address personalization issues of image captioning, which have not been
discussed yet in previous research. For a query image, we aim to generate a
descriptive sentence, accounting for prior knowledge such as the user's active
vocabularies in previous documents. As applications of personalized image
captioning, we tackle two post automation tasks: hashtag prediction and post
generation, on our newly collected Instagram dataset, consisting of 1.1M posts
from 6.3K users. We propose a novel captioning model named Context Sequence
Memory Network (CSMN). Its unique updates over previous memory network models
include (i) exploiting memory as a repository for multiple types of context
information, (ii) appending previously generated words into memory to capture
long-term information without suffering from the vanishing gradient problem,
and (iii) adopting CNN memory structure to jointly represent nearby ordered
memory slots for better context understanding. With quantitative evaluation and
user studies via Amazon Mechanical Turk, we show the effectiveness of the three
novel features of CSMN and its performance enhancement for personalized image
captioning over state-of-the-art captioning models.