pwc-archive/papers-with-abstracts huggingface dataset:
Optimal Minimal Margin Maximization with Boosting
Authors: Allan Grønlund, Kasper Green Larsen, Alexander Mathiasen
Date: 2019-01-30 00:00:00

Boosting algorithms produce a classifier by iteratively combining base
hypotheses. It has been observed experimentally that the generalization error
keeps improving even after achieving zero training error. One popular
explanation attributes this to improvements in margins. A common goal in a long
line of research, is to maximize the smallest margin using as few base
hypotheses as possible, culminating with the AdaBoostV algorithm by (R{\"a}tsch
and Warmuth [JMLR'04]). The AdaBoostV algorithm was later conjectured to yield
an optimal trade-off between number of hypotheses trained and the minimal
margin over all training points (Nie et al. [JMLR'13]). Our main contribution
is a new algorithm refuting this conjecture. Furthermore, we prove a lower
bound which implies that our new algorithm is optimal.
Blurred Images Lead to Bad Local Minima
Authors: Gal Katzhendler, Daphna Weinshall
Date: 2019-01-30 00:00:00

Blurred Images Lead to Bad Local Minima
Tensorized Embedding Layers for Efficient Model Compression
Authors: Oleksii Hrinchuk, Valentin Khrulkov, Leyla Mirvakhabova, Elena Orlova, Ivan Oseledets
Date: 2019-01-30 00:00:00
Tasks: Language Modelling, Machine Translation, model, Model Compression, Sentiment Analysis

The embedding layers transforming input words into real vectors are the key components of deep neural networks used in natural language processing. However, when the vocabulary is large, the corresponding weight matrices can be enormous, which precludes their deployment in a limited resource setting. We introduce a novel way of parametrizing embedding layers based on the Tensor Train (TT) decomposition, which allows compressing the model significantly at the cost of a negligible drop or even a slight gain in performance. We evaluate our method on a wide range of benchmarks in natural language processing and analyze the trade-off between performance and compression ratios for a wide range of architectures, from MLPs to LSTMs and Transformers.
Human-centric light sensing and estimation from RGBD images: The invisible light switch
Authors: Theodore Tsesmelis, Irtiza Hasan, Marco Cristani, Alessio Del Bue, Fabio Galasso
Date: 2019-01-30 00:00:00

Lighting design in indoor environments is of primary importance for at least
two reasons: 1) people should perceive an adequate light; 2) an effective
lighting design means consistent energy saving. We present the Invisible Light
Switch (ILS) to address both aspects. ILS dynamically adjusts the room
illumination level to save energy while maintaining constant the light level
perception of the users. So the energy saving is invisible to them. Our
proposed ILS leverages a radiosity model to estimate the light level which is
perceived by a person within an indoor environment, taking into account the
person position and her/his viewing frustum (head pose). ILS may therefore dim
those luminaires, which are not seen by the user, resulting in an effective
energy saving, especially in large open offices (where light may otherwise be
ON everywhere for a single person). To quantify the system performance, we have
collected a new dataset where people wear luxmeter devices while working in
office rooms. The luxmeters measure the amount of light (in Lux) reaching the
people gaze, which we consider a proxy to their illumination level perception.
Our initial results are promising: in a room with 8 LED luminaires, the energy
consumption in a day may be reduced from 18585 to 6206 watts with ILS
(currently needing 1560 watts for operations). While doing so, the drop in
perceived lighting decreases by just 200 lux, a value considered negligible
when the original illumination level is above 1200 lux, as is normally the case
in offices.
Clustering with Jointly Learned Nonlinear Transforms Over Discriminating Min-Max Similarity/Dissimilarity Assignment
Authors: Dimche Kostadinov, Behrooz Razeghi, Taras Holotyak, Slava Voloshynovskiy
Date: 2019-01-30 00:00:00
Tasks: Clustering, Image Clustering

This paper presents a novel clustering concept that is based on jointly
learned nonlinear transforms (NTs) with priors on the information loss and the
discrimination. We introduce a clustering principle that is based on evaluation
of a parametric min-max measure for the discriminative prior. The decomposition
of the prior measure allows to break down the assignment into two steps. In the
first step, we apply NTs to a data point in order to produce candidate NT
representations. In the second step, we preform the actual assignment by
evaluating the parametric measure over the candidate NT representations.
Numerical experiments on image clustering task validate the potential of the
proposed approach. The evaluation shows advantages in comparison to the
state-of-the-art clustering methods.
Ensemble-based kernel learning for a class of data assimilation problems with imperfect forward simulators
Authors: Xiaodong Luo
Date: 2019-01-30 00:00:00
Tasks: BIG-bench Machine Learning

Simulator imperfection, often known as model error, is ubiquitous in
practical data assimilation problems. Despite the enormous efforts dedicated to
addressing this problem, properly handling simulator imperfection in data
assimilation remains to be a challenging task. In this work, we propose an
approach to dealing with simulator imperfection from a point of view of
functional approximation that can be implemented through a certain machine
learning method, such as kernel-based learning adopted in the current work. To
this end, we start from considering a class of supervised learning problems,
and then identify similarities between supervised learning and variational data
assimilation. These similarities found the basis for us to develop an
ensemble-based learning framework to tackle supervised learning problems, while
achieving various advantages of ensemble-based methods over the variational
ones. After establishing the ensemble-based learning framework, we proceed to
investigate the integration of ensemble-based learning into an ensemble-based
data assimilation framework to handle simulator imperfection. In the course of
our investigations, we also develop a strategy to tackle the issue of
multi-modality in supervised-learning problems, and transfer this strategy to
data assimilation problems to help improve assimilation performance. For
demonstration, we apply the ensemble-based learning framework and the
integrated, ensemble-based data assimilation framework to a supervised learning
problem and a data assimilation problem with an imperfect forward simulator,
respectively. The experiment results indicate that both frameworks achieve good
performance in relevant case studies, and that functional approximation through
machine learning may serve as a viable way to account for simulator
imperfection in data assimilation problems.
Distributionally Robust and Multi-Objective Nonnegative Matrix Factorization
Authors: Nicolas Gillis, Le Thi Khanh Hien, Valentin Leplat, Vincent Y. F. Tan
Date: 2019-01-30 00:00:00
Tasks: Dimensionality Reduction

Nonnegative matrix factorization (NMF) is a linear dimensionality reduction technique for analyzing nonnegative data. A key aspect of NMF is the choice of the objective function that depends on the noise model (or statistics of the noise) assumed on the data. In many applications, the noise model is unknown and difficult to estimate. In this paper, we define a multi-objective NMF (MO-NMF) problem, where several objectives are combined within the same NMF model. We propose to use Lagrange duality to judiciously optimize for a set of weights to be used within the framework of the weighted-sum approach, that is, we minimize a single objective function which is a weighted sum of the all objective functions. We design a simple algorithm based on multiplicative updates to minimize this weighted sum. We show how this can be used to find distributionally robust NMF (DR-NMF) solutions, that is, solutions that minimize the largest error among all objectives, using a dual approach solved via a heuristic inspired from the Frank-Wolfe algorithm. We illustrate the effectiveness of this approach on synthetic, document and audio data sets. The results show that DR-NMF is robust to our incognizance of the noise model of the NMF problem.
Autonomous Cars: Vision based Steering Wheel Angle Estimation
Authors: Kemal Alkin Gunbay, Mert Arikan, Mehmet Turkan
Date: 2019-01-30 00:00:00
Tasks: Autonomous Vehicles, Self-Driving Cars

Machine learning models, which are frequently used in self-driving cars, are
trained by matching the captured images of the road and the measured angle of
the steering wheel. The angle of the steering wheel is generally fetched from
steering angle sensor, which is tightly-coupled to the physical aspects of the
vehicle at hand. Therefore, a model-agnostic autonomous car-kit is very
difficult to be developed and autonomous vehicles need more training data. The
proposed vision based steering angle estimation system argues a new approach
which basically matches the images of the road captured by an outdoor camera
and the images of the steering wheel from an onboard camera, avoiding the
burden of collecting model-dependent training data and the use of any other
electromechanical hardware.
Reference-less Quality Estimation of Text Simplification Systems
Authors: Louis Martin, Samuel Humeau, Pierre-Emmanuel Mazaré, Antoine Bordes, Éric Villemonte de la Clergerie, Benoît Sagot
Date: 2019-01-30 00:00:00
Proceeding: WS 2018 11
Tasks: Machine Translation, Sentence, Text Simplification, Translation

The evaluation of text simplification (TS) systems remains an open challenge.
As the task has common points with machine translation (MT), TS is often
evaluated using MT metrics such as BLEU. However, such metrics require high
quality reference data, which is rarely available for TS. TS has the advantage
over MT of being a monolingual task, which allows for direct comparisons to be
made between the simplified text and its original version. In this paper, we
compare multiple approaches to reference-less quality estimation of
sentence-level text simplification systems, based on the dataset used for the
QATS 2016 shared task. We distinguish three different dimensions:
gram-maticality, meaning preservation and simplicity. We show that n-gram-based
MT metrics such as BLEU and METEOR correlate the most with human judgment of
grammaticality and meaning preservation, whereas simplicity is best evaluated
by basic length-based metrics.
A study for Image compression using Re-Pair algorithm
Authors: Pasquale De Luca, Vincenzo Maria Russiello, Raffaele Ciro Sannino, Lorenzo Valente
Date: 2019-01-30 00:00:00
Tasks: Image Compression, Text Compression

The compression is an important topic in computer science which allows we to
storage more amount of data on our data storage. There are several techniques
to compress any file. In this manuscript will be described the most important
algorithm to compress images such as JPEG and it will be compared with another
method to retrieve good reason to not use this method on images. So to compress
the text the most encoding technique known is the Huffman Encoding which it
will be explained in exhaustive way. In this manuscript will showed how to
compute a text compression method on images in particular the method and the
reason to choice a determinate image format against the other. The method
studied and analyzed in this manuscript is the Re-Pair algorithm which is
purely for grammatical context to be compress. At the and it will be showed the
good result of this application.
Unsupervised Scalable Representation Learning for Multivariate Time Series
Authors: Jean-Yves Franceschi, Aymeric Dieuleveut, Martin Jaggi
Date: 2019-01-30 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: BIG-bench Machine Learning, Representation Learning, Time Series, Time Series Analysis, Triplet

Time series constitute a challenging data type for machine learning algorithms, due to their highly variable lengths and sparse labeling in practice. In this paper, we tackle this challenge by proposing an unsupervised method to learn universal embeddings of time series. Unlike previous works, it is scalable with respect to their length and we demonstrate the quality, transferability and practicability of the learned representations with thorough experiments and comparisons. To this end, we combine an encoder based on causal dilated convolutions with a novel triplet loss employing time-based negative sampling, obtaining general-purpose representations for variable length and multivariate time series.
Compositionality for Recursive Neural Networks
Authors: Martha Lewis
Date: 2019-01-30 00:00:00

Modelling compositionality has been a longstanding area of research in the
field of vector space semantics. The categorical approach to compositionality
maps grammar onto vector spaces in a principled way, but comes under fire for
requiring the formation of very high-dimensional matrices and tensors, and
therefore being computationally infeasible. In this paper I show how a linear
simplification of recursive neural tensor network models can be mapped directly
onto the categorical approach, giving a way of computing the required matrices
and tensors. This mapping suggests a number of lines of research for both
categorical compositional vector space models of meaning and for recursive
neural network models of compositionality.
A Mobile Robot Generating Video Summaries of Seniors' Indoor Activities
Authors: Chih-Yuan Yang, Heeseung Yun, Srenavis Varadaraj, Jane Yung-jen Hsu
Date: 2019-01-30 00:00:00
Tasks: Action Recognition, Human Detection, Person Identification, Pose Estimation, Video Summarization

We develop a system which generates summaries from seniors' indoor-activity videos captured by a social robot to help remote family members know their seniors' daily activities at home. Unlike the traditional video summarization datasets, indoor videos captured from a moving robot poses additional challenges, namely, (i) the video sequences are very long (ii) a significant number of video-frames contain no-subject or with subjects at ill-posed locations and scales (iii) most of the well-posed frames contain highly redundant information. To address this problem, we propose to \hl{exploit} pose estimation \hl{for detecting} people in frames\hl{. This guides the robot} to follow the user and capture effective videos. We use person identification to distinguish a target senior from other people. We \hl{also make use of} action recognition to analyze seniors' major activities at different moments, and develop a video summarization method to select diverse and representative keyframes as summaries.
Learning Position Evaluation Functions Used in Monte Carlo Softmax Search
Authors: Harukazu Igarashi, Yuichi Morioka, Kazumasa Yamamoto
Date: 2019-01-30 00:00:00
Tasks: Position, Reinforcement Learning

This paper makes two proposals for Monte Carlo Softmax Search, which is a recently proposed method that is classified as a selective search like the Monte Carlo Tree Search. The first proposal separately defines the node-selection and backup policies to allow researchers to freely design a node-selection policy based on their searching strategies and confirms the principal variation produced by the Monte Carlo Softmax Search to that produced by a minimax search. The second proposal modifies commonly used learning methods for positional evaluation functions. In our new proposals, evaluation functions are learned by Monte Carlo sampling, which is performed with the backup policy in the search tree produced by Monte Carlo Softmax Search. The learning methods under consideration include supervised learning, reinforcement learning, regression learning, and search bootstrapping. Our sampling-based learning not only uses current positions and principal variations but also the internal nodes and important variations of a search tree. This step reduces the number of games necessary for learning. New learning rules are derived for sampling-based learning based on the Monte Carlo Softmax Search and combinations of the modified learning methods are also proposed in this paper.
Online Pandora's Boxes and Bandits
Authors: Hossein Esfandiari, Mohammadtaghi Hajiaghayi, Brendan Lucier, Michael Mitzenmacher
Date: 2019-01-30 00:00:00
Tasks: Decision Making, Reinforcement Learning

We consider online variations of the Pandora's box problem (Weitzman. 1979),
a standard model for understanding issues related to the cost of acquiring
information for decision-making. Our problem generalizes both the classic
Pandora's box problem and the prophet inequality framework. Boxes are presented
online, each with a random value and cost drew jointly from some known
distribution. Pandora chooses online whether to open each box given its cost,
and then chooses irrevocably whether to keep the revealed prize or pass on it.
We aim for approximation algorithms against adversaries that can choose the
largest prize over any opened box, and use optimal offline policies to decide
which boxes to open (without knowledge of the value inside). We consider
variations where Pandora can collect multiple prizes subject to feasibility
constraints, such as cardinality, matroid, or knapsack constraints. We also
consider variations related to classic multi-armed bandit problems from
reinforcement learning. Our results use a reduction-based framework where we
separate the issues of the cost of acquiring information from the online
decision process of which prizes to keep. Our work shows that in many
scenarios, Pandora can achieve a good approximation to the best possible
performance.
Probability Functional Descent: A Unifying Perspective on GANs, Variational Inference, and Reinforcement Learning
Authors: Casey Chu, Jose Blanchet, Peter Glynn
Date: 2019-01-30 00:00:00
Tasks: BIG-bench Machine Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Variational Inference

This paper provides a unifying view of a wide range of problems of interest in machine learning by framing them as the minimization of functionals defined on the space of probability measures. In particular, we show that generative adversarial networks, variational inference, and actor-critic methods in reinforcement learning can all be seen through the lens of our framework. We then discuss a generic optimization algorithm for our formulation, called probability functional descent (PFD), and show how this algorithm recovers existing methods developed independently in the settings mentioned earlier.
End-to-End Learned Early Classification of Time Series for In-Season Crop Type Mapping
Authors: Marc Rußwurm, Nicolas Courty, Rémi Emonet, Sébastien Lefèvre, Devis Tuia, Romain Tavenard
Date: 2019-01-30 00:00:00
Tasks: Classification, Crop Classification, Crop Type Mapping, Early  Classification, General Classification, Time Series, Time Series Analysis, Time Series Classification

Remote sensing satellites capture the cyclic dynamics of our Planet in regular time intervals recorded in satellite time series data. End-to-end trained deep learning models use this time series data to make predictions at a large scale, for instance, to produce up-to-date crop cover maps. Most time series classification approaches focus on the accuracy of predictions. However, the earliness of the prediction is also of great importance since coming to an early decision can make a crucial difference in time-sensitive applications. In this work, we present an End-to-End Learned Early Classification of Time Series (ELECTS) model that estimates a classification score and a probability of whether sufficient data has been observed to come to an early and still accurate decision. ELECTS is modular: any deep time series classification model can adopt the ELECTS conceptual idea by adding a second prediction head that outputs a probability of stopping the classification. The ELECTS loss function then optimizes the overall model on a balanced objective of earliness and accuracy. Our experiments on four crop classification datasets from Europe and Africa show that ELECTS allows reaching state-of-the-art accuracy while reducing the quantity of data massively to be downloaded, stored, and processed. The source code is available at https://github.com/marccoru/elects.
Effective weakly supervised semantic frame induction using expression sharing in hierarchical hidden Markov models
Authors: Janneke van de Loo, Jort F. Gemmeke, Guy De Pauw, Bart Ons, Walter Daelemans, Hugo Van hamme
Date: 2019-01-30 00:00:00
Tasks: Weakly Supervised Classification

We present a framework for the induction of semantic frames from utterances
in the context of an adaptive command-and-control interface. The system is
trained on an individual user's utterances and the corresponding semantic
frames representing controls. During training, no prior information on the
alignment between utterance segments and frame slots and values is available.
In addition, semantic frames in the training data can contain information that
is not expressed in the utterances. To tackle this weakly supervised
classification task, we propose a framework based on Hidden Markov Models
(HMMs). Structural modifications, resulting in a hierarchical HMM, and an
extension called expression sharing are introduced to minimize the amount of
training time and effort required for the user.
  The dataset used for the present study is PATCOR, which contains commands
uttered in the context of a vocally guided card game, Patience. Experiments
were carried out on orthographic and phonetic transcriptions of commands,
segmented on different levels of n-gram granularity. The experimental results
show positive effects of all the studied system extensions, with some effect
differences between the different input representations. Moreover, evaluation
experiments on held-out data with the optimal system configuration show that
the extended system is able to achieve high accuracies with relatively small
amounts of training data.
Doubly Sparse: Sparse Mixture of Sparse Experts for Efficient Softmax Inference
Authors: Shun Liao, Ting Chen, Tian Lin, Denny Zhou, Chong Wang
Date: 2019-01-30 00:00:00
Proceeding: ICLR 2019 5
Tasks: image-classification, Image Classification, Language Modeling, Language Modelling, Machine Translation, Translation

Computations for the softmax function are significantly expensive when the number of output classes is large. In this paper, we present a novel softmax inference speedup method, Doubly Sparse Softmax (DS-Softmax), that leverages sparse mixture of sparse experts to efficiently retrieve top-k classes. Different from most existing methods that require and approximate a fixed softmax, our method is learning-based and can adapt softmax weights for a better inference speedup. In particular, our method learns a two-level hierarchy which divides entire output class space into several partially overlapping experts. Each expert is sparse and only contains a subset of output classes. To find top-k classes, a sparse mixture enables us to find the most probable expert quickly, and the sparse expert enables us to search within a small-scale softmax. We empirically conduct evaluation on several real-world tasks, including neural machine translation, language modeling and image classification, and demonstrate that significant computation reductions can be achieved at no performance loss.