pwc-archive/papers-with-abstracts huggingface dataset:
Data-driven model for the identification of the rock type at a drilling bit
Authors: Nikita Klyuchnikov, Alexey Zaytsev, Arseniy Gruzdev, Georgiy Ovchinnikov, Ksenia Antipova, Leyla Ismailova, Ekaterina Muravleva, Evgeny Burnaev, Artyom Semenikhin, Alexey Cherepanov, Vitaliy Koryabkin, Igor Simon, Alexey Tsurgan, Fedor Krasnov, Dmitry Koroteev
Date: 2018-06-08 00:00:00
Tasks: BIG-bench Machine Learning, General Classification

Directional oil well drilling requires high precision of the wellbore
positioning inside the productive area. However, due to specifics of
engineering design, sensors that explicitly determine the type of the drilled
rock are located farther than 15m from the drilling bit. As a result, the
target area runaways can be detected only after this distance, which in turn,
leads to a loss in well productivity and the risk of the need for an expensive
re-boring operation.
  We present a novel approach for identifying rock type at the drilling bit
based on machine learning classification methods and data mining on sensors
readings. We compare various machine-learning algorithms, examine extra
features coming from mathematical modeling of drilling mechanics, and show that
the real-time rock type classification error can be reduced from 13.5 % to 9 %.
The approach is applicable for precise directional drilling in relatively thin
target intervals of complex shapes and generalizes appropriately to new wells
that are different from the ones used for training the machine learning model.
Generating Liquid Simulations with Deformation-aware Neural Networks
Authors: Lukas Prantl, Boris Bonev, Nils Thuerey
Date: 2017-04-25 00:00:00
Proceeding: ICLR 2019 5

We propose a novel approach for deformation-aware neural networks that learn
the weighting and synthesis of dense volumetric deformation fields. Our method
specifically targets the space-time representation of physical surfaces from
liquid simulations. Liquids exhibit highly complex, non-linear behavior under
changing simulation conditions such as different initial conditions. Our
algorithm captures these complex phenomena in two stages: a first neural
network computes a weighting function for a set of pre-computed deformations,
while a second network directly generates a deformation field for refining the
surface. Key for successful training runs in this setting is a suitable loss
function that encodes the effect of the deformations, and a robust calculation
of the corresponding gradients. To demonstrate the effectiveness of our
approach, we showcase our method with several complex examples of flowing
liquids with topology changes. Our representation makes it possible to rapidly
generate the desired implicit surfaces. We have implemented a mobile
application to demonstrate that real-time interactions with complex liquid
effects are possible with our approach.
Learning in Integer Latent Variable Models with Nested Automatic Differentiation
Authors: Daniel Sheldon, Kevin Winner, Debora Sujono
Date: 2018-06-08 00:00:00
Proceeding: ICML 2018 7

We develop nested automatic differentiation (AD) algorithms for exact
inference and learning in integer latent variable models. Recently, Winner,
Sujono, and Sheldon showed how to reduce marginalization in a class of integer
latent variable models to evaluating a probability generating function which
contains many levels of nested high-order derivatives. We contribute faster and
more stable AD algorithms for this challenging problem and a novel algorithm to
compute exact gradients for learning. These contributions lead to significantly
faster and more accurate learning algorithms, and are the first AD algorithms
whose running time is polynomial in the number of levels of nesting.
Spreading vectors for similarity search
Authors: Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Hervé Jégou
Date: 2018-06-08 00:00:00
Proceeding: ICLR 2019 5
Tasks: Quantization, Triplet

Discretizing multi-dimensional data distributions is a fundamental step of modern indexing methods. State-of-the-art techniques learn parameters of quantizers on training data for optimal performance, thus adapting quantizers to the data. In this work, we propose to reverse this paradigm and adapt the data to the quantizer: we train a neural net which last layer forms a fixed parameter-free quantizer, such as pre-defined points of a hyper-sphere. As a proxy objective, we design and train a neural network that favors uniformity in the spherical latent space, while preserving the neighborhood structure after the mapping. We propose a new regularizer derived from the Kozachenko--Leonenko differential entropy estimator to enforce uniformity and combine it with a locality-aware triplet loss. Experiments show that our end-to-end approach outperforms most learned quantization methods, and is competitive with the state of the art on widely adopted benchmarks. Furthermore, we show that training without the quantization step results in almost no difference in accuracy, but yields a generic catalyzer that can be applied with any subsequent quantizer.
Hearst Patterns Revisited: Automatic Hypernym Detection from Large Text Corpora
Authors: Stephen Roller, Douwe Kiela, Maximilian Nickel
Date: 2018-06-08 00:00:00
Proceeding: ACL 2018 7

Methods for unsupervised hypernym detection may broadly be categorized
according to two paradigms: pattern-based and distributional methods. In this
paper, we study the performance of both approaches on several hypernymy tasks
and find that simple pattern-based methods consistently outperform
distributional methods on common benchmark datasets. Our results show that
pattern-based models provide important contextual constraints which are not yet
captured in distributional methods.
The Well Tempered Lasso
Authors: Yuanzhi Li, Yoram Singer
Date: 2018-06-08 00:00:00
Tasks: regression

We study the complexity of the entire regularization path for least squares
regression with 1-norm penalty, known as the Lasso. Every regression parameter
in the Lasso changes linearly as a function of the regularization value. The
number of changes is regarded as the Lasso's complexity. Experimental results
using exact path following exhibit polynomial complexity of the Lasso in the
problem size. Alas, the path complexity of the Lasso on artificially designed
regression problems is exponential.
  We use smoothed analysis as a mechanism for bridging the gap between worst
case settings and the de facto low complexity. Our analysis assumes that the
observed data has a tiny amount of intrinsic noise. We then prove that the
Lasso's complexity is polynomial in the problem size. While building upon the
seminal work of Spielman and Teng on smoothed complexity, our analysis is
morally different as it is divorced from specific path following algorithms. We
verify the validity of our analysis in experiments with both worst case
settings and real datasets. The empirical results we obtain closely match our
analysis.
Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation
Authors: Daniel Stoller, Sebastian Ewert, Simon Dixon
Date: 2018-06-08 00:00:00
Tasks: Audio Source Separation, Music Source Separation

Models for audio source separation usually operate on the magnitude spectrum,
which ignores phase information and makes separation performance dependant on
hyper-parameters for the spectral front-end. Therefore, we investigate
end-to-end source separation in the time-domain, which allows modelling phase
information and avoids fixed spectral transformations. Due to high sampling
rates for audio, employing a long temporal input context on the sample level is
difficult, but required for high quality separation results because of
long-range temporal correlations. In this context, we propose the Wave-U-Net,
an adaptation of the U-Net to the one-dimensional time domain, which repeatedly
resamples feature maps to compute and combine features at different time
scales. We introduce further architectural improvements, including an output
layer that enforces source additivity, an upsampling technique and a
context-aware prediction framework to reduce output artifacts. Experiments for
singing voice separation indicate that our architecture yields a performance
comparable to a state-of-the-art spectrogram-based U-Net architecture, given
the same data. Finally, we reveal a problem with outliers in the currently used
SDR evaluation metrics and suggest reporting rank-based statistics to alleviate
this problem.
Averaging Stochastic Gradient Descent on Riemannian Manifolds
Authors: Nilesh Tripuraneni, Nicolas Flammarion, Francis Bach, Michael. I. Jordan
Date: 2018-02-26 00:00:00
Tasks: Riemannian optimization

We consider the minimization of a function defined on a Riemannian manifold
$\mathcal{M}$ accessible only through unbiased estimates of its gradients. We
develop a geometric framework to transform a sequence of slowly converging
iterates generated from stochastic gradient descent (SGD) on $\mathcal{M}$ to
an averaged iterate sequence with a robust and fast $O(1/n)$ convergence rate.
We then present an application of our framework to geodesically-strongly-convex
(and possibly Euclidean non-convex) problems. Finally, we demonstrate how these
ideas apply to the case of streaming $k$-PCA, where we show how to accelerate
the slow rate of the randomized power method (without requiring knowledge of
the eigengap) into a robust algorithm achieving the optimal rate of
convergence.
Multilingual Sentiment Analysis: An RNN-Based Framework for Limited Data
Authors: Ethem F. Can, Aysu Ezen-Can, Fazli Can
Date: 2018-06-08 00:00:00
Tasks: Sentiment Analysis, Word Embeddings

Sentiment analysis is a widely studied NLP task where the goal is to
determine opinions, emotions, and evaluations of users towards a product, an
entity or a service that they are reviewing. One of the biggest challenges for
sentiment analysis is that it is highly language dependent. Word embeddings,
sentiment lexicons, and even annotated data are language specific. Further,
optimizing models for each language is very time consuming and labor intensive
especially for recurrent neural network models. From a resource perspective, it
is very challenging to collect data for different languages.
  In this paper, we look for an answer to the following research question: can
a sentiment analysis model trained on a language be reused for sentiment
analysis in other languages, Russian, Spanish, Turkish, and Dutch, where the
data is more limited? Our goal is to build a single model in the language with
the largest dataset available for the task, and reuse it for languages that
have limited resources. For this purpose, we train a sentiment analysis model
using recurrent neural networks with reviews in English. We then translate
reviews in other languages and reuse this model to evaluate the sentiments.
Experimental results show that our robust approach of single model trained on
English reviews statistically significantly outperforms the baselines in
several different languages.
Evaluating CBR Similarity Functions for BAM Switching in Networks with Dynamic Traffic Profile
Authors: Eliseu Oliveira, Rafael Freitas, Joberto Martins
Date: 2018-06-08 00:00:00
Tasks: Management

In an increasingly complex scenario for network management, a solution that
allows configuration in more autonomous way with less intervention of the
network manager is expected. This paper presents an evaluation of similarity
functions that are necessary in the context of using a learning strategy for
finding solutions. The learning approach considered is based on Case-Based
Reasoning (CBR) and is applied to a network scenario where different Bandwidth
Allocation Models (BAMs) behaviors are used and must be eventually switched
looking for the best possible network operation. In this context, it is
required to identify and configure an adequate similarity function that will be
used in the learning process to recover similar solutions previously
considered. This paper introduces the similarity functions, explains the
relevant aspects of the learning process in which the similarity function plays
a role and, finally, presents a proof of concept for a specific similarity
function adopted. Results show that the similarity function was capable to get
similar results from the existing use case database. As such, the use of
similarity functions with CBR technique has proved to be potentially
satisfactory for supporting BAM switching decisions mostly driven by the
dynamics of input traffic profile.
Reinforcing Adversarial Robustness using Model Confidence Induced by Adversarial Training
Authors: Xi Wu, Uyeong Jang, Jiefeng Chen, Lingjiao Chen, Somesh Jha
Date: 2017-11-21 00:00:00
Proceeding: ICML 2018 7
Tasks: Adversarial Robustness

In this paper we study leveraging confidence information induced by
adversarial training to reinforce adversarial robustness of a given
adversarially trained model. A natural measure of confidence is $\|F({\bf
x})\|_\infty$ (i.e. how confident $F$ is about its prediction?). We start by
analyzing an adversarial training formulation proposed by Madry et al.. We
demonstrate that, under a variety of instantiations, an only somewhat good
solution to their objective induces confidence to be a discriminator, which can
distinguish between right and wrong model predictions in a neighborhood of a
point sampled from the underlying distribution. Based on this, we propose
Highly Confident Near Neighbor (${\tt HCNN}$), a framework that combines
confidence information and nearest neighbor search, to reinforce adversarial
robustness of a base model. We give algorithms in this framework and perform a
detailed empirical study. We report encouraging experimental results that
support our analysis, and also discuss problems we observed with existing
adversarial training.
Simple Bayesian Algorithms for Best Arm Identification
Authors: Daniel Russo
Date: 2016-02-26 00:00:00
Tasks: Thompson Sampling

This paper considers the optimal adaptive allocation of measurement effort
for identifying the best among a finite set of options or designs. An
experimenter sequentially chooses designs to measure and observes noisy signals
of their quality with the goal of confidently identifying the best design after
a small number of measurements. This paper proposes three simple and intuitive
Bayesian algorithms for adaptively allocating measurement effort, and
formalizes a sense in which these seemingly naive rules are the best possible.
One proposal is top-two probability sampling, which computes the two designs
with the highest posterior probability of being optimal, and then randomizes to
select among these two. One is a variant of top-two sampling which considers
not only the probability a design is optimal, but the expected amount by which
its quality exceeds that of other designs. The final algorithm is a modified
version of Thompson sampling that is tailored for identifying the best design.
We prove that these simple algorithms satisfy a sharp optimality property. In a
frequentist setting where the true quality of the designs is fixed, one hopes
the posterior definitively identifies the optimal design, in the sense that
that the posterior probability assigned to the event that some other design is
optimal converges to zero as measurements are collected. We show that under the
proposed algorithms this convergence occurs at an exponential rate, and the
corresponding exponent is the best possible among all allocation
Neural Message Passing with Edge Updates for Predicting Properties of Molecules and Materials
Authors: Peter Bjørn Jørgensen, Karsten Wedel Jacobsen, Mikkel N. Schmidt
Date: 2018-06-08 00:00:00
Tasks: Drug Discovery, Formation Energy

Neural message passing on molecular graphs is one of the most promising
methods for predicting formation energy and other properties of molecules and
materials. In this work we extend the neural message passing model with an edge
update network which allows the information exchanged between atoms to depend
on the hidden state of the receiving atom. We benchmark the proposed model on
three publicly available datasets (QM9, The Materials Project and OQMD) and
show that the proposed model yields superior prediction of formation energies
and other properties on all three datasets in comparison with the best
published results. Furthermore we investigate different methods for
constructing the graph used to represent crystalline structures and we find
that using a graph based on K-nearest neighbors achieves better prediction
accuracy than using maximum distance cutoff or the Voronoi tessellation graph.
Structured Output Learning with Abstention: Application to Accurate Opinion Prediction
Authors: Alexandre Garcia, Slim Essid, Chloé Clavel, Florence d'Alché-Buc
Date: 2018-03-22 00:00:00
Proceeding: ICML 2018 7
Tasks: Opinion Mining, Prediction, Sentence

Motivated by Supervised Opinion Analysis, we propose a novel framework
devoted to Structured Output Learning with Abstention (SOLA). The structure
prediction model is able to abstain from predicting some labels in the
structured output at a cost chosen by the user in a flexible way. For that
purpose, we decompose the problem into the learning of a pair of predictors,
one devoted to structured abstention and the other, to structured output
prediction. To compare fully labeled training data with predictions potentially
containing abstentions, we define a wide class of asymmetric abstention-aware
losses. Learning is achieved by surrogate regression in an appropriate feature
space while prediction with abstention is performed by solving a new pre-image
problem. Thus, SOLA extends recent ideas about Structured Output Prediction via
surrogate problems and calibration theory and enjoys statistical guarantees on
the resulting excess risk. Instantiated on a hierarchical abstention-aware
loss, SOLA is shown to be relevant for fine-grained opinion mining and gives
state-of-the-art results on this task. Moreover, the abstention-aware
representations can be used to competitively predict user-review ratings based
on a sentence-level opinion predictor.
Fidelity-based Probabilistic Q-learning for Control of Quantum Systems
Authors: Chunlin Chen, Daoyi Dong, Han-Xiong Li, Jian Chu, Tzyh-Jong Tarn
Date: 2018-06-08 00:00:00
Tasks: Q-Learning, Reinforcement Learning

The balance between exploration and exploitation is a key problem for
reinforcement learning methods, especially for Q-learning. In this paper, a
fidelity-based probabilistic Q-learning (FPQL) approach is presented to
naturally solve this problem and applied for learning control of quantum
systems. In this approach, fidelity is adopted to help direct the learning
process and the probability of each action to be selected at a certain state is
updated iteratively along with the learning process, which leads to a natural
exploration strategy instead of a pointed one with configured parameters. A
probabilistic Q-learning (PQL) algorithm is first presented to demonstrate the
basic idea of probabilistic action selection. Then the FPQL algorithm is
presented for learning control of quantum systems. Two examples (a spin- 1/2
system and a lamda-type atomic system) are demonstrated to test the performance
of the FPQL algorithm. The results show that FPQL algorithms attain a better
balance between exploration and exploitation, and can also avoid local optimal
policies and accelerate the learning process.
Black Box FDR
Authors: Wesley Tansey, Yixin Wang, David M. Blei, Raul Rabadan
Date: 2018-06-08 00:00:00
Proceeding: ICML 2018 7

Analyzing large-scale, multi-experiment studies requires scientists to test
each experimental outcome for statistical significance and then assess the
results as a whole. We present Black Box FDR (BB-FDR), an empirical-Bayes
method for analyzing multi-experiment studies when many covariates are gathered
per experiment. BB-FDR learns a series of black box predictive models to boost
power and control the false discovery rate (FDR) at two stages of study
analysis. In Stage 1, it uses a deep neural network prior to report which
experiments yielded significant outcomes. In Stage 2, a separate black box
model of each covariate is used to select features that have significant
predictive power across all experiments. In benchmarks, BB-FDR outperforms
competing state-of-the-art methods in both stages of analysis. We apply BB-FDR
to two real studies on cancer drug efficacy. For both studies, BB-FDR increases
the proportion of significant outcomes discovered and selects variables that
reveal key genomic drivers of drug sensitivity and resistance in cancer.
Hierarchy of GANs for learning embodied self-awareness model
Authors: Mahdyar Ravanbakhsh, Mohamad Baydoun, Damian Campo, Pablo Marin, David Martin, Lucio Marcenaro, Carlo S. Regazzoni
Date: 2018-06-08 00:00:00

In recent years several architectures have been proposed to learn embodied
agents complex self-awareness models. In this paper, dynamic incremental
self-awareness (SA) models are proposed that allow experiences done by an agent
to be modeled in a hierarchical fashion, starting from more simple situations
to more structured ones. Each situation is learned from subsets of private
agent perception data as a model capable to predict normal behaviors and detect
abnormalities. Hierarchical SA models have been already proposed using low
dimensional sensorial inputs. In this work, a hierarchical model is introduced
by means of a cross-modal Generative Adversarial Networks (GANs) processing
high dimensional visual data. Different levels of the GANs are detected in a
self-supervised manner using GANs discriminators decision boundaries. Real
experiments on semi-autonomous ground vehicles are presented.
Composite Functional Gradient Learning of Generative Adversarial Models
Authors: Rie Johnson, Tong Zhang
Date: 2018-01-19 00:00:00
Proceeding: ICML 2018 7
Tasks: Image Generation

This paper first presents a theory for generative adversarial methods that
does not rely on the traditional minimax formulation. It shows that with a
strong discriminator, a good generator can be learned so that the KL divergence
between the distributions of real data and generated data improves after each
functional gradient step until it converges to zero. Based on the theory, we
propose a new stable generative adversarial method. A theoretical insight into
the original GAN from this new viewpoint is also provided. The experiments on
image generation show the effectiveness of our new method.
Geodesic convolutional neural networks on Riemannian manifolds
Authors: Jonathan Masci, Davide Boscaini, Michael M. Bronstein, Pierre Vandergheynst
Date: 2015-01-26 00:00:00
Tasks: Retrieval

Feature descriptors play a crucial role in a wide range of geometry analysis
and processing applications, including shape correspondence, retrieval, and
segmentation. In this paper, we introduce Geodesic Convolutional Neural
Networks (GCNN), a generalization of the convolutional networks (CNN) paradigm
to non-Euclidean manifolds. Our construction is based on a local geodesic
system of polar coordinates to extract "patches", which are then passed through
a cascade of filters and linear and non-linear operators. The coefficients of
the filters and linear combination weights are optimization variables that are
learned to minimize a task-specific cost function. We use GCNN to learn
invariant shape features, allowing to achieve state-of-the-art performance in
problems such as shape description, retrieval, and correspondence.