pwc-archive/papers-with-abstracts huggingface dataset:
Efficient Learning for Deep Quantum Neural Networks
Authors: Kerstin Beer, Dmytro Bondarenko, Terry Farrelly, Tobias J. Osborne, Robert Salzmann, Ramona Wolf
Date: 2019-02-27 00:00:00

Neural networks enjoy widespread success in both research and industry and,
with the imminent advent of quantum technology, it is now a crucial challenge
to design quantum neural networks for fully quantum learning tasks. Here we
propose the use of quantum neurons as a building block for quantum feed-forward
neural networks capable of universal quantum computation. We describe the
efficient training of these networks using the fidelity as a cost function and
provide both classical and efficient quantum implementations. Our method allows
for fast optimisation with reduced memory requirements: the number of qudits
required scales with only the width, allowing the optimisation of deep
networks. We benchmark our proposal for the quantum task of learning an unknown
unitary and find remarkable generalisation behaviour and a striking robustness
to noisy training data.
Fix Your Features: Stationary and Maximally Discriminative Embeddings using Regular Polytope (Fixed Classifier) Networks
Authors: Federico Pernici, Matteo Bruni, Claudio Baecchi, Alberto del Bimbo
Date: 2019-02-27 00:00:00
Tasks: General Classification

Neural networks are widely used as a model for classification in a large
variety of tasks. Typically, a learnable transformation (i.e. the classifier)
is placed at the end of such models returning a value for each class used for
classification. This transformation plays an important role in determining how
the generated features change during the learning process.
  In this work we argue that this transformation not only can be fixed (i.e.
set as non trainable) with no loss of accuracy, but it can also be used to
learn stationary and maximally discriminative embeddings.
  We show that the stationarity of the embedding and its maximal discriminative
representation can be theoretically justified by setting the weights of the
fixed classifier to values taken from the coordinate vertices of three regular
polytopes available in $\mathbb{R}^d$, namely: the $d$-Simplex, the $d$-Cube
and the $d$-Orthoplex. These regular polytopes have the maximal amount of
symmetry that can be exploited to generate stationary features angularly
centered around their corresponding fixed weights.
  Our approach improves and broadens the concept of a fixed classifier,
recently proposed in \cite{hoffer2018fix}, to a larger class of fixed
classifier models. Experimental results confirm both the theoretical analysis
and the generalization capability of the proposed method.
Adaptive Hedging under Delayed Feedback
Authors: Alexander Korotin, Vladimir V'yugin, Evgeny Burnaev
Date: 2019-02-27 00:00:00

The article is devoted to investigating the application of hedging strategies to online expert weight allocation under delayed feedback. As the main result, we develop the General Hedging algorithm $\mathcal{G}$ based on the exponential reweighing of experts' losses. We build the artificial probabilistic framework and use it to prove the adversarial loss bounds for the algorithm $\mathcal{G}$ in the delayed feedback setting. The designed algorithm $\mathcal{G}$ can be applied to both countable and continuous sets of experts. We also show how algorithm $\mathcal{G}$ extends classical Hedge (Multiplicative Weights) and adaptive Fixed Share algorithms to the delayed feedback and derive their regret bounds for the delayed setting by using our main result.
StyleRemix: An Interpretable Representation for Neural Image Style Transfer
Authors: Hongmin Xu, Qiang Li, Wenbo Zhang, Wen Zheng
Date: 2019-02-27 00:00:00
Tasks: Style Transfer

Multi-Style Transfer (MST) intents to capture the high-level visual
vocabulary of different styles and expresses these vocabularies in a joint
model to transfer each specific style. Recently, Style Embedding Learning (SEL)
based methods represent each style with an explicit set of parameters to
perform MST task. However, most existing SEL methods either learn explicit
style representation with numerous independent parameters or learn a relatively
black-box style representation, which makes them difficult to control the
stylized results. In this paper, we outline a novel MST model, StyleRemix, to
compactly and explicitly integrate multiple styles into one network. By
decomposing diverse styles into the same basis, StyleRemix represents a
specific style in a continuous vector space with 1-dimensional coefficients.
With the interpretable style representation, StyleRemix not only enables the
style visualization task but also allows several ways of remixing styles in the
smooth style embedding space.~Extensive experiments demonstrate the
effectiveness of StyleRemix on various MST tasks compared to state-of-the-art
SEL approaches.
Learning to Generate Questions by Learning What not to Generate
Authors: Bang Liu, Mingjun Zhao, Di Niu, Kunfeng Lai, Yancheng He, Haojie Wei, Yu Xu
Date: 2019-02-27 00:00:00
Tasks: Multi-Task Learning, Question Answering, Question Generation, Question-Generation

Automatic question generation is an important technique that can improve the
training of question answering, help chatbots to start or continue a
conversation with humans, and provide assessment materials for educational
purposes. Existing neural question generation models are not sufficient mainly
due to their inability to properly model the process of how each word in the
question is selected, i.e., whether repeating the given passage or being
generated from a vocabulary. In this paper, we propose our Clue Guided Copy
Network for Question Generation (CGC-QG), which is a sequence-to-sequence
generative model with copying mechanism, yet employing a variety of novel
components and techniques to boost the performance of question generation. In
CGC-QG, we design a multi-task labeling strategy to identify whether a question
word should be copied from the input passage or be generated instead, guiding
the model to learn the accurate boundaries between copying and generation.
Furthermore, our input passage encoder takes as input, among a diverse range of
other features, the prediction made by a clue word predictor, which helps
identify whether each word in the input passage is a potential clue to be
copied into the target question. The clue word predictor is designed based on a
novel application of Graph Convolutional Networks onto a syntactic dependency
tree representation of each passage, thus being able to predict clue words only
based on their context in the passage and their relative positions to the
answer in the tree. We jointly train the clue prediction as well as question
generation with multi-task learning and a number of practical strategies to
reduce the complexity. Extensive evaluations show that our model significantly
improves the performance of question generation and out-performs all previous
state-of-the-art neural question generation models by a substantial margin.
Equi-normalization of Neural Networks
Authors: Pierre Stock, Benjamin Graham, Rémi Gribonval, Hervé Jégou
Date: 2019-02-27 00:00:00
Proceeding: ICLR 2019 5

Modern neural networks are over-parametrized. In particular, each rectified
linear hidden unit can be modified by a multiplicative factor by adjusting
input and output weights, without changing the rest of the network. Inspired by
the Sinkhorn-Knopp algorithm, we introduce a fast iterative method for
minimizing the L2 norm of the weights, equivalently the weight decay
regularizer. It provably converges to a unique solution. Interleaving our
algorithm with SGD during training improves the test accuracy. For small
batches, our approach offers an alternative to batch-and group-normalization on
CIFAR-10 and ImageNet with a ResNet-18.
Computing Nonlinear Eigenfunctions via Gradient Flow Extinction
Authors: Leon Bungert, Martin Burger, Daniel Tenbrinck
Date: 2019-02-27 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, Graph Clustering, Spectral Graph Clustering

In this work we investigate the computation of nonlinear eigenfunctions via
the extinction profiles of gradient flows. We analyze a scheme that recursively
subtracts such eigenfunctions from given data and show that this procedure
yields a decomposition of the data into eigenfunctions in some cases as the
1-dimensional total variation, for instance. We discuss results of numerical
experiments in which we use extinction profiles and the gradient flow for the
task of spectral graph clustering as used, e.g., in machine learning
applications.
Representation Learning with Weighted Inner Product for Universal Approximation of General Similarities
Authors: Geewook Kim, Akifumi Okuno, Kazuki Fukui, Hidetoshi Shimodaira
Date: 2019-02-27 00:00:00
Tasks: Graph Embedding, Model Selection, Representation Learning

We propose $\textit{weighted inner product similarity}$ (WIPS) for neural network-based graph embedding. In addition to the parameters of neural networks, we optimize the weights of the inner product by allowing positive and negative values. Despite its simplicity, WIPS can approximate arbitrary general similarities including positive definite, conditionally positive definite, and indefinite kernels. WIPS is free from similarity model selection, since it can learn any similarity models such as cosine similarity, negative Poincar\'e distance and negative Wasserstein distance. Our experiments show that the proposed method can learn high-quality distributed representations of nodes from real datasets, leading to an accurate approximation of similarities as well as high performance in inductive tasks.
Provable Approximations for Constrained $\ell_p$ Regression
Authors: Ibrahim Jubran, David Cohn, Dan Feldman
Date: 2019-02-27 00:00:00
Tasks: regression

The $\ell_p$ linear regression problem is to minimize $f(x)=||Ax-b||_p$ over
$x\in\mathbb{R}^d$, where $A\in\mathbb{R}^{n\times d}$, $b\in \mathbb{R}^n$,
and $p>0$. To avoid overfitting and bound $||x||_2$, the constrained $\ell_p$
regression minimizes $f(x)$ over every unit vector $x\in\mathbb{R}^d$. This
makes the problem non-convex even for the simplest case $d=p=2$. Instead, ridge
regression is used to minimize the Lagrange form $f(x)+\lambda ||x||_2$ over
$x\in\mathbb{R}^d$, which yields a convex problem in the price of calibrating
the regularization parameter $\lambda>0$. We provide the first provable
constant factor approximation algorithm that solves the constrained $\ell_p$
regression directly, for every constant $p,d\geq 1$. Using core-sets, its
running time is $O(n \log n)$ including extensions for streaming and
distributed (big) data. In polynomial time, it can handle outliers, $p\in
(0,1)$ and minimize $f(x)$ over every $x$ and permutation of rows in $A$.
Experimental results are also provided, including open source and comparison to
existing software.
Hypernetwork functional image representation
Authors: Sylwester Klocek, Łukasz Maziarka, Maciej Wołczyk, Jacek Tabor, Jakub Nowak, Marek Śmieja
Date: 2019-02-27 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

Motivated by the human way of memorizing images we introduce their functional representation, where an image is represented by a neural network. For this purpose, we construct a hypernetwork which takes an image and returns weights to the target network, which maps point from the plane (representing positions of the pixel) into its corresponding color in the image. Since the obtained representation is continuous, one can easily inspect the image at various resolutions and perform on it arbitrary continuous operations. Moreover, by inspecting interpolations we show that such representation has some properties characteristic to generative models. To evaluate the proposed mechanism experimentally, we apply it to image super-resolution problem. Despite using a single model for various scaling factors, we obtained results comparable to existing super-resolution methods.
Unbiased LambdaMART: An Unbiased Pairwise Learning-to-Rank Algorithm
Authors: Ziniu Hu, Yang Wang, Qu Peng, Hang Li
Date: 2018-09-16 00:00:00
Tasks: Learning-To-Rank, Position

Although click data is widely used in search systems in practice, so far the
inherent bias, most notably position bias, has prevented it from being used in
training of a ranker for search, i.e., learning-to-rank. Recently, a number of
authors have proposed new techniques referred to as 'unbiased
learning-to-rank', which can reduce position bias and train a relatively
high-performance ranker using click data. Most of the algorithms, based on the
inverse propensity weighting (IPW) principle, first estimate the click bias at
each position, and then train an unbiased ranker with the estimated biases
using a learning-to-rank algorithm. However, there has not been a method for
pairwise learning-to-rank that can jointly conduct debiasing of click data and
training of a ranker using a pairwise loss function. In this paper, we propose
a novel algorithm, which can jointly estimate the biases at click positions and
the biases at unclick positions, and learn an unbiased ranker. Experiments on
benchmark data show that our algorithm can significantly outperform existing
algorithms. In addition, an online A/B Testing at a commercial search engine
shows that our algorithm can effectively conduct debiasing of click data and
enhance relevance ranking.
Social Credibility Incorporating Semantic Analysis and Machine Learning: A Survey of the State-of-the-Art and Future Research Directions
Authors: Bilal Abu-Salih, Bushra Bremie, Pornpit Wongthongtham, Kevin Duan, Tomayess Issa, Kit Yan Chan, Mohammad Alhabashneh, Teshreen Albtoush, Sulaiman Alqahtani, Abdullah Alqahtani, Muteeb Alahmari, Naser Alshareef, Abdulaziz Albahlal
Date: 2019-02-27 00:00:00

The wealth of Social Big Data (SBD) represents a unique opportunity for
organisations to obtain the excessive use of such data abundance to increase
their revenues. Hence, there is an imperative need to capture, load, store,
process, analyse, transform, interpret, and visualise such manifold social
datasets to develop meaningful insights that are specific to an application
domain. This paper lays the theoretical background by introducing the
state-of-the-art literature review of the research topic. This is associated
with a critical evaluation of the current approaches, and fortified with
certain recommendations indicated to bridge the research gap.
Reducing Artificial Neural Network Complexity: A Case Study on Exoplanet Detection
Authors: Sebastiaan Koning, Caspar Greeven, Eric Postma
Date: 2019-02-27 00:00:00
Tasks: General Classification, Self-Learning, Time Series, Time Series Analysis

Despite their successes in the field of self-learning AI, Convolutional
Neural Networks (CNNs) suffer from having too many trainable parameters,
impacting computational performance. Several approaches have been proposed to
reduce the number of parameters in the visual domain, the Inception
architecture [Szegedy et al., 2016] being a prominent example. This raises the
question whether the number of trainable parameters in CNNs can also be reduced
for 1D inputs, such as time-series data, without incurring a substantial loss
in classification performance. We propose and examine two methods for
complexity reduction in AstroNet [Shallue & Vanderburg, 2018], a CNN for
automatic classification of time-varying brightness data of stars to detect
exoplanets. The first method makes only a tactical reduction of layers in
AstroNet while the second method also modifies the original input data by means
of a Gaussian pyramid. We conducted our experiments with various degrees of
dropout regularization. Our results show only a non-substantial loss in
accuracy compared to the original AstroNet, while reducing training time up to
85 percent. These results show potential for similar reductions in other CNN
applications while largely retaining accuracy.
Query Scheduling in the Presence of Complex User Profiles
Authors: Haggai Roitman, Avigdor Gal, Louiqa Raschid
Date: 2019-02-27 00:00:00
Tasks: Scheduling

Advances in Web technology enable personalization proxies that assist users
in satisfying their complex information monitoring and aggregation needs
through the repeated querying of multiple volatile data sources. Such proxies
face a scalability challenge when trying to maximize the number of clients
served while at the same time fully satisfying clients' complex user profiles.
In this work we use an abstraction of complex execution intervals (CEIs)
constructed over simple execution intervals (EIs) represents user profiles and
use existing offline approximation as a baseline for maximizing completeness of
capturing CEIs. We present three heuristic solutions for the online problem of
query scheduling to satisfy complex user profiles. The first only considers
properties of individual EIs while the other two exploit properties of all EIs
in the CEI. We use an extensive set of experiments on real traces and synthetic
data to show that heuristics that exploit knowledge of the CEIs dominate across
multiple parameter settings.
Can learning from natural image denoising be used for seismic data interpolation?
Authors: Hao Zhang, Xiuyan Yang, Jianwei Ma
Date: 2019-02-27 00:00:00
Tasks: De-aliasing, Denoising, Image Denoising

We propose a convolutional neural network (CNN) denoising based method for seismic data interpolation. It provides a simple and efficient way to break though the lack problem of geophysical training labels that are often required by deep learning methods. The new method consists of two steps: (1) Train a set of CNN denoisers from natural image clean-noisy pairs to learn denoising; (2) Integrate the trained CNN denoisers into project onto convex set (POCS) framework to perform seismic data interpolation. The method alleviates the demanding of seismic big data with similar features as applications of end-to-end deep learning on seismic data interpolation. Additionally, the proposed method is flexible for many cases of traces missing because missing cases are not involved in the training step, and thus it is of plug-and-play nature. These indicate the high generalizability of our approach and the reduction of the need of the problem-specific training. Primary results on synthetic and field data show promising interpolation performances of the presented CNN-POCS method in terms of signal-to-noise ratio, de-aliasing and weak-feature reconstruction, in comparison with traditional $f$-$x$ prediction filtering and curvelet transform based POCS methods.
User-based collaborative filtering approach for content recommendation in OpenCourseWare platforms
Authors: Nikola Tomasevic, Dejan Paunovic, Sanja Vranes
Date: 2019-02-27 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

A content recommender system or a recommendation system represents a subclass
of information filtering systems which seeks to predict the user preferences,
i.e. the content that would be most likely positively "rated" by the user.
Nowadays, the recommender systems of OpenCourseWare (OCW) platforms typically
generate a list of recommendations in one of two ways, i.e. through the
content-based filtering, or user-based collaborative filtering (CF). In this
paper, the conceptual idea of the content recommendation module was provided,
which is capable of proposing the related decks (presentations, educational
material, etc.) to the user having in mind past user activities, preferences,
type and content similarity, etc. It particularly analyses suitable techniques
for implementation of the user-based CF approach and user-related features that
are relevant for the content evaluation. The proposed approach also envisages a
hybrid recommendation system as a combination of user-based and content-based
approaches in order to provide a holistic and efficient solution for content
recommendation. Finally, for evaluation and testing purposes, a designated
content recommendation module was implemented as part of the SlideWiki
authoring OCW platform.
Cross validation in sparse linear regression with piecewise continuous nonconvex penalties and its acceleration
Authors: Tomoyuki Obuchi, Ayaka Sakata
Date: 2019-02-27 00:00:00
Tasks: regression

We investigate the signal reconstruction performance of sparse linear regression in the presence of noise when piecewise continuous nonconvex penalties are used. Among such penalties, we focus on the SCAD penalty. The contributions of this study are three-fold: We first present a theoretical analysis of a typical reconstruction performance, using the replica method, under the assumption that each component of the design matrix is given as an independent and identically distributed (i.i.d.) Gaussian variable. This clarifies the superiority of the SCAD estimator compared with $\ell_1$ in a wide parameter range, although the nonconvex nature of the penalty tends to lead to solution multiplicity in certain regions. This multiplicity is shown to be connected to replica symmetry breaking in the spin-glass theory. We also show that the global minimum of the mean square error between the estimator and the true signal is located in the replica symmetric phase. Second, we develop an approximate formula efficiently computing the cross-validation error without actually conducting the cross-validation, which is also applicable to the non-i.i.d. design matrices. It is shown that this formula is only applicable to the unique solution region and tends to be unstable in the multiple solution region. We implement instability detection procedures, which allows the approximate formula to stand alone and resultantly enables us to draw phase diagrams for any specific dataset. Third, we propose an annealing procedure, called nonconvexity annealing, to obtain the solution path efficiently. Numerical simulations are conducted on simulated datasets to examine these results to verify the theoretical results consistency and the approximate formula efficiency. Another numerical experiment on a real-world dataset is conducted; its results are consistent with those of earlier studies using the $\ell_0$ formulation.
Domain-Constrained Advertising Keyword Generation
Authors: Hao Zhou, Minlie Huang, Yishun Mao, Changlei Zhu, Peng Shu, Xiaoyan Zhu
Date: 2019-02-27 00:00:00
Tasks: Reinforcement Learning, Retrieval

Advertising (ad for short) keyword suggestion is important for sponsored
search to improve online advertising and increase search revenue. There are two
common challenges in this task. First, the keyword bidding problem: hot ad
keywords are very expensive for most of the advertisers because more
advertisers are bidding on more popular keywords, while unpopular keywords are
difficult to discover. As a result, most ads have few chances to be presented
to the users. Second, the inefficient ad impression issue: a large proportion
of search queries, which are unpopular yet relevant to many ad keywords, have
no ads presented on their search result pages. Existing retrieval-based or
matching-based methods either deteriorate the bidding competition or are unable
to suggest novel keywords to cover more queries, which leads to inefficient ad
impressions. To address the above issues, this work investigates to use
generative neural networks for keyword generation in sponsored search. Given a
purchased keyword (a word sequence) as input, our model can generate a set of
keywords that are not only relevant to the input but also satisfy the domain
constraint which enforces that the domain category of a generated keyword is as
expected. Furthermore, a reinforcement learning algorithm is proposed to
adaptively utilize domain-specific information in keyword generation. Offline
evaluation shows that the proposed model can generate keywords that are
diverse, novel, relevant to the source keyword, and accordant with the domain
constraint. Online evaluation shows that generative models can improve coverage
(COV), click-through rate (CTR), and revenue per mille (RPM) substantially in
sponsored search.