pwc-archive/papers-with-abstracts huggingface dataset:
SMSSVD - SubMatrix Selection Singular Value Decomposition
Authors: Rasmus Henningsson, Magnus Fontes
Date: 2017-10-23 00:00:00
Tasks: Dimensionality Reduction

High throughput biomedical measurements normally capture multiple overlaid
biologically relevant signals and often also signals representing different
types of technical artefacts like e.g. batch effects. Signal identification and
decomposition are accordingly main objectives in statistical biomedical
modeling and data analysis. Existing methods, aimed at signal reconstruction
and deconvolution, in general, are either supervised, contain parameters that
need to be estimated or present other types of ad hoc features. We here
introduce SubMatrix Selection SingularValue Decomposition (SMSSVD), a
parameter-free unsupervised signal decomposition and dimension reduction
method, designed to reduce noise, adaptively for each low-rank-signal in a
given data matrix, and represent the signals in the data in a way that enable
unbiased exploratory analysis and reconstruction of multiple overlaid signals,
including identifying groups of variables that drive different signals.
  The Submatrix Selection Singular Value Decomposition (SMSSVD) method produces
a denoised signal decomposition from a given data matrix. The SMSSVD method
guarantees orthogonality between signal components in a straightforward manner
and it is designed to make automation possible. We illustrate SMSSVD by
applying it to several real and synthetic datasets and compare its performance
to golden standard methods like PCA (Principal Component Analysis) and SPC
(Sparse Principal Components, using Lasso constraints). The SMSSVD is
computationally efficient and despite being a parameter-free method, in
general, outperforms existing statistical learning methods.
  A Julia implementation of SMSSVD is openly available on GitHub
(https://github.com/rasmushenningsson/SMSSVD.jl).
An iterative closest point method for measuring the level of similarity of 3d log scans in wood industry
Authors: Cyrine Selma, Hind Haouzi, Philippe Thomas, Jonathan Gaudreault, Michael Morin
Date: 2017-10-23 00:00:00
Tasks: BIG-bench Machine Learning

In the Canadian's lumber industry, simulators are used to predict the lumbers
resulting from the sawing of a log at a given sawmill. Giving a log or several
logs' 3D scans as input, simulators perform a real-time job to predict the
lumbers. These simulators, however, tend to be slow at processing large volume
of wood. We thus explore an alternative approximation techniques based on the
Iterative Closest Point (ICP) algorithm to identify the already processed log
to which an unseen log resembles the most. The main benefit of the ICP approach
is that it can easily handle 3D scans with a variable number of points. We
compare this ICP-based nearest neighbor predictor, to predictors built using
machine learning algorithms such as the K-nearest-neighbor (kNN) and Random
Forest (RF). The implemented ICP-based predictor enabled us to identify key
points in using the 3D scans directly for distance calculation. The long-term
goal of this ongoing research is to integrated ICP distance calculations and
machine learning.
Image reconstruction with imperfect forward models and applications in deblurring
Authors: Yury Korolev, Jan Lellmann
Date: 2017-08-03 00:00:00
Tasks: Deblurring, Image Reconstruction

We present and analyse an approach to image reconstruction problems with
imperfect forward models based on partially ordered spaces - Banach lattices.
In this approach, errors in the data and in the forward models are described
using order intervals. The method can be characterised as the lattice analogue
of the residual method, where the feasible set is defined by linear inequality
constraints. The study of this feasible set is the main contribution of this
paper. Convexity of this feasible set is examined in several settings and
modifications for introducing additional information about the forward operator
are considered. Numerical examples demonstrate the performance of the method in
deblurring with errors in the blurring kernel.
Model Distillation with Knowledge Transfer from Face Classification to Alignment and Verification
Authors: Chong Wang, Xipeng Lan, Yangang Zhang
Date: 2017-09-09 00:00:00
Proceeding: ICLR 2018 1
Tasks: Classification, Face Recognition, General Classification, Knowledge Distillation, Model Compression, Retrieval, Transfer Learning

Knowledge distillation is a potential solution for model compression. The
idea is to make a small student network imitate the target of a large teacher
network, then the student network can be competitive to the teacher one. Most
previous studies focus on model distillation in the classification task, where
they propose different architects and initializations for the student network.
However, only the classification task is not enough, and other related tasks
such as regression and retrieval are barely considered. To solve the problem,
in this paper, we take face recognition as a breaking point and propose model
distillation with knowledge transfer from face classification to alignment and
verification. By selecting appropriate initializations and targets in the
knowledge transfer, the distillation can be easier in non-classification tasks.
Experiments on the CelebA and CASIA-WebFace datasets demonstrate that the
student network can be competitive to the teacher one in alignment and
verification, and even surpasses the teacher network under specific compression
rates. In addition, to achieve stronger knowledge transfer, we also use a
common initialization trick to improve the distillation performance of
classification. Evaluations on the CASIA-Webface and large-scale MS-Celeb-1M
datasets show the effectiveness of this simple trick.
Supervised Quantum Learning without Measurements
Authors: Unai Alvarez-Rodriguez, Lucas Lamata, Pablo Escandell-Montero, José D. Martín-Guerrero, Enrique Solano
Date: 2016-12-16 00:00:00
Tasks: BIG-bench Machine Learning, Quantum Machine Learning

We propose a quantum machine learning algorithm for efficiently solving a
class of problems encoded in quantum controlled unitary operations. The central
physical mechanism of the protocol is the iteration of a quantum time-delayed
equation that introduces feedback in the dynamics and eliminates the necessity
of intermediate measurements. The performance of the quantum algorithm is
analyzed by comparing the results obtained in numerical simulations with the
outcome of classical machine learning methods for the same problem. The use of
time-delayed equations enhances the toolbox of the field of quantum machine
learning, which may enable unprecedented applications in quantum technologies.
Accelerating GMM-based patch priors for image restoration: Three ingredients for a 100$\times$ speed-up
Authors: Shibin Parameswaran, Charles-Alban Deledalle, Loïc Denis, Truong Q. Nguyen
Date: 2017-10-23 00:00:00
Tasks: CPU, Deblurring, Denoising, GPU, Image Restoration, Super-Resolution

Image restoration methods aim to recover the underlying clean image from
corrupted observations. The Expected Patch Log-likelihood (EPLL) algorithm is a
powerful image restoration method that uses a Gaussian mixture model (GMM)
prior on the patches of natural images. Although it is very effective for
restoring images, its high runtime complexity makes EPLL ill-suited for most
practical applications. In this paper, we propose three approximations to the
original EPLL algorithm. The resulting algorithm, which we call the fast-EPLL
(FEPLL), attains a dramatic speed-up of two orders of magnitude over EPLL while
incurring a negligible drop in the restored image quality (less than 0.5 dB).
We demonstrate the efficacy and versatility of our algorithm on a number of
inverse problems such as denoising, deblurring, super-resolution, inpainting
and devignetting. To the best of our knowledge, FEPLL is the first algorithm
that can competitively restore a 512x512 pixel image in under 0.5s for all the
degradations mentioned above without specialized code optimizations such as CPU
parallelization or GPU implementation.
Aggregating Algorithm for Prediction of Packs
Authors: Dmitry Adamskiy, Tony Bellotti, Raisa Dzhamtyrova, Yuri Kalnishkan
Date: 2017-10-23 00:00:00
Tasks: Prediction

This paper formulates the protocol for prediction of packs, which a special
case of prediction under delayed feedback. Under this protocol, the learner
must make a few predictions without seeing the outcomes and then the outcomes
are revealed. We develop the theory of prediction with expert advice for packs.
By applying Vovk's Aggregating Algorithm to this problem we obtain a number of
algorithms with tight upper bounds. We carry out empirical experiments on
housing data.
Self-paced Learning for Weakly Supervised Evidence Discovery in Multimedia Event Search
Authors: Mengyi Liu, Lu Jiang, Shiguang Shan, Alexander G. Hauptmann
Date: 2016-08-12 00:00:00
Tasks: Event Detection

Multimedia event detection has been receiving increasing attention in recent
years. Besides recognizing an event, the discovery of evidences (which is
refered to as "recounting") is also crucial for user to better understand the
searching result. Due to the difficulty of evidence annotation, only limited
supervision of event labels are available for training a recounting model. To
deal with the problem, we propose a weakly supervised evidence discovery method
based on self-paced learning framework, which follows a learning process from
easy "evidences" to gradually more complex ones, and simultaneously exploit
more and more positive evidence samples from numerous weakly annotated video
segments. Moreover, to evaluate our method quantitatively, we also propose two
metrics, \textit{PctOverlap} and \textit{F1-score}, for measuring the
performance of evidence localization specifically. The experiments are
conducted on a subset of TRECVID MED dataset and demonstrate the promising
results obtained by our method.
Evidence of an exponential speed-up in the solution of hard optimization problems
Authors: Fabio L. Traversa, Pietro Cicotti, Forrest Sheldon, Massimiliano Di Ventra
Date: 2017-10-23 00:00:00

Optimization problems pervade essentially every scientific discipline and
industry. Many such problems require finding a solution that maximizes the
number of constraints satisfied. Often, these problems are particularly
difficult to solve because they belong to the NP-hard class, namely algorithms
that always find a solution in polynomial time are not known. Over the past
decades, research has focused on developing heuristic approaches that attempt
to find an approximation to the solution. However, despite numerous research
efforts, in many cases even approximations to the optimal solution are hard to
find, as the computational time for further refining a candidate solution grows
exponentially with input size. Here, we show a non-combinatorial approach to
hard optimization problems that achieves an exponential speed-up and finds
better approximations than the current state-of-the-art. First, we map the
optimization problem into a boolean circuit made of specially designed,
self-organizing logic gates, which can be built with (non-quantum) electronic
components; the equilibrium points of the circuit represent the approximation
to the problem at hand. Then, we solve its associated non-linear ordinary
differential equations numerically, towards the equilibrium points. We
demonstrate this exponential gain by comparing a sequential MatLab
implementation of our solver with the winners of the 2016 Max-SAT competition
on a variety of hard optimization instances. We show empirical evidence that
our solver scales linearly with the size of the problem, both in time and
memory, and argue that this property derives from the collective behavior of
the simulated physical circuit. Our approach can be applied to other types of
optimization problems and the results presented here have far-reaching
consequences in many fields.
Efficient Online Minimization for Low-Rank Subspace Clustering
Authors: Jie Shen, Ping Li, Huan Xu
Date: 2015-03-28 00:00:00
Tasks: Clustering

Low-rank representation~(LRR) has been a significant method for segmenting
data that are generated from a union of subspaces. It is, however, known that
solving the LRR program is challenging in terms of time complexity and memory
footprint, in that the size of the nuclear norm regularized matrix is
$n$-by-$n$ (where $n$ is the number of samples). In this paper, we thereby
develop a fast online implementation of LRR that reduces the memory cost from
$O(n^2)$ to $O(pd)$, with $p$ being the ambient dimension and $d$ being some
estimated rank~($d < p \ll n$). The crux for this end is a non-convex
reformulation of the LRR program, which pursues the basis dictionary that
generates the (uncorrupted) observations. We build the theoretical guarantee
that the sequence of the solutions produced by our algorithm converges to a
stationary point of the empirical and the expected loss function
asymptotically. Extensive experiments on synthetic and realistic datasets
further substantiate that our algorithm is fast, robust and memory efficient.
SideEye: A Generative Neural Network Based Simulator of Human Peripheral Vision
Authors: Lex Fridman, Benedikt Jenik, Shaiyan Keshvari, Bryan Reimer, Christoph Zetzsche, Ruth Rosenholtz
Date: 2017-06-14 00:00:00

Foveal vision makes up less than 1% of the visual field. The other 99% is
peripheral vision. Precisely what human beings see in the periphery is both
obvious and mysterious in that we see it with our own eyes but can't visualize
what we see, except in controlled lab experiments. Degradation of information
in the periphery is far more complex than what might be mimicked with a radial
blur. Rather, behaviorally-validated models hypothesize that peripheral vision
measures a large number of local texture statistics in pooling regions that
overlap and grow with eccentricity. In this work, we develop a new method for
peripheral vision simulation by training a generative neural network on a
behaviorally-validated full-field synthesis model. By achieving a 21,000 fold
reduction in running time, our approach is the first to combine realism and
speed of peripheral vision simulation to a degree that provides a whole new way
to approach visual design: through peripheral visualization.
Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks
Authors: Chunjie Luo, Jianfeng Zhan, Lei Wang, Qiang Yang
Date: 2017-02-20 00:00:00

Traditionally, multi-layer neural networks use dot product between the output
vector of previous layer and the incoming weight vector as the input to
activation function. The result of dot product is unbounded, thus increases the
risk of large variance. Large variance of neuron makes the model sensitive to
the change of input distribution, thus results in poor generalization, and
aggravates the internal covariate shift which slows down the training. To bound
dot product and decrease the variance, we propose to use cosine similarity or
centered cosine similarity (Pearson Correlation Coefficient) instead of dot
product in neural networks, which we call cosine normalization. We compare
cosine normalization with batch, weight and layer normalization in
fully-connected neural networks as well as convolutional networks on the data
sets of MNIST, 20NEWS GROUP, CIFAR-10/100 and SVHN. Experiments show that
cosine normalization achieves better performance than other normalization
techniques.
Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
Authors: Yudong Chen, Lili Su, Jiaming Xu
Date: 2017-05-16 00:00:00
Tasks: BIG-bench Machine Learning, Federated Learning

We consider the problem of distributed statistical machine learning in
adversarial settings, where some unknown and time-varying subset of working
machines may be compromised and behave arbitrarily to prevent an accurate model
from being learned. This setting captures the potential adversarial attacks
faced by Federated Learning -- a modern machine learning paradigm that is
proposed by Google researchers and has been intensively studied for ensuring
user privacy. Formally, we focus on a distributed system consisting of a
parameter server and $m$ working machines. Each working machine keeps $N/m$
data samples, where $N$ is the total number of samples. The goal is to
collectively learn the underlying true model parameter of dimension $d$.
  In classical batch gradient descent methods, the gradients reported to the
server by the working machines are aggregated via simple averaging, which is
vulnerable to a single Byzantine failure. In this paper, we propose a Byzantine
gradient descent method based on the geometric median of means of the
gradients. We show that our method can tolerate $q \le (m-1)/2$ Byzantine
failures, and the parameter estimate converges in $O(\log N)$ rounds with an
estimation error of $\sqrt{d(2q+1)/N}$, hence approaching the optimal error
rate $\sqrt{d/N}$ in the centralized and failure-free setting. The total
computational complexity of our algorithm is of $O((Nd/m) \log N)$ at each
working machine and $O(md + kd \log^3 N)$ at the central server, and the total
communication cost is of $O(m d \log N)$. We further provide an application of
our general results to the linear regression problem.
  A key challenge arises in the above problem is that Byzantine failures create
arbitrary and unspecified dependency among the iterations and the aggregated
gradients. We prove that the aggregated gradient converges uniformly to the
true gradient function.
A First Step in Combining Cognitive Event Features and Natural Language Representations to Predict Emotions
Authors: Andres Campero, Bjarke Felbo, Joshua B. Tenenbaum, Rebecca Saxe
Date: 2017-10-23 00:00:00

We explore the representational space of emotions by combining methods from
different academic fields. Cognitive science has proposed appraisal theory as a
view on human emotion with previous research showing how human-rated abstract
event features can predict fine-grained emotions and capture the similarity
space of neural patterns in mentalizing brain regions. At the same time,
natural language processing (NLP) has demonstrated how transfer and multitask
learning can be used to cope with scarcity of annotated data for text modeling.
  The contribution of this work is to show that appraisal theory can be
combined with NLP for mutual benefit. First, fine-grained emotion prediction
can be improved to human-level performance by using NLP representations in
addition to appraisal features. Second, using the appraisal features as
auxiliary targets during training can improve predictions even when only text
is available as input. Third, we obtain a representation with a similarity
matrix that better correlates with the neural activity across regions. Best
results are achieved when the model is trained to simultaneously predict
appraisals, emotions and emojis using a shared representation.
  While these results are preliminary, the integration of cognitive
neuroscience and NLP techniques opens up an interesting direction for future
research.
Sequential Matrix Completion
Authors: Annie Marsden, Sergio Bacallado
Date: 2017-10-23 00:00:00
Tasks: Collaborative Filtering, Matrix Completion, Recommendation Systems, Thompson Sampling

We propose a novel algorithm for sequential matrix completion in a
recommender system setting, where the $(i,j)$th entry of the matrix corresponds
to a user $i$'s rating of product $j$. The objective of the algorithm is to
provide a sequential policy for user-product pair recommendation which will
yield the highest possible ratings after a finite time horizon. The algorithm
uses a Gamma process factor model with two posterior-focused bandit policies,
Thompson Sampling and Information-Directed Sampling. While Thompson Sampling
shows competitive performance in simulations, state-of-the-art performance is
obtained from Information-Directed Sampling, which makes its recommendations
based off a ratio between the expected reward and a measure of information
gain. To our knowledge, this is the first implementation of Information
Directed Sampling on large real datasets.
  This approach contributes to a recent line of research on bandit approaches
to collaborative filtering including Kawale et al. (2015), Li et al. (2010),
Bresler et al. (2014), Li et al. (2016), Deshpande & Montanari (2012), and Zhao
et al. (2013). The setting of this paper, as has been noted in Kawale et al.
(2015) and Zhao et al. (2013), presents significant challenges to bounding
regret after finite horizons. We discuss these challenges in relation to
simpler models for bandits with side information, such as linear or gaussian
process bandits, and hope the experiments presented here motivate further
research toward theoretical guarantees.
Bringing Semantic Structures to User Intent Detection in Online Medical Queries
Authors: Chenwei Zhang, Nan Du, Wei Fan, Yaliang Li, Chun-Ta Lu, Philip S. Yu
Date: 2017-10-22 00:00:00
Tasks: Intent Detection, Multi-Task Learning, Sentence

The Internet has revolutionized healthcare by offering medical information
ubiquitously to patients via web search. The healthcare status, complex medical
information needs of patients are expressed diversely and implicitly in their
medical text queries. Aiming to better capture a focused picture of user's
medical-related information search and shed insights on their healthcare
information access strategies, it is challenging yet rewarding to detect
structured user intentions from their diversely expressed medical text queries.
We introduce a graph-based formulation to explore structured concept
transitions for effective user intent detection in medical queries, where each
node represents a medical concept mention and each directed edge indicates a
medical concept transition. A deep model based on multi-task learning is
introduced to extract structured semantic transitions from user queries, where
the model extracts word-level medical concept mentions as well as
sentence-level concept transitions collectively. A customized graph-based
mutual transfer loss function is designed to impose explicit constraints and
further exploit the contribution of mentioning a medical concept word to the
implication of a semantic transition. We observe an 8% relative improvement in
AUC and 23% relative reduction in coverage error by comparing the proposed
model with the best baseline model for the concept transition inference task on
real-world medical text queries.
Deep Cropping via Attention Box Prediction and Aesthetics Assessment
Authors: Wenguan Wang, Jianbing Shen
Date: 2017-10-22 00:00:00
Proceeding: ICCV 2017 10
Tasks: Prediction

We model the photo cropping problem as a cascade of attention box regression
and aesthetic quality classification, based on deep learning. A neural network
is designed that has two branches for predicting attention bounding box and
analyzing aesthetics, respectively. The predicted attention box is treated as
an initial crop window where a set of cropping candidates are generated around
it, without missing important information. Then, aesthetics assessment is
employed to select the final crop as the one with the best aesthetic quality.
With our network, cropping candidates share features within full-image
convolutional feature maps, thus avoiding repeated feature computation and
leading to higher computation efficiency. Via leveraging rich data for
attention prediction and aesthetics assessment, the proposed method produces
high-quality cropping results, even with the limited availability of training
data for photo cropping. The experimental results demonstrate the competitive
results and fast processing speed (5 fps with all steps).
ActivityNet Challenge 2017 Summary
Authors: Bernard Ghanem, Juan Carlos Niebles, Cees Snoek, Fabian Caba Heilbron, Humam Alwassel, Ranjay Khrisna, Victor Escorcia, Kenji Hata, Shyamal Buch
Date: 2017-10-22 00:00:00
Tasks: Activity Recognition

The ActivityNet Large Scale Activity Recognition Challenge 2017 Summary:
results and challenge participants papers.