pwc-archive/papers-with-abstracts huggingface dataset:
Adaptive Hard Thresholding for Near-optimal Consistent Robust Regression
Authors: Arun Sai Suggala, Kush Bhatia, Pradeep Ravikumar, Prateek Jain
Date: 2019-03-19 00:00:00
Tasks: regression

We study the problem of robust linear regression with response variable
corruptions. We consider the oblivious adversary model, where the adversary
corrupts a fraction of the responses in complete ignorance of the data. We
provide a nearly linear time estimator which consistently estimates the true
regression vector, even with $1-o(1)$ fraction of corruptions. Existing results
in this setting either don't guarantee consistent estimates or can only handle
a small fraction of corruptions. We also extend our estimator to robust sparse
linear regression and show that similar guarantees hold in this setting.
Finally, we apply our estimator to the problem of linear regression with
heavy-tailed noise and show that our estimator consistently estimates the
regression vector even when the noise has unbounded variance (e.g., Cauchy
distribution), for which most existing results don't even apply. Our estimator
is based on a novel variant of outlier removal via hard thresholding in which
the threshold is chosen adaptively and crucially relies on randomness to escape
bad fixed points of the non-convex hard thresholding operation.
Dynamic Learning of Sequential Choice Bandit Problem under Marketing Fatigue
Authors: Junyu Cao, Wei Sun
Date: 2019-03-19 00:00:00
Tasks: Combinatorial Optimization, Marketing

Motivated by the observation that overexposure to unwanted marketing
activities leads to customer dissatisfaction, we consider a setting where a
platform offers a sequence of messages to its users and is penalized when users
abandon the platform due to marketing fatigue. We propose a novel sequential
choice model to capture multiple interactions taking place between the platform
and its user: Upon receiving a message, a user decides on one of the three
actions: accept the message, skip and receive the next message, or abandon the
platform. Based on user feedback, the platform dynamically learns users'
abandonment distribution and their valuations of messages to determine the
length of the sequence and the order of the messages, while maximizing the
cumulative payoff over a horizon of length T. We refer to this online learning
task as the sequential choice bandit problem. For the offline combinatorial
optimization problem, we show that an efficient polynomial-time algorithm
exists. For the online problem, we propose an algorithm that balances
exploration and exploitation, and characterize its regret bound. Lastly, we
demonstrate how to extend the model with user contexts to incorporate
personalization.
Surprises in High-Dimensional Ridgeless Least Squares Interpolation
Authors: Trevor Hastie, Andrea Montanari, Saharon Rosset, Ryan J. Tibshirani
Date: 2019-03-19 00:00:00
Tasks: Vocal Bursts Intensity Prediction

Interpolators -- estimators that achieve zero training error -- have attracted growing attention in machine learning, mainly because state-of-the art neural networks appear to be models of this type. In this paper, we study minimum $\ell_2$ norm (``ridgeless'') interpolation in high-dimensional least squares regression. We consider two different models for the feature distribution: a linear model, where the feature vectors $x_i \in {\mathbb R}^p$ are obtained by applying a linear transform to a vector of i.i.d.\ entries, $x_i = \Sigma^{1/2} z_i$ (with $z_i \in {\mathbb R}^p$); and a nonlinear model, where the feature vectors are obtained by passing the input through a random one-layer neural network, $x_i = \varphi(W z_i)$ (with $z_i \in {\mathbb R}^d$, $W \in {\mathbb R}^{p \times d}$ a matrix of i.i.d.\ entries, and $\varphi$ an activation function acting componentwise on $W z_i$). We recover -- in a precise quantitative way -- several phenomena that have been observed in large-scale neural networks and kernel machines, including the "double descent" behavior of the prediction risk, and the potential benefits of overparametrization.
Stochastic (Approximate) Proximal Point Methods: Convergence, Optimality, and Adaptivity
Authors: Hilal Asi, John C. Duchi
Date: 2018-10-12 00:00:00

We develop model-based methods for solving stochastic convex optimization problems, introducing the approximate-proximal point, or aProx, family, which includes stochastic subgradient, proximal point, and bundle methods. When the modeling approaches we propose are appropriately accurate, the methods enjoy stronger convergence and robustness guarantees than classical approaches, even though the model-based methods typically add little to no computational overhead over stochastic subgradient methods. For example, we show that improved models converge with probability 1 and enjoy optimal asymptotic normality results under weak assumptions; these methods are also adaptive to a natural class of what we term easy optimization problems, achieving linear convergence under appropriate strong growth conditions on the objective. Our substantial experimental investigation shows the advantages of more accurate modeling over standard subgradient methods across many smooth and non-smooth optimization problems.
Web Annotation as a First Class Object
Authors: Paolo Ciccarese, Stian Soiland-Reyes, Tim Clark
Date: 2013-10-24 00:00:00
Tasks: Object

Scholars have made handwritten notes and comments in books and manuscripts
for centuries. Today's blogs and news sites typically invite users to express
their opinions on the published content; URLs allow web resources to be shared
with accompanying annotations and comments using third-party services like
Twitter or Facebook. These contributions have until recently been constrained
within specific services, making them second-class citizens of the Web.
  Web Annotations are now emerging as fully independent Linked Data in their
own right, no longer restricted to plain textual comments in application silos.
Annotations can now range from bookmarks and comments, to fine-grained
annotations of a selection of, for example, a section of a frame within a video
stream. Technologies and standards now exist to create, publish, syndicate,
mash-up and consume, finely targeted, semantically rich digital annotations on
practically any content, as first-class Web citizens. This development is being
driven by the need for collaboration and annotation reuse amongst domain
researchers, computer scientists, scientific publishers, and scholarly content
databases.
Mask-guided Style Transfer Network for Purifying Real Images
Authors: Tongtong Zhao, Yuxiao Yan, Jinjia Peng, Huibing Wang, Xianping Fu
Date: 2019-03-19 00:00:00
Tasks: Style Transfer

Recently, the progress of learning-by-synthesis has proposed a training model
for synthetic images, which can effectively reduce the cost of human and
material resources. However, due to the different distribution of synthetic
images compared with real images, the desired performance cannot be achieved.
To solve this problem, the previous method learned a model to improve the
realism of the synthetic images. Different from the previous methods, this
paper try to purify real image by extracting discriminative and robust features
to convert outdoor real images to indoor synthetic images. In this paper, we
first introduce the segmentation masks to construct RGB-mask pairs as inputs,
then we design a mask-guided style transfer network to learn style features
separately from the attention and bkgd(background) regions and learn content
features from full and attention region. Moreover, we propose a novel
region-level task-guided loss to restrain the features learnt from style and
content. Experiments were performed using mixed studies (qualitative and
quantitative) methods to demonstrate the possibility of purifying real images
in complex directions. We evaluate the proposed method on various public
datasets, including LPW, COCO and MPIIGaze. Experimental results show that the
proposed method is effective and achieves the state-of-the-art results.
Interactive segmentation of medical images through fully convolutional neural networks
Authors: Tomas Sakinis, Fausto Milletari, Holger Roth, Panagiotis Korfiatis, Petro Kostandy, Kenneth Philbrick, Zeynettin Akkus, Ziyue Xu, Daguang Xu, Bradley J. Erickson
Date: 2019-03-19 00:00:00
Tasks: Computed Tomography (CT), Diagnostic, Image Segmentation, Interactive Segmentation, Segmentation, Semantic Segmentation

Image segmentation plays an essential role in medicine for both diagnostic
and interventional tasks. Segmentation approaches are either manual,
semi-automated or fully-automated. Manual segmentation offers full control over
the quality of the results, but is tedious, time consuming and prone to
operator bias. Fully automated methods require no human effort, but often
deliver sub-optimal results without providing users with the means to make
corrections. Semi-automated approaches keep users in control of the results by
providing means for interaction, but the main challenge is to offer a good
trade-off between precision and required interaction. In this paper we present
a deep learning (DL) based semi-automated segmentation approach that aims to be
a "smart" interactive tool for region of interest delineation in medical
images. We demonstrate its use for segmenting multiple organs on computed
tomography (CT) of the abdomen. Our approach solves some of the most pressing
clinical challenges: (i) it requires only one to a few user clicks to deliver
excellent 2D segmentations in a fast and reliable fashion; (ii) it can
generalize to previously unseen structures and "corner cases"; (iii) it
delivers results that can be corrected quickly in a smart and intuitive way up
to an arbitrary degree of precision chosen by the user and (iv) ensures high
accuracy. We present our approach and compare it to other techniques and
previous work to show the advantages brought by our method.
Multi-Level Policy and Reward Reinforcement Learning for Image Captioning
Authors: An-An Liu1, Ning Xu1, Hanwang Zhang2, Weizhi Nie1, Yuting Su1, Yongdong Zhang
Date: 2018-06-15 00:00:00
Proceeding: IJCAI 2018 6
Tasks: Image Captioning, Natural Language Understanding, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sentence

Image captioning is one of the most challenging hallmarks of AI, due to its complexity in visual and natural language understanding. As it is essentially a sequential prediction task, recent advances in image captioning use Reinforcement Learning (RL) to better explore the dynamics of word-by-word generation. However, existing RL-based image captioning methods mainly rely on a single policy network and reward function that does not well fit the multi-level (word and sentence) and multi-modal (vision and language) nature of the task. To this end, we propose a novel multi-level policy and reward RL framework for image captioning. It contains two modules: 1) Multi-Level Policy Network that can adaptively fuse the word-level policy and the sentence-level policy for the word generation; and 2) Multi-Level Reward Function that collaboratively leverages both vision-language reward and language-language reward to guide the policy. Further, we propose a guidance term to bridge the policy and the reward for RL optimization. Extensive experiments and analysis on MSCOCO and Flick- r30k show that the proposed framework can achieve competing performances with respect to different evaluation metrics.
Harmless interpolation of noisy data in regression
Authors: Vidya Muthukumar, Kailas Vodrahalli, Vignesh Subramanian, Anant Sahai
Date: 2019-03-21 00:00:00
Tasks: regression

A continuing mystery in understanding the empirical success of deep neural networks is their ability to achieve zero training error and generalize well, even when the training data is noisy and there are more parameters than data points. We investigate this overparameterized regime in linear regression, where all solutions that minimize training error interpolate the data, including noise. We characterize the fundamental generalization (mean-squared) error of any interpolating solution in the presence of noise, and show that this error decays to zero with the number of features. Thus, overparameterization can be explicitly beneficial in ensuring harmless interpolation of noise. We discuss two root causes for poor generalization that are complementary in nature -- signal "bleeding" into a large number of alias features, and overfitting of noise by parsimonious feature selectors. For the sparse linear model with noise, we provide a hybrid interpolating scheme that mitigates both these issues and achieves order-optimal MSE over all possible interpolating solutions.
On Approximate Nonlinear Gaussian Message Passing On Factor Graphs
Authors: Eike Petersen, Christian Hoffmann, Philipp Rostalski
Date: 2019-03-21 00:00:00

Factor graphs have recently gained increasing attention as a unified
framework for representing and constructing algorithms for signal processing,
estimation, and control. One capability that does not seem to be well explored
within the factor graph tool kit is the ability to handle deterministic
nonlinear transformations, such as those occurring in nonlinear filtering and
smoothing problems, using tabulated message passing rules. In this
contribution, we provide general forward (filtering) and backward (smoothing)
approximate Gaussian message passing rules for deterministic nonlinear
transformation nodes in arbitrary factor graphs fulfilling a Markov property,
based on numerical quadrature procedures for the forward pass and a
Rauch-Tung-Striebel-type approximation of the backward pass. These message
passing rules can be employed for deriving many algorithms for solving
nonlinear problems using factor graphs, as is illustrated by the proposition of
a nonlinear modified Bryson-Frazier (MBF) smoother based on the presented
message passing rules.
Perturbed-History Exploration in Stochastic Linear Bandits
Authors: Branislav Kveton, Csaba Szepesvari, Mohammad Ghavamzadeh, Craig Boutilier
Date: 2019-03-21 00:00:00

We propose a new online algorithm for cumulative regret minimization in a stochastic linear bandit. The algorithm pulls the arm with the highest estimated reward in a linear model trained on its perturbed history. Therefore, we call it perturbed-history exploration in a linear bandit (LinPHE). The perturbed history is a mixture of observed rewards and randomly generated i.i.d. pseudo-rewards. We derive a $\tilde{O}(d \sqrt{n})$ gap-free bound on the $n$-round regret of LinPHE, where $d$ is the number of features. The key steps in our analysis are new concentration and anti-concentration bounds on the weighted sum of Bernoulli random variables. To show the generality of our design, we generalize LinPHE to a logistic model. We evaluate our algorithms empirically and show that they are practical.
Progressive Sparse Local Attention for Video object detection
Authors: Chaoxu Guo, Bin Fan, Jie Gu, Qian Zhang, Shiming Xiang, Veronique Prinet, Chunhong Pan
Date: 2019-03-21 00:00:00
Proceeding: ICCV 2019 10
Tasks: Object, object-detection, Object Detection, Optical Flow Estimation, Video Object Detection

Transferring image-based object detectors to the domain of videos remains a challenging problem. Previous efforts mostly exploit optical flow to propagate features across frames, aiming to achieve a good trade-off between accuracy and efficiency. However, introducing an extra model to estimate optical flow can significantly increase the overall model size. The gap between optical flow and high-level features can also hinder it from establishing spatial correspondence accurately. Instead of relying on optical flow, this paper proposes a novel module called Progressive Sparse Local Attention (PSLA), which establishes the spatial correspondence between features across frames in a local region with progressively sparser stride and uses the correspondence to propagate features. Based on PSLA, Recursive Feature Updating (RFU) and Dense Feature Transforming (DenseFT) are proposed to model temporal appearance and enrich feature representation respectively in a novel video object detection framework. Experiments on ImageNet VID show that our method achieves the best accuracy compared to existing methods with smaller model size and acceptable runtime speed.
PProCRC: Probabilistic Collaboration of Image Patches
Authors: Tapabrata Chakraborti, Brendan McCane, Steven Mills, Umapada Pal
Date: 2019-03-21 00:00:00
Tasks: Face Recognition

We present a conditional probabilistic framework for collaborative representation of image patches. It incorporates background compensation and outlier patch suppression into the main formulation itself, thus doing away with the need for pre-processing steps to handle the same. A closed form non-iterative solution of the cost function is derived. The proposed method (PProCRC) outperforms earlier CRC formulations: patch based (PCRC, GP-CRC) as well as the state-of-the-art probabilistic (ProCRC and EProCRC) on three fine-grained species recognition datasets (Oxford Flowers, Oxford-IIIT Pets and CUB Birds) using two CNN backbones (Vgg-19 and ResNet-50).
Finite Sample Analysis of Stochastic System Identification
Authors: Anastasios Tsiamis, George J. Pappas
Date: 2019-03-21 00:00:00
Tasks: parameter estimation, valid

In this paper, we analyze the finite sample complexity of stochastic system
identification using modern tools from machine learning and statistics. An
unknown discrete-time linear system evolves over time under Gaussian noise
without external inputs. The objective is to recover the system parameters as
well as the Kalman filter gain, given a single trajectory of output
measurements over a finite horizon of length $N$. Based on a subspace
identification algorithm and a finite number of $N$ output samples, we provide
non-asymptotic high-probability upper bounds for the system parameter
estimation errors. Our analysis uses recent results from random matrix theory,
self-normalized martingales and SVD robustness, in order to show that with high
probability the estimation errors decrease with a rate of $1/\sqrt{N}$. Our
non-asymptotic bounds not only agree with classical asymptotic results, but are
also valid even when the system is marginally stable.
Closed-Form Optimal Two-View Triangulation Based on Angular Errors
Authors: Seong Hun Lee, Javier Civera
Date: 2019-03-21 00:00:00
Proceeding: ICCV 2019 10
Tasks: Form, Vocal Bursts Valence Prediction

In this paper, we study closed-form optimal solutions to two-view triangulation with known internal calibration and pose. By formulating the triangulation problem as $L_1$ and $L_\infty$ minimization of angular reprojection errors, we derive the exact closed-form solutions that guarantee global optimality under respective cost functions. To the best of our knowledge, we are the first to present such solutions. Since the angular error is rotationally invariant, our solutions can be applied for any type of central cameras, be it perspective, fisheye or omnidirectional. Our methods also require significantly less computation than the existing optimal methods. Experimental results on synthetic and real datasets validate our theoretical derivations.
A Principled Approach for Learning Task Similarity in Multitask Learning
Authors: Changjian Shui, Mahdieh Abbasi, Louis-Émile Robitaille, Boyu Wang, Christian Gagné
Date: 2019-03-21 00:00:00

Multitask learning aims at solving a set of related tasks simultaneously, by exploiting the shared knowledge for improving the performance on individual tasks. Hence, an important aspect of multitask learning is to understand the similarities within a set of tasks. Previous works have incorporated this similarity information explicitly (e.g., weighted loss for each task) or implicitly (e.g., adversarial loss for feature adaptation), for achieving good empirical performances. However, the theoretical motivations for adding task similarity knowledge are often missing or incomplete. In this paper, we give a different perspective from a theoretical point of view to understand this practice. We first provide an upper bound on the generalization error of multitask learning, showing the benefit of explicit and implicit task similarity knowledge. We systematically derive the bounds based on two distinct task similarity metrics: H divergence and Wasserstein distance. From these theoretical results, we revisit the Adversarial Multi-task Neural Network, proposing a new training algorithm to learn the task relation coefficients and neural network parameters iteratively. We assess our new algorithm empirically on several benchmarks, showing not only that we find interesting and robust task relations, but that the proposed approach outperforms the baselines, reaffirming the benefits of theoretical insight in algorithm design.
Levelling the Playing Field: A Comprehensive Comparison of Visual Place Recognition Approaches under Changing Conditions
Authors: Mubariz Zaffar, Ahmad Khaliq, Shoaib Ehsan, Michael Milford, Klaus McDonald-Maier
Date: 2019-03-21 00:00:00
Tasks: Visual Place Recognition

In recent years there has been significant improvement in the capability of
Visual Place Recognition (VPR) methods, building on the success of both
hand-crafted and learnt visual features, temporal filtering and usage of
semantic scene information. The wide range of approaches and the relatively
recent growth in interest in the field has meant that a wide range of datasets
and assessment methodologies have been proposed, often with a focus only on
precision-recall type metrics, making comparison difficult. In this paper we
present a comprehensive approach to evaluating the performance of 10
state-of-the-art recently-developed VPR techniques, which utilizes three
standardized metrics: (a) Matching Performance b) Matching Time c) Memory
Footprint. Together this analysis provides an up-to-date and widely
encompassing snapshot of the various strengths and weaknesses of contemporary
approaches to the VPR problem. The aim of this work is to help move this
particular research field towards a more mature and unified approach to the
problem, enabling better comparison and hence more progress to be made in
future research.
Scanning Probe State Recognition With Multi-Class Neural Network Ensembles
Authors: O. Gordon, P. D'Hondt, L. Knijff, S. Freeney, F. Junqueira, P. Moriarty, I. Swart
Date: 2019-03-21 00:00:00

One of the largest obstacles facing scanning probe microscopy is the constant
need to correct flaws in the scanning probe in situ. This is currently a
manual, time-consuming process that would benefit greatly from automation. Here
we introduce a convolutional neural network protocol that enables automated
recognition of a variety of desirable and undesirable scanning probe tip states
on both metal and non-metal surfaces. By combining the best performing models
into majority voting ensembles, we find that the desirable states of H:Si(100)
can be distinguished with a mean precision of 0.89 and an average
receiver-operator-characteristic curve area of 0.95. More generally, high and
low-quality tips can be distinguished with a mean precision of 0.96 and near
perfect area-under-curve of 0.98. With trivial modifications, we also
successfully automatically identify undesirable, non-surface-specific states on
surfaces of Au(111) and Cu(111). In these cases we find mean precisions of 0.95
and 0.75 and area-under-curves of 0.98 and 0.94, respectively.