pwc-archive/papers-with-abstracts huggingface dataset:
Single-Forward-Step Projective Splitting: Exploiting Cocoercivity
Authors: Patrick R. Johnstone, Jonathan Eckstein
Date: 2019-02-24 00:00:00

This work describes a new variant of projective splitting for solving maximal monotone inclusions and complicated convex optimization problems. In the new version, cocoercive operators can be processed with a single forward step per iteration. In the convex optimization context, cocoercivity is equivalent to Lipschitz differentiability. Prior forward-step versions of projective splitting did not fully exploit cocoercivity and required two forward steps per iteration for such operators. Our new single-forward-step method establishes a symmetry between projective splitting algorithms, the classical forward-backward splitting method (FB), and Tseng's forward-backward-forward method (FBF). The new procedure allows for larger stepsizes for cocoercive operators: the stepsize bound is $2\beta$ for a $\beta$-cocoercive operator, the same bound as has been established for FB. We show that FB corresponds to an unattainable boundary case of the parameters in the new procedure. Unlike FB, the new method allows for a backtracking procedure when the cocoercivity constant is unknown. Proving convergence of the algorithm requires some departures from the prior proof framework for projective splitting. We close with some computational tests establishing competitive performance for the method.
Nonlinear generalization of the monotone single index model
Authors: Zeljko Kereta, Timo Klock, Valeriya Naumova
Date: 2019-02-24 00:00:00
Tasks: model, regression

Single index model is a powerful yet simple model, widely used in statistics, machine learning, and other scientific fields. It models the regression function as $g(<a,x>)$, where a is an unknown index vector and x are the features. This paper deals with a nonlinear generalization of this framework to allow for a regressor that uses multiple index vectors, adapting to local changes in the responses. To do so we exploit the conditional distribution over function-driven partitions, and use linear regression to locally estimate index vectors. We then regress by applying a kNN type estimator that uses a localized proxy of the geodesic metric. We present theoretical guarantees for estimation of local index vectors and out-of-sample prediction, and demonstrate the performance of our method with experiments on synthetic and real-world data sets, comparing it with state-of-the-art methods.
Automatic ISP image quality tuning using non-linear optimization
Authors: Jun Nishimura, Timo Gerasimow, Sushma Rao, Aleksandar Sutic, Chyuan-Tyng Wu, Gilad Michael
Date: 2019-02-24 00:00:00

Image Signal Processor (ISP) comprises of various blocks to reconstruct image
sensor raw data to final image consumed by human visual system or computer
vision applications. Each block typically has many tuning parameters due to the
complexity of the operation. These need to be hand tuned by Image Quality (IQ)
experts, which takes considerable amount of time. In this paper, we present an
automatic IQ tuning using nonlinear optimization and automatic reference
generation algorithms. The proposed method can produce high quality IQ in
minutes as compared with weeks of hand-tuned results by IQ experts. In
addition, the proposed method can work with any algorithms without being aware
of their specific implementation. It was found successful on multiple different
processing blocks such as noise reduction, demosaic, and sharpening.
Artificial Constraints and Lipschitz Hints for Unconstrained Online Learning
Authors: Ashok Cutkosky
Date: 2019-02-24 00:00:00

We provide algorithms that guarantee regret $R_T(u)\le \tilde O(G\|u\|^3 +
G(\|u\|+1)\sqrt{T})$ or $R_T(u)\le \tilde O(G\|u\|^3T^{1/3} + GT^{1/3}+
G\|u\|\sqrt{T})$ for online convex optimization with $G$-Lipschitz losses for
any comparison point $u$ without prior knowledge of either $G$ or $\|u\|$.
Previous algorithms dispense with the $O(\|u\|^3)$ term at the expense of
knowledge of one or both of these parameters, while a lower bound shows that
some additional penalty term over $G\|u\|\sqrt{T}$ is necessary. Previous
penalties were exponential while our bounds are polynomial in all quantities.
Further, given a known bound $\|u\|\le D$, our same techniques allow us to
design algorithms that adapt optimally to the unknown value of $\|u\|$ without
requiring knowledge of $G$.
Efficient Private Algorithms for Learning Large-Margin Halfspaces
Authors: Huy L. Nguyen, Jonathan Ullman, Lydia Zakynthinou
Date: 2019-02-24 00:00:00

We present new differentially private algorithms for learning a large-margin halfspace. In contrast to previous algorithms, which are based on either differentially private simulations of the statistical query model or on private convex optimization, the sample complexity of our algorithms depends only on the margin of the data, and not on the dimension. We complement our results with a lower bound, showing that the dependence of our upper bounds on the margin is optimal.
Learning to Perform Role-Filler Binding with Schematic Knowledge
Authors: Catherine Chen, Qihong Lu, Andre Beukers, Christopher Baldassano, Kenneth A. Norman
Date: 2019-02-24 00:00:00
Tasks: Question Answering, Sentence

Through specific experiences, humans learn relationships underlying the structure of events in the world. Schema theory suggests that we organize this information in mental frameworks called "schemata," which represent our knowledge of the structure of the world. Generalizing knowledge of structural relationships to new situations requires role-filler binding, the ability to associate specific "fillers" with abstract "roles." For instance, when we hear the sentence "Alice ordered a tea from Bob," the role-filler bindings "Alice:customer," "tea:drink," and "Bob:barista" allow us to understand and make inferences about the sentence. We can perform these bindings for arbitrary fillers -- we understand this sentence even if we have never heard the names "Alice," "tea," or "Bob" before. In this work, we define a model as capable of performing role-filler binding if it can recall arbitrary fillers corresponding to a specified role, even when these pairings violate correlations seen during training. Previous work found that models can learn this ability when explicitly told what the roles and fillers are, or when given fillers seen during training. We show that networks with external memory can learn these relationships with fillers not seen during training and without explicitly labeled role-filler bindings, and show that analyses inspired by neural decoding can provide a means of understanding what the networks have learned.
Combining Online Learning Guarantees
Authors: Ashok Cutkosky
Date: 2019-02-24 00:00:00

We show how to take any two parameter-free online learning algorithms with
different regret guarantees and obtain a single algorithm whose regret is the
minimum of the two base algorithms. Our method is embarrassingly simple: just
add the iterates. This trick can generate efficient algorithms that adapt to
many norms simultaneously, as well as providing diagonal-style algorithms that
still maintain dimension-free guarantees. We then proceed to show how a variant
on this idea yields a black-box procedure for generating optimistic online
learning algorithms. This yields the first optimistic regret guarantees in the
unconstrained setting and generically increases adaptivity. Further, our
optimistic algorithms are guaranteed to do no worse than their non-optimistic
counterparts regardless of the quality of the optimistic estimates provided to
the algorithm.
High Dimensional Restrictive Federated Model Selection with multi-objective Bayesian Optimization over shifted distributions
Authors: Xudong Sun, Andrea Bommert, Florian Pfisterer, Jörg Rahnenführer, Michel Lang, Bernd Bischl
Date: 2019-02-24 00:00:00
Tasks: Bayesian Optimization, BIG-bench Machine Learning, Federated Learning, Model Selection

A novel machine learning optimization process coined Restrictive Federated Model Selection (RFMS) is proposed under the scenario, for example, when data from healthcare units can not leave the site it is situated on and it is forbidden to carry out training algorithms on remote data sites due to either technical or privacy and trust concerns. To carry out a clinical research under this scenario, an analyst could train a machine learning model only on local data site, but it is still possible to execute a statistical query at a certain cost in the form of sending a machine learning model to some of the remote data sites and get the performance measures as feedback, maybe due to prediction being usually much cheaper. Compared to federated learning, which is optimizing the model parameters directly by carrying out training across all data sites, RFMS trains model parameters only on one local data site but optimizes hyper-parameters across other data sites jointly since hyper-parameters play an important role in machine learning performance. The aim is to get a Pareto optimal model with respective to both local and remote unseen prediction losses, which could generalize well across data sites. In this work, we specifically consider high dimensional data with shifted distributions over data sites. As an initial investigation, Bayesian Optimization especially multi-objective Bayesian Optimization is used to guide an adaptive hyper-parameter optimization process to select models under the RFMS scenario. Empirical results show that solely using the local data site to tune hyper-parameters generalizes poorly across data sites, compared to methods that utilize the local and remote performances. Furthermore, in terms of dominated hypervolumes, multi-objective Bayesian Optimization algorithms show increased performance across multiple data sites among other candidates.
U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instrument
Authors: S. M. Kamrul Hasan, Cristian A. Linte
Date: 2019-02-24 00:00:00
Tasks: Data Augmentation, Decoder, Instance Segmentation, Segmentation, Semantic Segmentation

Conventional therapy approaches limit surgeons' dexterity control due to
limited field-of-view. With the advent of robot-assisted surgery, there has
been a paradigm shift in medical technology for minimally invasive surgery.
However, it is very challenging to track the position of the surgical
instruments in a surgical scene, and accurate detection & identification of
surgical tools is paramount. Deep learning-based semantic segmentation in
frames of surgery videos has the potential to facilitate this task. In this
work, we modify the U-Net architecture named U-NetPlus, by introducing a
pre-trained encoder and re-design the decoder part, by replacing the transposed
convolution operation with an upsampling operation based on nearest-neighbor
(NN) interpolation. To further improve performance, we also employ a very fast
and flexible data augmentation technique. We trained the framework on 8 x 225
frame sequences of robotic surgical videos, available through the MICCAI 2017
EndoVis Challenge dataset and tested it on 8 x 75 frame and 2 x 300 frame
videos. Using our U-NetPlus architecture, we report a 90.20% DICE for binary
segmentation, 76.26% DICE for instrument part segmentation, and 46.07% for
instrument type (i.e., all instruments) segmentation, outperforming the results
of previous techniques implemented and tested on these data.
Chronic-Pain Protective Behavior Detection with Deep Learning
Authors: Chongyang Wang, Temitayo A. Olugbade, Akhil Mathur, Amanda C. De C. Williams, Nicholas D. Lane, Nadia Bianchi-Berthouze
Date: 2019-02-24 00:00:00
Tasks: Deep Learning, Management

In chronic pain rehabilitation, physiotherapists adapt physical activity to patients' performance based on their expression of protective behavior, gradually exposing them to feared but harmless and essential everyday activities. As rehabilitation moves outside the clinic, technology should automatically detect such behavior to provide similar support. Previous works have shown the feasibility of automatic protective behavior detection (PBD) within a specific activity. In this paper, we investigate the use of deep learning for PBD across activity types, using wearable motion capture and surface electromyography data collected from healthy participants and people with chronic pain. We approach the problem by continuously detecting protective behavior within an activity rather than estimating its overall presence. The best performance reaches mean F1 score of 0.82 with leave-one-subject-out cross validation. When protective behavior is modelled per activity type, performance is mean F1 score of 0.77 for bend-down, 0.81 for one-leg-stand, 0.72 for sit-to-stand, 0.83 for stand-to-sit, and 0.67 for reach-forward. This performance reaches excellent level of agreement with the average experts' rating performance suggesting potential for personalized chronic pain management at home. We analyze various parameters characterizing our approach to understand how the results could generalize to other PBD datasets and different levels of ground truth granularity.
Transferability of Deep Learning Algorithms for Malignancy Detection in Confocal Laser Endomicroscopy Images from Different Anatomical Locations of the Upper Gastrointestinal Tract
Authors: Marc Aubreville, Miguel Goncalves, Christian Knipfer, Nicolai Oetter, Helmut Neumann, Florian Stelzle, Christopher Bohr, Andreas Maier
Date: 2019-02-24 00:00:00
Tasks: Anatomy, General Classification, Transfer Learning

Squamous Cell Carcinoma (SCC) is the most common cancer type of the epithelium and is often detected at a late stage. Besides invasive diagnosis of SCC by means of biopsy and histo-pathologic assessment, Confocal Laser Endomicroscopy (CLE) has emerged as noninvasive method that was successfully used to diagnose SCC in vivo. For interpretation of CLE images, however, extensive training is required, which limits its applicability and use in clinical practice of the method. To aid diagnosis of SCC in a broader scope, automatic detection methods have been proposed. This work compares two methods with regard to their applicability in a transfer learning sense, i.e. training on one tissue type (from one clinical team) and applying the learnt classification system to another entity (different anatomy, different clinical team). Besides a previously proposed, patch-based method based on convolutional neural networks, a novel classification method on image level (based on a pre-trained Inception V.3 network with dedicated preprocessing and interpretation of class activation maps) is proposed and evaluated. The newly presented approach improves recognition performance, yielding accuracies of 91.63% on the first data set (oral cavity) and 92.63% on a joint data set. The generalization from oral cavity to the second data set (vocal folds) lead to similar area-under-the-ROC curve values than a direct training on the vocal folds data set, indicating good generalization.
An Online Learning Approach to Model Predictive Control
Authors: Nolan Wagener, Ching-An Cheng, Jacob Sacks, Byron Boots
Date: 2019-02-24 00:00:00
Tasks: Decision Making, model, Model Predictive Control

Model predictive control (MPC) is a powerful technique for solving dynamic control tasks. In this paper, we show that there exists a close connection between MPC and online learning, an abstract theoretical framework for analyzing online decision making in the optimization literature. This new perspective provides a foundation for leveraging powerful online learning algorithms to design MPC algorithms. Specifically, we propose a new algorithm based on dynamic mirror descent (DMD), an online learning algorithm that is designed for non-stationary setups. Our algorithm, Dynamic Mirror Descent Model Predictive Control (DMD-MPC), represents a general family of MPC algorithms that includes many existing techniques as special instances. DMD-MPC also provides a fresh perspective on previous heuristics used in MPC and suggests a principled way to design new MPC algorithms. In the experimental section of this paper, we demonstrate the flexibility of DMD-MPC, presenting a set of new MPC algorithms on a simple simulated cartpole and a simulated and real-world aggressive driving task. Videos of the real-world experiments can be found at https://youtu.be/vZST3v0_S9w and https://youtu.be/MhuqiHo2t98.
A Formalization of The Natural Gradient Method for General Similarity Measures
Authors: Anton Mallasto, Tom Dela Haije, Aasa Feragen
Date: 2019-02-24 00:00:00

In optimization, the natural gradient method is well-known for likelihood
maximization. The method uses the Kullback-Leibler divergence, corresponding
infinitesimally to the Fisher-Rao metric, which is pulled back to the parameter
space of a family of probability distributions. This way, gradients with
respect to the parameters respect the Fisher-Rao geometry of the space of
distributions, which might differ vastly from the standard Euclidean geometry
of the parameter space, often leading to faster convergence. However, when
minimizing an arbitrary similarity measure between distributions, it is
generally unclear which metric to use. We provide a general framework that,
given a similarity measure, derives a metric for the natural gradient. We then
discuss connections between the natural gradient method and multiple other
optimization techniques in the literature. Finally, we provide computations of
the formal natural gradient to show overlap with well-known cases and to
compute natural gradients in novel frameworks.
Synchronous Bidirectional Inference for Neural Sequence Generation
Authors: Jiajun Zhang, Long Zhou, Yang Zhao, Cheng-qing Zong
Date: 2019-02-24 00:00:00
Tasks: Abstractive Text Summarization, Machine Translation, Translation

In sequence to sequence generation tasks (e.g. machine translation and
abstractive summarization), inference is generally performed in a left-to-right
manner to produce the result token by token. The neural approaches, such as
LSTM and self-attention networks, are now able to make full use of all the
predicted history hypotheses from left side during inference, but cannot
meanwhile access any future (right side) information and usually generate
unbalanced outputs in which left parts are much more accurate than right ones.
In this work, we propose a synchronous bidirectional inference model to
generate outputs using both left-to-right and right-to-left decoding
simultaneously and interactively. First, we introduce a novel beam search
algorithm that facilitates synchronous bidirectional decoding. Then, we present
the core approach which enables left-to-right and right-to-left decoding to
interact with each other, so as to utilize both the history and future
predictions simultaneously during inference. We apply the proposed model to
both LSTM and self-attention networks. In addition, we propose two strategies
for parameter optimization. The extensive experiments on machine translation
and abstractive summarization demonstrate that our synchronous bidirectional
inference model can achieve remarkable improvements over the strong baselines.
Training GANs with Centripetal Acceleration
Authors: Wei Peng, Yu-Hong Dai, HUI ZHANG, Li-Zhi Cheng
Date: 2019-02-24 00:00:00

Training generative adversarial networks (GANs) often suffers from cyclic
behaviors of iterates. Based on a simple intuition that the direction of
centripetal acceleration of an object moving in uniform circular motion is
toward the center of the circle, we present the Simultaneous Centripetal
Acceleration (SCA) method and the Alternating Centripetal Acceleration (ACA)
method to alleviate the cyclic behaviors. Under suitable conditions, gradient
descent methods with either SCA or ACA are shown to be linearly convergent for
bilinear games. Numerical experiments are conducted by applying ACA to existing
gradient-based algorithms in a GAN setup scenario, which demonstrate the
superiority of ACA.
Text Analysis in Adversarial Settings: Does Deception Leave a Stylistic Trace?
Authors: Tommi Gröndahl, N. Asokan
Date: 2019-02-24 00:00:00
Tasks: text-classification, Text Classification

Textual deception constitutes a major problem for online security. Many
studies have argued that deceptiveness leaves traces in writing style, which
could be detected using text classification techniques. By conducting an
extensive literature review of existing empirical work, we demonstrate that
while certain linguistic features have been indicative of deception in certain
corpora, they fail to generalize across divergent semantic domains. We suggest
that deceptiveness as such leaves no content-invariant stylistic trace, and
textual similarity measures provide superior means of classifying texts as
potentially deceptive. Additionally, we discuss forms of deception beyond
semantic content, focusing on hiding author identity by writing style
obfuscation. Surveying the literature on both author identification and
obfuscation techniques, we conclude that current style transformation methods
fail to achieve reliable obfuscation while simultaneously ensuring semantic
faithfulness to the original text. We propose that future work in style
transformation should pay particular attention to disallowing semantically
drastic changes.
Privacy Preserving Location Data Publishing: A Machine Learning Approach
Authors: Sina Shaham, Ming Ding, Bo Liu, Shuping Dang, Zihuai Lin, Jun Li
Date: 2019-02-24 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, Multiple Sequence Alignment, Privacy Preserving, User Identification

Publishing datasets plays an essential role in open data research and promoting transparency of government agencies. However, such data publication might reveal users' private information. One of the most sensitive sources of data is spatiotemporal trajectory datasets. Unfortunately, merely removing unique identifiers cannot preserve the privacy of users. Adversaries may know parts of the trajectories or be able to link the published dataset to other sources for the purpose of user identification. Therefore, it is crucial to apply privacy preserving techniques before the publication of spatiotemporal trajectory datasets. In this paper, we propose a robust framework for the anonymization of spatiotemporal trajectory datasets termed as machine learning based anonymization (MLA). By introducing a new formulation of the problem, we are able to apply machine learning algorithms for clustering the trajectories and propose to use $k$-means algorithm for this purpose. A variation of $k$-means algorithm is also proposed to preserve the privacy in overly sensitive datasets. Moreover, we improve the alignment process by considering multiple sequence alignment as part of the MLA. The framework and all the proposed algorithms are applied to TDrive and Geolife location datasets. The experimental results indicate a significantly higher utility of datasets by anonymization based on MLA framework.