pwc-archive/papers-with-abstracts huggingface dataset:
Real-time image-based instrument classification for laparoscopic surgery
Authors: Sebastian Bodenstedt, Antonia Ohnemus, Darko Katic, Anna-Laura Wekerle, Martin Wagner, Hannes Kenngott, Beat Müller-Stich, Rüdiger Dillmann, Stefanie Speidel
Date: 2018-08-01 00:00:00
Tasks: Classification, General Classification

During laparoscopic surgery, context-aware assistance systems aim to
alleviate some of the difficulties the surgeon faces. To ensure that the right
information is provided at the right time, the current phase of the
intervention has to be known. Real-time locating and classification the
surgical tools currently in use are key components of both an activity-based
phase recognition and assistance generation.
  In this paper, we present an image-based approach that detects and classifies
tools during laparoscopic interventions in real-time. First, potential
instrument bounding boxes are detected using a pixel-wise random forest
segmentation. Each of these bounding boxes is then classified using a cascade
of random forest. For this, multiple features, such as histograms over hue and
saturation, gradients and SURF feature, are extracted from each detected
bounding box.
  We evaluated our approach on five different videos from two different types
of procedures. We distinguished between the four most common classes of
instruments (LigaSure, atraumatic grasper, aspirator, clip applier) and
background. Our method succesfully located up to 86% of all instruments
respectively. On manually provided bounding boxes, we achieve a instrument type
recognition rate of up to 58% and on automatically detected bounding boxes up
to 49%.
  To our knowledge, this is the first approach that allows an image-based
classification of surgical tools in a laparoscopic setting in real-time.
Learning Dexterous In-Hand Manipulation
Authors: OpenAI, Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, Jonas Schneider, Szymon Sidor, Josh Tobin, Peter Welinder, Lilian Weng, Wojciech Zaremba
Date: 2018-08-01 00:00:00
Tasks: Friction, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We use reinforcement learning (RL) to learn dexterous in-hand manipulation
policies which can perform vision-based object reorientation on a physical
Shadow Dexterous Hand. The training is performed in a simulated environment in
which we randomize many of the physical properties of the system like friction
coefficients and an object's appearance. Our policies transfer to the physical
robot despite being trained entirely in simulation. Our method does not rely on
any human demonstrations, but many behaviors found in human manipulation emerge
naturally, including finger gaiting, multi-finger coordination, and the
controlled use of gravity. Our results were obtained using the same distributed
RL system that was used to train OpenAI Five. We also include a video of our
results: https://youtu.be/jwSbzNHGflM
Shuffle-Then-Assemble: Learning Object-Agnostic Visual Relationship Features
Authors: Xu Yang, Hanwang Zhang, Jianfei Cai
Date: 2018-08-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Object, Triplet

Due to the fact that it is prohibitively expensive to completely annotate
visual relationships, i.e., the (obj1, rel, obj2) triplets, relationship models
are inevitably biased to object classes of limited pairwise patterns, leading
to poor generalization to rare or unseen object combinations. Therefore, we are
interested in learning object-agnostic visual features for more generalizable
relationship models. By "agnostic", we mean that the feature is less likely
biased to the classes of paired objects. To alleviate the bias, we propose a
novel \texttt{Shuffle-Then-Assemble} pre-training strategy. First, we discard
all the triplet relationship annotations in an image, leaving two unpaired
object domains without obj1-obj2 alignment. Then, our feature learning is to
recover possible obj1-obj2 pairs. In particular, we design a cycle of residual
transformations between the two domains, to capture shared but not
object-specific visual patterns. Extensive experiments on two visual
relationship benchmarks show that by using our pre-trained features, naive
relationship models can be consistently improved and even outperform other
state-of-the-art relationship models. Code has been made available at:
\url{https://github.com/yangxuntu/vrd}.
Instance-level Human Parsing via Part Grouping Network
Authors: Ke Gong, Xiaodan Liang, Yicheng Li, Yimin Chen, Ming Yang, Liang Lin
Date: 2018-08-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Edge Detection, Human Parsing, Human Part Segmentation, Representation Learning

Instance-level human parsing towards real-world human analysis scenarios is
still under-explored due to the absence of sufficient data resources and
technical difficulty in parsing multiple instances in a single pass. Several
related works all follow the "parsing-by-detection" pipeline that heavily
relies on separately trained detection models to localize instances and then
performs human parsing for each instance sequentially. Nonetheless, two
discrepant optimization targets of detection and parsing lead to suboptimal
representation learning and error accumulation for final results. In this work,
we make the first attempt to explore a detection-free Part Grouping Network
(PGN) for efficiently parsing multiple people in an image in a single pass. Our
PGN reformulates instance-level human parsing as two twinned sub-tasks that can
be jointly learned and mutually refined via a unified network: 1) semantic part
segmentation for assigning each pixel as a human part (e.g., face, arms); 2)
instance-aware edge detection to group semantic parts into distinct person
instances. Thus the shared intermediate representation would be endowed with
capabilities in both characterizing fine-grained parts and inferring instance
belongings of each part. Finally, a simple instance partition process is
employed to get final results during inference. We conducted experiments on
PASCAL-Person-Part dataset and our PGN outperforms all state-of-the-art
methods. Furthermore, we show its superiority on a newly collected multi-person
parsing dataset (CIHP) including 38,280 diverse images, which is the largest
dataset so far and can facilitate more advanced human analysis. The CIHP
benchmark and our source code are available at http://sysu-hcp.net/lip/.
Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network
Authors: Xinjing Cheng, Peng Wang, Ruigang Yang
Date: 2018-08-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Depth Estimation, Depth Prediction

Depth estimation from a single image is a fundamental problem in computer
vision. In this paper, we propose a simple yet effective convolutional spatial
propagation network (CSPN) to learn the affinity matrix for depth prediction.
Specifically, we adopt an efficient linear propagation model, where the
propagation is performed with a manner of recurrent convolutional operation,
and the affinity among neighboring pixels is learned through a deep
convolutional neural network (CNN). We apply the designed CSPN to two depth
estimation tasks given a single image: (1) To refine the depth output from
state-of-the-art (SOTA) existing methods; and (2) to convert sparse depth
samples to a dense depth map by embedding the depth samples within the
propagation procedure. The second task is inspired by the availability of
LIDARs that provides sparse but accurate depth measurements. We experimented
the proposed CSPN over two popular benchmarks for depth estimation, i.e. NYU v2
and KITTI, where we show that our proposed approach improves in not only
quality (e.g., 30% more reduction in depth error), but also speed (e.g., 2 to 5
times faster) than prior SOTA methods.
Sleep-wake classification via quantifying heart rate variability by convolutional neural network
Authors: John Malik, Yu-Lun Lo, Hau-Tieng Wu
Date: 2018-08-01 00:00:00
Tasks: General Classification, Heart Rate Variability, Specificity

Fluctuations in heart rate are intimately tied to changes in the
physiological state of the organism. We examine and exploit this relationship
by classifying a human subject's wake/sleep status using his instantaneous
heart rate (IHR) series. We use a convolutional neural network (CNN) to build
features from the IHR series extracted from a whole-night electrocardiogram
(ECG) and predict every 30 seconds whether the subject is awake or asleep. Our
training database consists of 56 normal subjects, and we consider three
different databases for validation; one is private, and two are public with
different races and apnea severities. On our private database of 27 subjects,
our accuracy, sensitivity, specificity, and AUC values for predicting the wake
stage are 83.1%, 52.4%, 89.4%, and 0.83, respectively. Validation performance
is similar on our two public databases. When we use the photoplethysmography
instead of the ECG to obtain the IHR series, the performance is also
comparable. A robustness check is carried out to confirm the obtained
performance statistics. This result advocates for an effective and scalable
method for recognizing changes in physiological state using non-invasive heart
rate monitoring. The CNN model adaptively quantifies IHR fluctuation as well as
its location in time and is suitable for differentiating between the wake and
sleep stages.
Action Anticipation By Predicting Future Dynamic Images
Authors: Cristian Rodriguez, Basura Fernando, Hongdong Li
Date: 2018-08-01 00:00:00
Tasks: Action Anticipation, Autonomous Driving, motion prediction

Human action-anticipation methods predict what is the future action by
observing only a few portion of an action in progress. This is critical for
applications where computers have to react to human actions as early as
possible such as autonomous driving, human-robotic interaction, assistive
robotics among others. In this paper, we present a method for human action
anticipation by predicting the most plausible future human motion. We represent
human motion using Dynamic Images and make use of tailored loss functions to
encourage a generative model to produce accurate future motion prediction. Our
method outperforms the currently best performing action-anticipation methods by
4% on JHMDB-21, 5.2% on UT-Interaction and 5.1% on UCF 101-24 benchmarks.
Multi-modal Cycle-consistent Generalized Zero-Shot Learning
Authors: Rafael Felix, B. G. Vijay Kumar, Ian Reid, Gustavo Carneiro
Date: 2018-08-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: General Classification, Generalized Zero-Shot Learning, Zero-Shot Learning

In generalized zero shot learning (GZSL), the set of classes are split into
seen and unseen classes, where training relies on the semantic features of the
seen and unseen classes and the visual representations of only the seen
classes, while testing uses the visual representations of the seen and unseen
classes. Current methods address GZSL by learning a transformation from the
visual to the semantic space, exploring the assumption that the distribution of
classes in the semantic and visual spaces is relatively similar. Such methods
tend to transform unseen testing visual representations into one of the seen
classes' semantic features instead of the semantic features of the correct
unseen class, resulting in low accuracy GZSL classification. Recently,
generative adversarial networks (GAN) have been explored to synthesize visual
representations of the unseen classes from their semantic features - the
synthesized representations of the seen and unseen classes are then used to
train the GZSL classifier. This approach has been shown to boost GZSL
classification accuracy, however, there is no guarantee that synthetic visual
representations can generate back their semantic feature in a multi-modal
cycle-consistent manner. This constraint can result in synthetic visual
representations that do not represent well their semantic features. In this
paper, we propose the use of such constraint based on a new regularization for
the GAN training that forces the generated visual features to reconstruct their
original semantic features. Once our model is trained with this multi-modal
cycle-consistent semantic compatibility, we can then synthesize more
representative visual representations for the seen and, more importantly, for
the unseen classes. Our proposed approach shows the best GZSL classification
results in the field in several publicly available datasets.
A Theory of Dichotomous Valuation with Applications to Variable Selection
Authors: Xingwei Hu
Date: 2018-08-01 00:00:00
Tasks: Variable Selection

An econometric or statistical model may undergo a marginal gain if we admit a new variable to the model, and a marginal loss if we remove an existing variable from the model. Assuming equality of opportunity among all candidate variables, we derive a valuation framework by the expected marginal gain and marginal loss in all potential modeling scenarios. However, marginal gain and loss are not symmetric; thus, we introduce three unbiased solutions. When used in variable selection, our new approaches significantly outperform several popular methods used in practice. The results also explore some novel traits of the Shapley value.
FMCode: A 3D In-the-Air Finger Motion Based User Login Framework for Gesture Interface
Authors: Duo Lu, Dijiang Huang
Date: 2018-08-01 00:00:00
Tasks: User Identification

Applications using gesture-based human-computer interface require a new user
login method with gestures because it does not have a traditional input method
to type a password. However, due to various challenges, existing gesture-based
authentication systems are generally considered too weak to be useful in
practice. In this paper, we propose a unified user login framework using 3D
in-air-handwriting, called FMCode. We define new types of features critical to
distinguish legitimate users from attackers and utilize Support Vector Machine
(SVM) for user authentication. The features and data-driven models are
specially designed to accommodate minor behavior variations that existing
gesture authentication methods neglect. In addition, we use deep neural network
approaches to efficiently identify the user based on his or her
in-air-handwriting, which avoids expansive account database search methods
employed by existing work. On a dataset collected by us with over 100 users,
our prototype system achieves 0.1% and 0.5% best Equal Error Rate (EER) for
user authentication, as well as 96.7% and 94.3% accuracy for user
identification, using two types of gesture input devices. Compared to existing
behavioral biometric systems using gesture and in-air-handwriting, our
framework achieves the state-of-the-art performance. In addition, our
experimental results show that FMCode is capable to defend against client-side
spoofing attacks, and it performs persistently in the long run. These results
and discoveries pave the way to practical usage of gesture-based user login
over the gesture interface.
EagleEye: Attack-Agnostic Defense against Adversarial Inputs (Technical Report)
Authors: Yujie Ji, Xinyang Zhang, Ting Wang
Date: 2018-08-01 00:00:00

Deep neural networks (DNNs) are inherently vulnerable to adversarial inputs:
such maliciously crafted samples trigger DNNs to misbehave, leading to
detrimental consequences for DNN-powered systems. The fundamental challenges of
mitigating adversarial inputs stem from their adaptive and variable nature.
Existing solutions attempt to improve DNN resilience against specific attacks;
yet, such static defenses can often be circumvented by adaptively engineered
inputs or by new attack variants.
  Here, we present EagleEye, an attack-agnostic adversarial tampering analysis
engine for DNN-powered systems. Our design exploits the {\em minimality
principle} underlying many attacks: to maximize the attack's evasiveness, the
adversary often seeks the minimum possible distortion to convert genuine inputs
to adversarial ones. We show that this practice entails the distinct
distributional properties of adversarial inputs in the input space. By
leveraging such properties in a principled manner, EagleEye effectively
discriminates adversarial inputs and even uncovers their correct classification
outputs. Through extensive empirical evaluation using a range of benchmark
datasets and DNN models, we validate EagleEye's efficacy. We further
investigate the adversary's possible countermeasures, which implies a difficult
dilemma for her: to evade EagleEye's detection, excessive distortion is
necessary, thereby significantly reducing the attack's evasiveness regarding
other detection mechanisms.
Toward Multimodal Interaction in Scalable Visual Digital Evidence Visualization Using Computer Vision Techniques and ISS
Authors: Serguei A. Mokhov, Miao Song, Jashanjot Singh, Joey Paquet, Mourad Debbabi, Sudhir Mudur
Date: 2018-08-01 00:00:00
Tasks: Management, multimodal interaction

Visualization requirements in Forensic Lucid have to do with different levels
of case knowledge abstraction, representation, aggregation, as well as the
operational aspects as the final long-term goal of this proposal. It
encompasses anything from the finer detailed representation of hierarchical
contexts to Forensic Lucid programs, to the documented evidence and its
management, its linkage to programs, to evaluation, and to the management of
GIPSY software networks. This includes an ability to arbitrarily switch between
those views combined with usable multimodal interaction. The purpose is to
determine how the findings can be applied to Forensic Lucid and investigation
case management. It is also natural to want a convenient and usable evidence
visualization, its semantic linkage and the reasoning machinery for it. Thus,
we propose a scalable management, visualization, and evaluation of digital
evidence using the modified interactive 3D documentary system - Illimitable
Space System - (ISS) to represent, semantically link, and provide a usable
interface to digital investigators that is navigable via different multimodal
interaction techniques using Computer Vision techniques including gestures, as
well as eye-gaze and audio.
Learning Stabilizable Dynamical Systems via Control Contraction Metrics
Authors: Sumeet Singh, Vikas Sindhwani, Jean-Jacques E. Slotine, Marco Pavone
Date: 2018-07-31 00:00:00
Tasks: continuous-control, Continuous Control, Model-based Reinforcement Learning, Reinforcement Learning

We propose a novel framework for learning stabilizable nonlinear dynamical
systems for continuous control tasks in robotics. The key idea is to develop a
new control-theoretic regularizer for dynamics fitting rooted in the notion of
stabilizability, which guarantees that the learned system can be accompanied by
a robust controller capable of stabilizing any open-loop trajectory that the
system may generate. By leveraging tools from contraction theory, statistical
learning, and convex optimization, we provide a general and tractable
semi-supervised algorithm to learn stabilizable dynamics, which can be applied
to complex underactuated systems. We validated the proposed algorithm on a
simulated planar quadrotor system and observed notably improved trajectory
generation and tracking performance with the control-theoretic regularized
model over models learned using traditional regression techniques, especially
when using a small number of demonstration examples. The results presented
illustrate the need to infuse standard model-based reinforcement learning
algorithms with concepts drawn from nonlinear control theory for improved
reliability.
Probability Calibration Trees
Authors: Tim Leathart, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer
Date: 2018-07-31 00:00:00
Tasks: regression

Obtaining accurate and well calibrated probability estimates from classifiers
is useful in many applications, for example, when minimising the expected cost
of classifications. Existing methods of calibrating probability estimates are
applied globally, ignoring the potential for improvements by applying a more
fine-grained model. We propose probability calibration trees, a modification of
logistic model trees that identifies regions of the input space in which
different probability calibration models are learned to improve performance. We
compare probability calibration trees to two widely used calibration
methods---isotonic regression and Platt scaling---and show that our method
results in lower root mean squared error on average than both methods, for
estimates produced by a variety of base learners.
An Ontology-Based Recommender System with an Application to the Star Trek Television Franchise
Authors: Paul Sheridan, Mikael Onsjö, Claudia Becerra, Sergio Jimenez, George Dueñas
Date: 2018-07-31 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems, Topic Models

Collaborative filtering based recommender systems have proven to be extremely successful in settings where user preference data on items is abundant. However, collaborative filtering algorithms are hindered by their weakness against the item cold-start problem and general lack of interpretability. Ontology-based recommender systems exploit hierarchical organizations of users and items to enhance browsing, recommendation, and profile construction. While ontology-based approaches address the shortcomings of their collaborative filtering counterparts, ontological organizations of items can be difficult to obtain for items that mostly belong to the same category (e.g., television series episodes). In this paper, we present an ontology-based recommender system that integrates the knowledge represented in a large ontology of literary themes to produce fiction content recommendations. The main novelty of this work is an ontology-based method for computing similarities between items and its integration with the classical Item-KNN (K-nearest neighbors) algorithm. As a study case, we evaluated the proposed method against other approaches by performing the classical rating prediction task on a collection of Star Trek television series episodes in an item cold-start scenario. This transverse evaluation provides insights into the utility of different information resources and methods for the initial stages of recommender system development. We found our proposed method to be a convenient alternative to collaborative filtering approaches for collections of mostly similar items, particularly when other content-based approaches are not applicable or otherwise unavailable. Aside from the new methods, this paper contributes a testbed for future research and an online framework to collaboratively extend the ontology of literary themes to cover other narrative content.