pwc-archive/papers-with-abstracts huggingface dataset:
A metric for sets of trajectories that is practical and mathematically consistent
Authors: José Bento, Jia Jie Zhu
Date: 2016-01-12 00:00:00

Metrics on the space of sets of trajectories are important for scientists in the field of computer vision, machine learning, robotics, and general artificial intelligence. However, existing notions of closeness between sets of trajectories are either mathematically inconsistent or of limited practical use. In this paper, we outline the limitations in the current mathematically-consistent metrics, which are based on OSPA (Schuhmacher et al. 2008); and the inconsistencies in the heuristic notions of closeness used in practice, whose main ideas are common to the CLEAR MOT measures (Keni and Rainer 2008) widely used in computer vision. In two steps, we then propose a new intuitive metric between sets of trajectories and address these limitations. First, we explain a solution that leads to a metric that is hard to compute. Then we modify this formulation to obtain a metric that is easy to compute while keeping the useful properties of the previous metric. Our notion of closeness is the first demonstrating the following three features: the metric 1) can be quickly computed, 2) incorporates confusion of trajectories' identity in an optimal way, and 3) is a metric in the mathematical sense.
Deep Thermal Imaging: Proximate Material Type Recognition in the Wild through Deep Learning of Spatial Surface Temperature Patterns
Authors: Youngjun Cho, Nadia Bianchi-Berthouze, Nicolai Marquardt, Simon J. Julier
Date: 2018-03-06 00:00:00
Tasks: Material Classification, Material Recognition, Thermal Infrared Object Tracking

We introduce Deep Thermal Imaging, a new approach for close-range automatic
recognition of materials to enhance the understanding of people and ubiquitous
technologies of their proximal environment. Our approach uses a low-cost mobile
thermal camera integrated into a smartphone to capture thermal textures. A deep
neural network classifies these textures into material types. This approach
works effectively without the need for ambient light sources or direct contact
with materials. Furthermore, the use of a deep learning network removes the
need to handcraft the set of features for different materials. We evaluated the
performance of the system by training it to recognise 32 material types in both
indoor and outdoor environments. Our approach produced recognition accuracies
above 98% in 14,860 images of 15 indoor materials and above 89% in 26,584
images of 17 outdoor materials. We conclude by discussing its potentials for
real-time use in HCI applications and future directions.
Joint 3D Proposal Generation and Object Detection from View Aggregation
Authors: Jason Ku, Melissa Mozifian, Jungwook Lee, Ali Harakeh, Steven Waslander
Date: 2017-12-06 00:00:00
Tasks: 3D Object Detection, Autonomous Driving, Autonomous Vehicles, General Classification, Object, object-detection, Object Detection, Region Proposal

We present AVOD, an Aggregate View Object Detection network for autonomous
driving scenarios. The proposed neural network architecture uses LIDAR point
clouds and RGB images to generate features that are shared by two subnetworks:
a region proposal network (RPN) and a second stage detector network. The
proposed RPN uses a novel architecture capable of performing multimodal feature
fusion on high resolution feature maps to generate reliable 3D object proposals
for multiple object classes in road scenes. Using these proposals, the second
stage detection network performs accurate oriented 3D bounding box regression
and category classification to predict the extents, orientation, and
classification of objects in 3D space. Our proposed architecture is shown to
produce state of the art results on the KITTI 3D object detection benchmark
while running in real time with a low memory footprint, making it a suitable
candidate for deployment on autonomous vehicles. Code is at:
https://github.com/kujason/avod
Synthesizing Neural Network Controllers with Probabilistic Model based Reinforcement Learning
Authors: Juan Camilo Gamboa Higuera, David Meger, Gregory Dudek
Date: 2018-03-06 00:00:00
Tasks: Model-based Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We present an algorithm for rapidly learning controllers for robotics
systems. The algorithm follows the model-based reinforcement learning paradigm,
and improves upon existing algorithms; namely Probabilistic learning in Control
(PILCO) and a sample-based version of PILCO with neural network dynamics
(Deep-PILCO). We propose training a neural network dynamics model using
variational dropout with truncated Log-Normal noise. This allows us to obtain a
dynamics model with calibrated uncertainty, which can be used to simulate
controller executions via rollouts. We also describe set of techniques,
inspired by viewing PILCO as a recurrent neural network model, that are crucial
to improve the convergence of the method. We test our method on a variety of
benchmark tasks, demonstrating data-efficiency that is competitive with PILCO,
while being able to optimize complex neural network controllers. Finally, we
assess the performance of the algorithm for learning motor controllers for a
six legged autonomous underwater vehicle. This demonstrates the potential of
the algorithm for scaling up the dimensionality and dataset sizes, in more
complex control tasks.
Stochastic Variational Video Prediction
Authors: Mohammad Babaeizadeh, Chelsea Finn, Dumitru Erhan, Roy H. Campbell, Sergey Levine
Date: 2017-10-30 00:00:00
Proceeding: ICLR 2018 1
Tasks: Prediction, Video Generation, Video Prediction

Predicting the future in real-world settings, particularly from raw sensory
observations such as images, is exceptionally challenging. Real-world events
can be stochastic and unpredictable, and the high dimensionality and complexity
of natural images requires the predictive model to build an intricate
understanding of the natural world. Many existing methods tackle this problem
by making simplifying assumptions about the environment. One common assumption
is that the outcome is deterministic and there is only one plausible future.
This can lead to low-quality predictions in real-world settings with stochastic
dynamics. In this paper, we develop a stochastic variational video prediction
(SV2P) method that predicts a different possible future for each sample of its
latent variables. To the best of our knowledge, our model is the first to
provide effective stochastic multi-frame prediction for real-world video. We
demonstrate the capability of the proposed method in predicting detailed future
frames of videos on multiple real-world datasets, both action-free and
action-conditioned. We find that our proposed method produces substantially
improved video predictions when compared to the same model without
stochasticity, and to other stochastic video prediction methods. Our SV2P
implementation will be open sourced upon publication.
Learning monocular visual odometry with dense 3D mapping from dense 3D flow
Authors: Cheng Zhao, Li Sun, Pulak Purkait, Tom Duckett, Rustam Stolkin
Date: 2018-03-06 00:00:00
Tasks: Monocular Visual Odometry, Visual Odometry

This paper introduces a fully deep learning approach to monocular SLAM, which
can perform simultaneous localization using a neural network for learning
visual odometry (L-VO) and dense 3D mapping. Dense 2D flow and a depth image
are generated from monocular images by sub-networks, which are then used by a
3D flow associated layer in the L-VO network to generate dense 3D flow. Given
this 3D flow, the dual-stream L-VO network can then predict the 6DOF relative
pose and furthermore reconstruct the vehicle trajectory. In order to learn the
correlation between motion directions, the Bivariate Gaussian modelling is
employed in the loss function. The L-VO network achieves an overall performance
of 2.68% for average translational error and 0.0143 deg/m for average
rotational error on the KITTI odometry benchmark. Moreover, the learned depth
is fully leveraged to generate a dense 3D map. As a result, an entire visual
SLAM system, that is, learning monocular odometry combined with dense 3D
mapping, is achieved.
Hybrid Multi-camera Visual Servoing to Moving Target
Authors: Hanz Cuevas-Velasquez, Nanbo Li, Radim Tylecek, Marcelo Saval-Calvo, Robert B. Fisher
Date: 2018-03-06 00:00:00

Visual servoing is a well-known task in robotics. However, there are still
challenges when multiple visual sources are combined to accurately guide the
robot or occlusions appear. In this paper we present a novel visual servoing
approach using hybrid multi-camera input data to lead a robot arm accurately to
dynamically moving target points in the presence of partial occlusions. The
approach uses four RGBD sensors as Eye-to-Hand (EtoH) visual input, and an
arm-mounted stereo camera as Eye-in-Hand (EinH). A Master supervisor task
selects between using the EtoH or the EinH, depending on the distance between
the robot and target. The Master also selects the subset of EtoH cameras that
best perceive the target. When the EinH sensor is used, if the target becomes
occluded or goes out of the sensor's view-frustum, the Master switches back to
the EtoH sensors to re-track the object. Using this adaptive visual input data,
the robot is then controlled using an iterative planner that uses position,
orientation and joint configuration to estimate the trajectory. Since the
target is dynamic, this trajectory is updated every time-step. Experiments show
good performance in four different situations: tracking a ball, targeting a
bulls-eye, guiding a straw to a mouth and delivering an item to a moving hand.
The experiments cover both simple situations such as a ball that is mostly
visible from all cameras, and more complex situations such as the mouth which
is partially occluded from some of the sensors.
Zero-Shot Sketch-Image Hashing
Authors: Yuming Shen, Li Liu, Fumin Shen, Ling Shao
Date: 2018-03-06 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Retrieval, Representation Learning, Retrieval, Sketch-Based Image Retrieval

Recent studies show that large-scale sketch-based image retrieval (SBIR) can
be efficiently tackled by cross-modal binary representation learning methods,
where Hamming distance matching significantly speeds up the process of
similarity search. Providing training and test data subjected to a fixed set of
pre-defined categories, the cutting-edge SBIR and cross-modal hashing works
obtain acceptable retrieval performance. However, most of the existing methods
fail when the categories of query sketches have never been seen during
training. In this paper, the above problem is briefed as a novel but realistic
zero-shot SBIR hashing task. We elaborate the challenges of this special task
and accordingly propose a zero-shot sketch-image hashing (ZSIH) model. An
end-to-end three-network architecture is built, two of which are treated as the
binary encoders. The third network mitigates the sketch-image heterogeneity and
enhances the semantic relations among data by utilizing the Kronecker fusion
layer and graph convolution, respectively. As an important part of ZSIH, we
formulate a generative hashing scheme in reconstructing semantic knowledge
representations for zero-shot retrieval. To the best of our knowledge, ZSIH is
the first zero-shot hashing work suitable for SBIR and cross-modal search.
Comprehensive experiments are conducted on two extended datasets, i.e., Sketchy
and TU-Berlin with a novel zero-shot train-test split. The proposed model
remarkably outperforms related works.
Personalized Exposure Control Using Adaptive Metering and Reinforcement Learning
Authors: Huan Yang, Baoyuan Wang, Noranart Vesdapunt, Minyi Guo, Sing Bing Kang
Date: 2018-03-06 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose a reinforcement learning approach for real-time exposure control
of a mobile camera that is personalizable. Our approach is based on Markov
Decision Process (MDP). In the camera viewfinder or live preview mode, given
the current frame, our system predicts the change in exposure so as to optimize
the trade-off among image quality, fast convergence, and minimal temporal
oscillation. We model the exposure prediction function as a fully convolutional
neural network that can be trained through Gaussian policy gradient in an
end-to-end fashion. As a result, our system can associate scene semantics with
exposure values; it can also be extended to personalize the exposure
adjustments for a user and device. We improve the learning performance by
incorporating an adaptive metering module that links semantics with exposure.
This adaptive metering module generalizes the conventional spot or matrix
metering techniques. We validate our system using the MIT FiveK and our own
datasets captured using iPhone 7 and Google Pixel. Experimental results show
that our system exhibits stable real-time behavior while improving visual
quality compared to what is achieved through native camera control.
Multilayer bootstrap networks
Authors: Xiao-Lei Zhang
Date: 2014-08-05 00:00:00
Tasks: Clustering, Dimensionality Reduction

Multilayer bootstrap network builds a gradually narrowed multilayer nonlinear
network from bottom up for unsupervised nonlinear dimensionality reduction.
Each layer of the network is a nonparametric density estimator. It consists of
a group of k-centroids clusterings. Each clustering randomly selects data
points with randomly selected features as its centroids, and learns a one-hot
encoder by one-nearest-neighbor optimization. Geometrically, the nonparametric
density estimator at each layer projects the input data space to a
uniformly-distributed discrete feature space, where the similarity of two data
points in the discrete feature space is measured by the number of the nearest
centroids they share in common. The multilayer network gradually reduces the
nonlinear variations of data from bottom up by building a vast number of
hierarchical trees implicitly on the original data space. Theoretically, the
estimation error caused by the nonparametric density estimator is proportional
to the correlation between the clusterings, both of which are reduced by the
randomization steps.
ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content
Authors: Demetris Marnerides, Thomas Bashford-Rogers, Jonathan Hatchett, Kurt Debattista
Date: 2018-03-06 00:00:00
Tasks: inverse tone mapping, Inverse-Tone-Mapping, Quantization, Tone Mapping

High dynamic range (HDR) imaging provides the capability of handling real world lighting as opposed to the traditional low dynamic range (LDR) which struggles to accurately represent images with higher dynamic range. However, most imaging content is still available only in LDR. This paper presents a method for generating HDR content from LDR content based on deep Convolutional Neural Networks (CNNs) termed ExpandNet. ExpandNet accepts LDR images as input and generates images with an expanded range in an end-to-end fashion. The model attempts to reconstruct missing information that was lost from the original signal due to quantization, clipping, tone mapping or gamma correction. The added information is reconstructed from learned features, as the network is trained in a supervised fashion using a dataset of HDR images. The approach is fully automatic and data driven; it does not require any heuristics or human expertise. ExpandNet uses a multiscale architecture which avoids the use of upsampling layers to improve image quality. The method performs well compared to expansion/inverse tone mapping operators quantitatively on multiple metrics, even for badly exposed inputs.
Methodology to analyze the accuracy of 3D objects reconstructed with collaborative robot based monocular LSD-SLAM
Authors: Sergey Triputen, Atmaraaj Gopal, Thomas Weber, Christian Hofert, Kristiaan Schreve, Matthias Ratsch
Date: 2018-03-06 00:00:00
Tasks: 3D Reconstruction, Motion Planning, Robot Navigation

SLAM systems are mainly applied for robot navigation while research on
feasibility for motion planning with SLAM for tasks like bin-picking, is
scarce. Accurate 3D reconstruction of objects and environments is important for
planning motion and computing optimal gripper pose to grasp objects. In this
work, we propose the methods to analyze the accuracy of a 3D environment
reconstructed using a LSD-SLAM system with a monocular camera mounted onto the
gripper of a collaborative robot. We discuss and propose a solution to the pose
space conversion problem. Finally, we present several criteria to analyze the
3D reconstruction accuracy. These could be used as guidelines to improve the
accuracy of 3D reconstructions with monocular LSD-SLAM and other SLAM based
solutions.
Deep Information Networks
Authors: Giulio Franzese, Monica Visintin
Date: 2018-03-06 00:00:00

We describe a novel classifier with a tree structure, designed using
information theory concepts. This Information Network is made of information
nodes, that compress the input data, and multiplexers, that connect two or more
input nodes to an output node. Each information node is trained, independently
of the others, to minimize a local cost function that minimizes the mutual
information between its input and output with the constraint of keeping a given
mutual information between its output and the target (information bottleneck).
We show that the system is able to provide good results in terms of accuracy,
while it shows many advantages in terms of modularity and reduced complexity.
MIMO Graph Filters for Convolutional Neural Networks
Authors: Fernando Gama, Antonio G. Marques, Alejandro Ribeiro, Geert Leus
Date: 2018-03-06 00:00:00

Superior performance and ease of implementation have fostered the adoption of
Convolutional Neural Networks (CNNs) for a wide array of inference and
reconstruction tasks. CNNs implement three basic blocks: convolution, pooling
and pointwise nonlinearity. Since the two first operations are well-defined
only on regular-structured data such as audio or images, application of CNNs to
contemporary datasets where the information is defined in irregular domains is
challenging. This paper investigates CNNs architectures to operate on signals
whose support can be modeled using a graph. Architectures that replace the
regular convolution with a so-called linear shift-invariant graph filter have
been recently proposed. This paper goes one step further and, under the
framework of multiple-input multiple-output (MIMO) graph filters, imposes
additional structure on the adopted graph filters, to obtain three new (more
parsimonious) architectures. The proposed architectures result in a lower
number of model parameters, reducing the computational complexity, facilitating
the training, and mitigating the risk of overfitting. Simulations show that the
proposed simpler architectures achieve similar performance as more complex
models.
CliNER 2.0: Accessible and Accurate Clinical Concept Extraction
Authors: Willie Boag, Elena Sergeeva, Saurabh Kulshreshtha, Peter Szolovits, Anna Rumshisky, Tristan Naumann
Date: 2018-03-06 00:00:00
Tasks: Clinical Concept Extraction, Decision Making, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Clinical notes often describe important aspects of a patient's stay and are
therefore critical to medical research. Clinical concept extraction (CCE) of
named entities - such as problems, tests, and treatments - aids in forming an
understanding of notes and provides a foundation for many downstream clinical
decision-making tasks. Historically, this task has been posed as a standard
named entity recognition (NER) sequence tagging problem, and solved with
feature-based methods using handengineered domain knowledge. Recent advances,
however, have demonstrated the efficacy of LSTM-based models for NER tasks,
including CCE. This work presents CliNER 2.0, a simple-to-install, open-source
tool for extracting concepts from clinical text. CliNER 2.0 uses a word- and
character- level LSTM model, and achieves state-of-the-art performance. For
ease of use, the tool also includes pre-trained models available for public
use.
Early Start Intention Detection of Cyclists Using Motion History Images and a Deep Residual Network
Authors: Stefan Zernetsch, Viktor Kress, Bernhard Sick, Konrad Doll
Date: 2018-03-06 00:00:00
Tasks: General Classification, image-classification, Image Classification

In this article, we present a novel approach to detect starting motions of
cyclists in real world traffic scenarios based on Motion History Images (MHIs).
The method uses a deep Convolutional Neural Network (CNN) with a residual
network architecture (ResNet), which is commonly used in image classification
and detection tasks. By combining MHIs with a ResNet classifier and performing
a frame by frame classification of the MHIs, we are able to detect starting
motions in image sequences. The detection is performed using a wide angle
stereo camera system at an urban intersection. We compare our algorithm to an
existing method to detect movement transitions of pedestrians that uses MHIs in
combination with a Histograms of Oriented Gradients (HOG) like descriptor and a
Support Vector Machine (SVM), which we adapted to cyclists. To train and
evaluate the methods a dataset containing MHIs of 394 cyclist starting motions
was created. The results show that both methods can be used to detect starting
motions of cyclists. Using the SVM approach, we were able to safely detect
starting motions 0.506 s on average after the bicycle starts moving with an
F1-score of 97.7%. The ResNet approach achieved an F1-score of 100% at an
average detection time of 0.144 s. The ResNet approach outperformed the SVM
approach in both robustness against false positive detections and detection
time.
Precise but Natural Specification for Robot Tasks
Authors: Ivan Gavran, Brendon Boldt, Eva Darulova, Rupak Majumdar
Date: 2018-03-06 00:00:00
Tasks: Task Planning

We present Flipper, a natural language interface for describing high-level
task specifications for robots that are compiled into robot actions. Flipper
starts with a formal core language for task planning that allows expressing
rich temporal specifications and uses a semantic parser to provide a natural
language interface. Flipper provides immediate visual feedback by executing an
automatically constructed plan of the task in a graphical user interface. This
allows the user to resolve potentially ambiguous interpretations. Flipper
extends itself via naturalization: its users can add definitions for
utterances, from which Flipper induces new rules and adds them to the core
language, gradually growing a more and more natural task specification
language. Flipper improves the naturalization by generalizing the definition
provided by users. Unlike other task-specification systems, Flipper enables
natural language interactions while maintaining the expressive power and formal
precision of a programming language. We show through an initial user study that
natural language interactions and generalization can considerably ease the
description of tasks. Moreover, over time, users employ more and more concepts
outside of the initial core language. Such extensions are available to the
Flipper community, and users can use concepts that others have defined.