pwc-archive/papers-with-abstracts huggingface dataset:
Folded Recurrent Neural Networks for Future Video Prediction
Authors: Marc Oliu, Javier Selva, Sergio Escalera
Date: 2017-12-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Decoder, Prediction, Specificity, Video Prediction

Future video prediction is an ill-posed Computer Vision problem that recently
received much attention. Its main challenges are the high variability in video
content, the propagation of errors through time, and the non-specificity of the
future frames: given a sequence of past frames there is a continuous
distribution of possible futures. This work introduces bijective Gated
Recurrent Units, a double mapping between the input and output of a GRU layer.
This allows for recurrent auto-encoders with state sharing between encoder and
decoder, stratifying the sequence representation and helping to prevent
capacity problems. We show how with this topology only the encoder or decoder
needs to be applied for input encoding and prediction, respectively. This
reduces the computational cost and avoids re-encoding the predictions when
generating a sequence of frames, mitigating the propagation of errors.
Furthermore, it is possible to remove layers from an already trained model,
giving an insight to the role performed by each layer and making the model more
explainable. We evaluate our approach on three video datasets, outperforming
state of the art prediction results on MMNIST and UCF101, and obtaining
competitive results on KTH with 2 and 3 times less memory usage and
computational cost than the best scored approach.
Land cover mapping at very high resolution with rotation equivariant CNNs: towards small yet accurate models
Authors: Diego Marcos, Michele Volpi, Benjamin Kellenberger, Devis Tuia
Date: 2018-03-16 00:00:00

In remote sensing images, the absolute orientation of objects is arbitrary.
Depending on an object's orientation and on a sensor's flight path, objects of
the same semantic class can be observed in different orientations in the same
image. Equivariance to rotation, in this context understood as responding with
a rotated semantic label map when subject to a rotation of the input image, is
therefore a very desirable feature, in particular for high capacity models,
such as Convolutional Neural Networks (CNNs). If rotation equivariance is
encoded in the network, the model is confronted with a simpler task and does
not need to learn specific (and redundant) weights to address rotated versions
of the same object class. In this work we propose a CNN architecture called
Rotation Equivariant Vector Field Network (RotEqNet) to encode rotation
equivariance in the network itself. By using rotating convolutions as building
blocks and passing only the the values corresponding to the maximally
activating orientation throughout the network in the form of orientation
encoding vector fields, RotEqNet treats rotated versions of the same object
with the same filter bank and therefore achieves state-of-the-art performances
even when using very small architectures trained from scratch. We test RotEqNet
in two challenging sub-decimeter resolution semantic labeling problems, and
show that we can perform better than a standard CNN while requiring one order
of magnitude less parameters.
Semantic Augmented Reality Environment with Material-Aware Physical Interactions
Authors: Long Chen, Karl Francis, Wen Tang
Date: 2017-08-03 00:00:00
Tasks: Scene Understanding

In Augmented Reality (AR) environment, realistic interactions between the
virtual and real objects play a crucial role in user experience. Much of recent
advances in AR has been largely focused on developing geometry-aware
environment, but little has been done in dealing with interactions at the
semantic level. High-level scene understanding and semantic descriptions in AR
would allow effective design of complex applications and enhanced user
experience. In this paper, we present a novel approach and a prototype system
that enables the deeper understanding of semantic properties of the real world
environment, so that realistic physical interactions between the real and the
virtual objects can be generated. A material-aware AR environment has been
created based on the deep material learning using a fully convolutional network
(FCN). The state-of-the-art dense Simultaneous Localisation and Mapping (SLAM)
has been used for the semantic mapping. Together with efficient accelerated 3D
ray casting, natural and realistic physical interactions are generated for
interactive AR games. Our approach has significant impact on the future
development of advanced AR systems and applications.
Coordinating users of shared facilities via data-driven predictive assistants and game theory
Authors: Philipp Geiger, Michel Besserve, Justus Winkelmann, Claudius Proissl, Bernhard Schölkopf
Date: 2018-03-16 00:00:00
Tasks: Time Series Analysis

We study data-driven assistants that provide congestion forecasts to users of shared facilities (roads, cafeterias, etc.), to support coordination between them, and increase efficiency of such collective systems. Key questions are: (1) when and how much can (accurate) predictions help for coordination, and (2) which assistant algorithms reach optimal predictions? First we lay conceptual ground for this setting where user preferences are a priori unknown and predictions influence outcomes. Addressing (1), we establish conditions under which self-fulfilling prophecies, i.e., "perfect" (probabilistic) predictions of what will happen, solve the coordination problem in the game-theoretic sense of selecting a Bayesian Nash equilibrium (BNE). Next we prove that such prophecies exist even in large-scale settings where only aggregated statistics about users are available. This entails a new (nonatomic) BNE existence result. Addressing (2), we propose two assistant algorithms that sequentially learn from users' reactions, together with optimality/convergence guarantees. We validate one of them in a large real-world experiment.
A Multi-Scheme Ensemble Using Coopetitive Soft-Gating With Application to Power Forecasting for Renewable Energy Generation
Authors: André Gensler, Bernhard Sick
Date: 2018-03-16 00:00:00
Tasks: Weather Forecasting

In this article, we propose a novel ensemble technique with a multi-scheme
weighting based on a technique called coopetitive soft gating. This technique
combines both, ensemble member competition and cooperation, in order to
maximize the overall forecasting accuracy of the ensemble. The proposed
algorithm combines the ideas of multiple ensemble paradigms (power forecasting
model ensemble, weather forecasting model ensemble, and lagged ensemble) in a
hierarchical structure. The technique is designed to be used in a flexible
manner on single and multiple weather forecasting models, and for a variety of
lead times. We compare the technique to other power forecasting models and
ensemble techniques with a flexible number of weather forecasting models, which
can have the same, or varying forecasting horizons. It is shown that the model
is able to outperform those models on a number of publicly available data sets.
The article closes with a discussion of properties of the proposed model which
are relevant in its application.
A Read-Write Memory Network for Movie Story Understanding
Authors: Seil Na, Sang-ho Lee, Ji-Sung Kim, Gunhee Kim
Date: 2017-09-27 00:00:00
Proceeding: ICCV 2017 10
Tasks: Video Story QA

We propose a novel memory network model named Read-Write Memory Network
(RWMN) to perform question and answering tasks for large-scale, multimodal
movie story understanding. The key focus of our RWMN model is to design the
read network and the write network that consist of multiple convolutional
layers, which enable memory read and write operations to have high capacity and
flexibility. While existing memory-augmented network models treat each memory
slot as an independent block, our use of multi-layered CNNs allows the model to
read and write sequential memory cells as chunks, which is more reasonable to
represent a sequential story because adjacent memory blocks often have strong
correlations. For evaluation, we apply our model to all the six tasks of the
MovieQA benchmark, and achieve the best accuracies on several tasks, especially
on the visual QA task. Our model shows a potential to better understand not
only the content in the story, but also more abstract information, such as
relationships between characters and the reasons for their actions.
Complex-YOLO: Real-time 3D Object Detection on Point Clouds
Authors: Martin Simon, Stefan Milz, Karl Amende, Horst-Michael Gross
Date: 2018-03-16 00:00:00
Tasks: 3D Object Detection, Autonomous Driving, Motion Planning, Object, object-detection, Object Detection, Region Proposal, regression

Lidar based 3D object detection is inevitable for autonomous driving, because
it directly links to environmental understanding and therefore builds the base
for prediction and motion planning. The capacity of inferencing highly sparse
3D data in real-time is an ill-posed problem for lots of other application
areas besides automated vehicles, e.g. augmented reality, personal robotics or
industrial automation. We introduce Complex-YOLO, a state of the art real-time
3D object detection network on point clouds only. In this work, we describe a
network that expands YOLOv2, a fast 2D standard object detector for RGB images,
by a specific complex regression strategy to estimate multi-class 3D boxes in
Cartesian space. Thus, we propose a specific Euler-Region-Proposal Network
(E-RPN) to estimate the pose of the object by adding an imaginary and a real
fraction to the regression network. This ends up in a closed complex space and
avoids singularities, which occur by single angle estimations. The E-RPN
supports to generalize well during training. Our experiments on the KITTI
benchmark suite show that we outperform current leading methods for 3D object
detection specifically in terms of efficiency. We achieve state of the art
results for cars, pedestrians and cyclists by being more than five times faster
than the fastest competitor. Further, our model is capable of estimating all
eight KITTI-classes, including Vans, Trucks or sitting pedestrians
simultaneously with high accuracy.
Triplet-Center Loss for Multi-View 3D Object Retrieval
Authors: Xinwei He, Yang Zhou, Zhichao Zhou, Song Bai, Xiang Bai
Date: 2018-03-16 00:00:00
Proceeding: CVPR 2018 6
Tasks: 3D Object Recognition, 3D Object Retrieval, 3D Shape Classification, 3D Shape Retrieval, General Classification, Metric Learning, Object, Object Recognition, Retrieval, Triplet

Most existing 3D object recognition algorithms focus on leveraging the strong
discriminative power of deep learning models with softmax loss for the
classification of 3D data, while learning discriminative features with deep
metric learning for 3D object retrieval is more or less neglected. In the
paper, we study variants of deep metric learning losses for 3D object
retrieval, which did not receive enough attention from this area. First , two
kinds of representative losses, triplet loss and center loss, are introduced
which could learn more discriminative features than traditional classification
loss. Then, we propose a novel loss named triplet-center loss, which can
further enhance the discriminative power of the features. The proposed
triplet-center loss learns a center for each class and requires that the
distances between samples and centers from the same class are closer than those
from different classes. Extensive experimental results on two popular 3D object
retrieval benchmarks and two widely-adopted sketch-based 3D shape retrieval
benchmarks consistently demonstrate the effectiveness of our proposed loss, and
significant improvements have been achieved compared with the
state-of-the-arts.
The ApolloScape Open Dataset for Autonomous Driving and its Application
Authors: Xinyu Huang, Peng Wang, Xinjing Cheng, Dingfu Zhou, Qichuan Geng, Ruigang Yang
Date: 2018-03-16 00:00:00
Tasks: Autonomous Driving, Instance Segmentation, Multi-Task Learning, Semantic Segmentation, Sensor Fusion

Autonomous driving has attracted tremendous attention especially in the past few years. The key techniques for a self-driving car include solving tasks like 3D map construction, self-localization, parsing the driving road and understanding objects, which enable vehicles to reason and act. However, large scale data set for training and system evaluation is still a bottleneck for developing robust perception models. In this paper, we present the ApolloScape dataset [1] and its applications for autonomous driving. Compared with existing public datasets from real scenes, e.g. KITTI [2] or Cityscapes [3], ApolloScape contains much large and richer labelling including holistic semantic dense point cloud for each site, stereo, per-pixel semantic labelling, lanemark labelling, instance segmentation, 3D car instance, high accurate location for every frame in various driving videos from multiple sites, cities and daytimes. For each task, it contains at lease 15x larger amount of images than SOTA datasets. To label such a complete dataset, we develop various tools and algorithms specified for each task to accelerate the labelling process, such as 3D-2D segment labeling tools, active labelling in videos etc. Depend on ApolloScape, we are able to develop algorithms jointly consider the learning and inference of multiple tasks. In this paper, we provide a sensor fusion scheme integrating camera videos, consumer-grade motion sensors (GPS/IMU), and a 3D semantic map in order to achieve robust self-localization and semantic segmentation for autonomous driving. We show that practically, sensor fusion and joint learning of multiple tasks are beneficial to achieve a more robust and accurate system. We expect our dataset and proposed relevant algorithms can support and motivate researchers for further development of multi-sensor fusion and multi-task learning in the field of computer vision.
Some HCI Priorities for GDPR-Compliant Machine Learning
Authors: Michael Veale, Reuben Binns, Max Van Kleek
Date: 2018-03-16 00:00:00
Tasks: BIG-bench Machine Learning, Fairness

In this short paper, we consider the roles of HCI in enabling the better
governance of consequential machine learning systems using the rights and
obligations laid out in the recent 2016 EU General Data Protection Regulation
(GDPR)---a law which involves heavy interaction with people and systems.
Focussing on those areas that relate to algorithmic systems in society, we
propose roles for HCI in legal contexts in relation to fairness, bias and
discrimination; data protection by design; data protection impact assessments;
transparency and explanations; the mitigation and understanding of automation
bias; and the communication of envisaged consequences of processing.
Semantic Segmentation of Pathological Lung Tissue with Dilated Fully Convolutional Networks
Authors: Marios Anthimopoulos, Stergios Christodoulidis, Lukas Ebner, Thomas Geiser, Andreas Christe, Stavroula Mougiakakou
Date: 2018-03-16 00:00:00
Tasks: Diagnostic, Semantic Segmentation

Early and accurate diagnosis of interstitial lung diseases (ILDs) is crucial
for making treatment decisions, but can be challenging even for experienced
radiologists. The diagnostic procedure is based on the detection and
recognition of the different ILD pathologies in thoracic CT scans, yet their
manifestation often appears similar. In this study, we propose the use of a
deep purely convolutional neural network for the semantic segmentation of ILD
patterns, as the basic component of a computer aided diagnosis (CAD) system for
ILDs. The proposed CNN, which consists of convolutional layers with dilated
filters, takes as input a lung CT image of arbitrary size and outputs the
corresponding label map. We trained and tested the network on a dataset of 172
sparsely annotated CT scans, within a cross-validation scheme. The training was
performed in an end-to-end and semi-supervised fashion, utilizing both labeled
and non-labeled image regions. The experimental results show significant
performance improvement with respect to the state of the art.
Quantum spectral analysis: frequency in time, with applications to signal and image processing
Authors: Mario Mastriani
Date: 2016-10-11 00:00:00
Tasks: Denoising, Edge Detection

A quantum time-dependent spectrum analysis, or simply, quantum spectral analysis (QSA) is presented in this work, and it is based on Schrodinger equation, which is a partial differential equation that describes how the quantum state of a non-relativistic physical system changes with time. In classic world is named frequency in time (FIT), which is presented here in opposition and as a complement of traditional spectral analysis frequency-dependent based on Fourier theory. Besides, FIT is a metric, which assesses the impact of the flanks of a signal on its frequency spectrum, which is not taken into account by Fourier theory and even less in real time. Even more, and unlike all derived tools from Fourier Theory (i.e., continuous, discrete, fast, short-time, fractional and quantum Fourier Transform, as well as, Gabor) FIT has the following advantages: a) compact support with excellent energy output treatment, b) low computational cost, O(N) for signals and O(N2) for images, c) it does not have phase uncertainties (indeterminate phase for magnitude = 0) as Discrete and Fast Fourier Transform (DFT, FFT, respectively), d) among others. In fact, FIT constitutes one side of a triangle (which from now on is closed) and it consists of the original signal in time, spectral analysis based on Fourier Theory and FIT. Thus a toolbox is completed, which it is essential for all applications of Digital Signal Processing (DSP) and Digital Image Processing (DIP); and, even, in the latter, FIT allows edge detection (which is called flank detection in case of signals), denoising, despeckling, compression, and superresolution of still images. Such applications include signals intelligence and imagery intelligence. On the other hand, we will present other DIP tools, which are also derived from the Schrodinger equation.
Natural and Effective Obfuscation by Head Inpainting
Authors: Qianru Sun, Liqian Ma, Seong Joon Oh, Luc van Gool, Bernt Schiele, Mario Fritz
Date: 2017-11-24 00:00:00
Proceeding: CVPR 2018 6

As more and more personal photos are shared online, being able to obfuscate
identities in such photos is becoming a necessity for privacy protection.
People have largely resorted to blacking out or blurring head regions, but they
result in poor user experience while being surprisingly ineffective against
state of the art person recognizers. In this work, we propose a novel head
inpainting obfuscation technique. Generating a realistic head inpainting in
social media photos is challenging because subjects appear in diverse
activities and head orientations. We thus split the task into two sub-tasks:
(1) facial landmark generation from image context (e.g. body pose) for seamless
hypothesis of sensible head pose, and (2) facial landmark conditioned head
inpainting. We verify that our inpainting method generates realistic person
images, while achieving superior obfuscation performance against automatic
person recognizers.
Cluster-Seeking James-Stein Estimators
Authors: K. Pavan Srinath, Ramji Venkataramanan
Date: 2016-02-01 00:00:00

This paper considers the problem of estimating a high-dimensional vector of
parameters $\boldsymbol{\theta} \in \mathbb{R}^n$ from a noisy observation. The
noise vector is i.i.d. Gaussian with known variance. For a squared-error loss
function, the James-Stein (JS) estimator is known to dominate the simple
maximum-likelihood (ML) estimator when the dimension $n$ exceeds two. The
JS-estimator shrinks the observed vector towards the origin, and the risk
reduction over the ML-estimator is greatest for $\boldsymbol{\theta}$ that lie
close to the origin. JS-estimators can be generalized to shrink the data
towards any target subspace. Such estimators also dominate the ML-estimator,
but the risk reduction is significant only when $\boldsymbol{\theta}$ lies
close to the subspace. This leads to the question: in the absence of prior
information about $\boldsymbol{\theta}$, how do we design estimators that give
significant risk reduction over the ML-estimator for a wide range of
$\boldsymbol{\theta}$?
  In this paper, we propose shrinkage estimators that attempt to infer the
structure of $\boldsymbol{\theta}$ from the observed data in order to construct
a good attracting subspace. In particular, the components of the observed
vector are separated into clusters, and the elements in each cluster shrunk
towards a common attractor. The number of clusters and the attractor for each
cluster are determined from the observed vector. We provide concentration
results for the squared-error loss and convergence results for the risk of the
proposed estimators. The results show that the estimators give significant risk
reduction over the ML-estimator for a wide range of $\boldsymbol{\theta}$,
particularly for large $n$. Simulation results are provided to support the
theoretical claims.
Super-resolution of spatiotemporal event-stream image captured by the asynchronous temporal contrast vision sensor
Authors: Hongmin Li, Guoqi Li, Hanchao Liu, Luping Shi
Date: 2018-02-07 00:00:00
Tasks: Event-based vision, Super-Resolution

Super-resolution (SR) is a useful technology to generate a high-resolution
(HR) visual output from the low-resolution (LR) visual inputs overcoming the
physical limitations of the cameras. However, SR has not been applied to
enhance the resolution of spatiotemporal event-stream images captured by the
frame-free dynamic vision sensors (DVSs). SR of event-stream image is
fundamentally different from existing frame-based schemes since basically each
pixel value of DVS images is an event sequence. In this work, a two-stage
scheme is proposed to solve the SR problem of the spatiotemporal event-stream
image. We use a nonhomogeneous Poisson point process to model the event
sequence, and sample the events of each pixel by simulating a nonhomogeneous
Poisson process according to the specified event number and rate function.
Firstly, the event number of each pixel of the HR DVS image is determined with
a sparse signal representation based method to obtain the HR event-count map
from that of the LR DVS recording. The rate function over time line of the
point process of each HR pixel is computed using a spatiotemporal filter on the
corresponding LR neighbor pixels. Secondly, the event sequence of each new
pixel is generated with a thinning based event sampling algorithm. Two metrics
are proposed to assess the event-stream SR results. The proposed method is
demonstrated through obtaining HR event-stream images from a series of DVS
recordings with the proposed method. Results show that the upscaled HR event
streams has perceptually higher spatial texture detail than the LR DVS images.
Besides, the temporal properties of the upscaled HR event streams match that of
the original input very well. This work enables many potential applications of
event-based vision.
Object Captioning and Retrieval with Natural Language
Authors: Anh Nguyen, Thanh-Toan Do, Ian Reid, Darwin G. Caldwell, Nikos G. Tsagarakis
Date: 2018-03-16 00:00:00
Tasks: Object, Retrieval

We address the problem of jointly learning vision and language to understand
the object in a fine-grained manner. The key idea of our approach is the use of
object descriptions to provide the detailed understanding of an object. Based
on this idea, we propose two new architectures to solve two related problems:
object captioning and natural language-based object retrieval. The goal of the
object captioning task is to simultaneously detect the object and generate its
associated description, while in the object retrieval task, the goal is to
localize an object given an input query. We demonstrate that both problems can
be solved effectively using hybrid end-to-end CNN-LSTM networks. The
experimental results on our new challenging dataset show that our methods
outperform recent methods by a fair margin, while providing a detailed
understanding of the object and having fast inference time. The source code
will be made available.