pwc-archive/papers-with-abstracts huggingface dataset:
Mixture of Regression Experts in fMRI Encoding
Authors: Subba Reddy Oota, Adithya Avvaru, Naresh Manwani, Raju S. Bapi
Date: 2018-11-26 00:00:00
Tasks: Mixture-of-Experts, regression

fMRI semantic category understanding using linguistic encoding models attempt
to learn a forward mapping that relates stimuli to the corresponding brain
activation. Classical encoding models use linear multi-variate methods to
predict the brain activation (all voxels) given the stimulus. However, these
methods essentially assume multiple regions as one large uniform region or
several independent regions, ignoring connections among them. In this paper, we
present a mixture of experts-based model where a group of experts captures
brain activity patterns related to particular regions of interest (ROI) and
also show the discrimination across different experts. The model is trained
word stimuli encoded as 25-dimensional feature vectors as input and the
corresponding brain responses as output. Given a new word (25-dimensional
feature vector), it predicts the entire brain activation as the linear
combination of multiple experts brain activations. We argue that each expert
learns a certain region of brain activations corresponding to its category of
words, which solves the problem of identifying the regions with a simple
encoding model. We showcase that proposed mixture of experts-based model indeed
learns region-based experts to predict the brain activations with high spatial
accuracy.
DONUT: CTC-based Query-by-Example Keyword Spotting
Authors: Loren Lugosch, Samuel Myer, Vikrant Singh Tomar
Date: 2018-11-26 00:00:00
Tasks: Keyword Spotting

Keyword spotting--or wakeword detection--is an essential feature for
hands-free operation of modern voice-controlled devices. With such devices
becoming ubiquitous, users might want to choose a personalized custom wakeword.
In this work, we present DONUT, a CTC-based algorithm for online
query-by-example keyword spotting that enables custom wakeword detection. The
algorithm works by recording a small number of training examples from the user,
generating a set of label sequence hypotheses from these training examples, and
detecting the wakeword by aggregating the scores of all the hypotheses given a
new audio recording. Our method combines the generalization and
interpretability of CTC-based keyword spotting with the user-adaptation and
convenience of a conventional query-by-example system. DONUT has low
computational requirements and is well-suited for both learning and inference
on embedded systems without requiring private user data to be uploaded to the
cloud.
Automatic Induction of Neural Network Decision Tree Algorithms
Authors: Chapman Siu
Date: 2018-11-26 00:00:00

This work presents an approach to automatically induction for non-greedy
decision trees constructed from neural network architecture. This construction
can be used to transfer weights when growing or pruning a decision tree,
allowing non-greedy decision tree algorithms to automatically learn and adapt
to the ideal architecture. In this work, we examine the underpinning ideas
within ensemble modelling and Bayesian model averaging which allow our neural
network to asymptotically approach the ideal architecture through weights
transfer. Experimental results demonstrate that this approach improves models
over fixed set of hyperparameters for decision tree models and decision forest
models.
DynamicGEM: A Library for Dynamic Graph Embedding Methods
Authors: Palash Goyal, Sujit Rokka Chhetri, Ninareh Mehrabi, Emilio Ferrara, Arquimedes Canedo
Date: 2018-11-26 00:00:00
Tasks: Dynamic graph embedding, General Classification, Graph Embedding, Graph Reconstruction, Link Prediction, Node Classification

DynamicGEM is an open-source Python library for learning node representations
of dynamic graphs. It consists of state-of-the-art algorithms for defining
embeddings of nodes whose connections evolve over time. The library also
contains the evaluation framework for four downstream tasks on the network:
graph reconstruction, static and temporal link prediction, node classification,
and temporal visualization. We have implemented various metrics to evaluate the
state-of-the-art methods, and examples of evolving networks from various
domains. We have easy-to-use functions to call and evaluate the methods and
have extensive usage documentation. Furthermore, DynamicGEM provides a template
to add new algorithms with ease to facilitate further research on the topic.
Environments for Lifelong Reinforcement Learning
Authors: Khimya Khetarpal, Shagun Sodhani, Sarath Chandar, Doina Precup
Date: 2018-11-26 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

To achieve general artificial intelligence, reinforcement learning (RL)
agents should learn not only to optimize returns for one specific task but also
to constantly build more complex skills and scaffold their knowledge about the
world, without forgetting what has already been learned. In this paper, we
discuss the desired characteristics of environments that can support the
training and evaluation of lifelong reinforcement learning agents, review
existing environments from this perspective, and propose recommendations for
devising suitable environments in the future.
Optimization of Information-Seeking Dialogue Strategy for Argumentation-Based Dialogue System
Authors: Hisao Katsumi, Takuya Hiraoka, Koichiro Yoshino, Kazeto Yamamoto, Shota Motoura, Kunihiko Sadamasa, Satoshi Nakamura
Date: 2018-11-26 00:00:00
Tasks: Deep Reinforcement Learning, Reinforcement Learning

Argumentation-based dialogue systems, which can handle and exchange arguments
through dialogue, have been widely researched. It is required that these
systems have sufficient supporting information to argue their claims
rationally; however, the systems often do not have enough of such information
in realistic situations. One way to fill in the gap is acquiring such missing
information from dialogue partners (information-seeking dialogue). Existing
information-seeking dialogue systems are based on handcrafted dialogue
strategies that exhaustively examine missing information. However, the proposed
strategies are not specialized in collecting information for constructing
rational arguments. Moreover, the number of system's inquiry candidates grows
in accordance with the size of the argument set that the system deal with. In
this paper, we formalize the process of information-seeking dialogue as Markov
decision processes (MDPs) and apply deep reinforcement learning (DRL) for
automatically optimizing a dialogue strategy. By utilizing DRL, our dialogue
strategy can successfully minimize objective functions, the number of turns it
takes for our system to collect necessary information in a dialogue. We
conducted dialogue experiments using two datasets from different domains of
argumentative dialogue. Experimental results show that the proposed
formalization based on MDP works well, and the policy optimized by DRL
outperformed existing heuristic dialogue strategies.
MIST: Multiple Instance Spatial Transformer Network
Authors: Baptiste Angles, Yuhe Jin, Simon Kornblith, Andrea Tagliasacchi, Kwang Moo Yi
Date: 2018-11-26 00:00:00
Tasks: Anomaly Detection In Surveillance Videos, Image Reconstruction

We propose a deep network that can be trained to tackle image reconstruction and classification problems that involve detection of multiple object instances, without any supervision regarding their whereabouts. The network learns to extract the most significant top-K patches, and feeds these patches to a task-specific network -- e.g., auto-encoder or classifier -- to solve a domain specific problem. The challenge in training such a network is the non-differentiable top-K selection process. To address this issue, we lift the training optimization problem by treating the result of top-K selection as a slack variable, resulting in a simple, yet effective, multi-stage training. Our method is able to learn to detect recurrent structures in the training dataset by learning to reconstruct images. It can also learn to localize structures when only knowledge on the occurrence of the object is provided, and in doing so it outperforms the state-of-the-art.
IGNOR: Image-guided Neural Object Rendering
Authors: Justus Thies, Michael Zollhöfer, Christian Theobalt, Marc Stamminger, Matthias Nießner
Date: 2018-11-26 00:00:00
Tasks: Image Generation, Novel View Synthesis, Object

We propose a learned image-guided rendering technique that combines the benefits of image-based rendering and GAN-based image synthesis. The goal of our method is to generate photo-realistic re-renderings of reconstructed objects for virtual and augmented reality applications (e.g., virtual showrooms, virtual tours \& sightseeing, the digital inspection of historical artifacts). A core component of our work is the handling of view-dependent effects. Specifically, we directly train an object-specific deep neural network to synthesize the view-dependent appearance of an object. As input data we are using an RGB video of the object. This video is used to reconstruct a proxy geometry of the object via multi-view stereo. Based on this 3D proxy, the appearance of a captured view can be warped into a new target view as in classical image-based rendering. This warping assumes diffuse surfaces, in case of view-dependent effects, such as specular highlights, it leads to artifacts. To this end, we propose EffectsNet, a deep neural network that predicts view-dependent effects. Based on these estimations, we are able to convert observed images to diffuse images. These diffuse images can be projected into other views. In the target view, our pipeline reinserts the new view-dependent effects. To composite multiple reprojected images to a final output, we learn a composition network that outputs photo-realistic results. Using this image-guided approach, the network does not have to allocate capacity on ``remembering'' object appearance, instead it learns how to combine the appearance of captured images. We demonstrate the effectiveness of our approach both qualitatively and quantitatively on synthetic as well as on real data.
Learning View Priors for Single-view 3D Reconstruction
Authors: Hiroharu Kato, Tatsuya Harada
Date: 2018-11-26 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Reconstruction, Object, Single-View 3D Reconstruction

There is some ambiguity in the 3D shape of an object when the number of
observed views is small. Because of this ambiguity, although a 3D object
reconstructor can be trained using a single view or a few views per object,
reconstructed shapes only fit the observed views and appear incorrect from the
unobserved viewpoints. To reconstruct shapes that look reasonable from any
viewpoint, we propose to train a discriminator that learns prior knowledge
regarding possible views. The discriminator is trained to distinguish the
reconstructed views of the observed viewpoints from those of the unobserved
viewpoints. The reconstructor is trained to correct unobserved views by fooling
the discriminator. Our method outperforms current state-of-the-art methods on
both synthetic and natural image datasets; this validates the effectiveness of
our method.
Bilateral Adversarial Training: Towards Fast Training of More Robust Models Against Adversarial Attacks
Authors: Jianyu Wang, Haichao Zhang
Date: 2018-11-26 00:00:00
Proceeding: ICCV 2019 10

In this paper, we study fast training of adversarially robust models. From the analyses of the state-of-the-art defense method, i.e., the multi-step adversarial training, we hypothesize that the gradient magnitude links to the model robustness. Motivated by this, we propose to perturb both the image and the label during training, which we call Bilateral Adversarial Training (BAT). To generate the adversarial label, we derive an closed-form heuristic solution. To generate the adversarial image, we use one-step targeted attack with the target label being the most confusing class. In the experiment, we first show that random start and the most confusing target attack effectively prevent the label leaking and gradient masking problem. Then coupled with the adversarial label part, our model significantly improves the state-of-the-art results. For example, against PGD100 white-box attack with cross-entropy loss, on CIFAR10, we achieve 63.7\% versus 47.2\%; on SVHN, we achieve 59.1\% versus 42.1\%. At last, the experiment on the very (computationally) challenging ImageNet dataset further demonstrates the effectiveness of our fast method.
Learning Robust Representations for Automatic Target Recognition
Authors: Justin A. Goodwin, Olivia M. Brown, Taylor W. Killian, Sung-Hyun Son
Date: 2018-11-26 00:00:00
Tasks: General Classification, Robust classification

Radio frequency (RF) sensors are used alongside other sensing modalities to
provide rich representations of the world. Given the high variability of
complex-valued target responses, RF systems are susceptible to attacks masking
true target characteristics from accurate identification. In this work, we
evaluate different techniques for building robust classification architectures
exploiting learned physical structure in received synthetic aperture radar
signals of simulated 3D targets.
Combining High-Level Features of Raw Audio Waves and Mel-Spectrograms for Audio Tagging
Authors: Marcel Lederle, Benjamin Wilhelm
Date: 2018-11-26 00:00:00
Tasks: Audio Tagging, Data Augmentation, General Classification, Task 2

In this paper, we describe our contribution to Task 2 of the DCASE 2018 Audio
Challenge. While it has become ubiquitous to utilize an ensemble of machine
learning methods for classification tasks to obtain better predictive
performance, the majority of ensemble methods combine predictions rather than
learned features. We propose a single-model method that combines learned
high-level features computed from log-scaled mel-spectrograms and raw audio
data. These features are learned separately by two Convolutional Neural
Networks, one for each input type, and then combined by densely connected
layers within a single network. This relatively simple approach along with data
augmentation ranks among the best two percent in the Freesound General-Purpose
Audio Tagging Challenge on Kaggle.
Time-Aware and View-Aware Video Rendering for Unsupervised Representation Learning
Authors: Shruti Vyas, Yogesh S Rawat, Mubarak Shah
Date: 2018-11-26 00:00:00
Tasks: Representation Learning

The recent success in deep learning has lead to various effective
representation learning methods for videos. However, the current approaches for
video representation require large amount of human labeled datasets for
effective learning. We present an unsupervised representation learning
framework to encode scene dynamics in videos captured from multiple viewpoints.
The proposed framework has two main components: Representation Learning Network
(RL-NET), which learns a representation with the help of Blending Network
(BL-NET), and Video Rendering Network (VR-NET), which is used for video
synthesis. The framework takes as input video clips from different viewpoints
and time, learns an internal representation and uses this representation to
render a video clip from an arbitrary given viewpoint and time. The ability of
the proposed network to render video frames from arbitrary viewpoints and time
enable it to learn a meaningful and robust representation of the scene
dynamics. We demonstrate the effectiveness of the proposed method in rendering
view-aware as well as time-aware video clips on two different real-world
datasets including UCF-101 and NTU-RGB+D. To further validate the effectiveness
of the learned representation, we use it for the task of view-invariant
activity classification where we observe a significant improvement (~26%) in
the performance on NTU-RGB+D dataset compared to the existing state-of-the art
methods.
LSTA: Long Short-Term Attention for Egocentric Action Recognition
Authors: Swathikiran Sudhakaran, Sergio Escalera, Oswald Lanz
Date: 2018-11-26 00:00:00
Proceeding: CVPR 2019 6
Tasks: Action Recognition, Activity Recognition, Egocentric Activity Recognition, Temporal Action Localization

Egocentric activity recognition is one of the most challenging tasks in video
analysis. It requires a fine-grained discrimination of small objects and their
manipulation. While some methods base on strong supervision and attention
mechanisms, they are either annotation consuming or do not take spatio-temporal
patterns into account. In this paper we propose LSTA as a mechanism to focus on
features from spatial relevant parts while attention is being tracked smoothly
across the video sequence. We demonstrate the effectiveness of LSTA on
egocentric activity recognition with an end-to-end trainable two-stream
architecture, achieving state of the art performance on four standard
benchmarks.
Attentive Relational Networks for Mapping Images to Scene Graphs
Authors: Mengshi Qi, Weijian Li, Zhengyuan Yang, Yunhong Wang, Jiebo Luo
Date: 2018-11-26 00:00:00
Proceeding: CVPR 2019 6
Tasks: Graph Generation, Object, object-detection, Object Detection, Relation, Scene Graph Generation

Scene graph generation refers to the task of automatically mapping an image
into a semantic structural graph, which requires correctly labeling each
extracted object and their interaction relationships. Despite the recent
success in object detection using deep learning techniques, inferring complex
contextual relationships and structured graph representations from visual data
remains a challenging topic. In this study, we propose a novel Attentive
Relational Network that consists of two key modules with an object detection
backbone to approach this problem. The first module is a semantic
transformation module utilized to capture semantic embedded relation features,
by translating visual features and linguistic features into a common semantic
space. The other module is a graph self-attention module introduced to embed a
joint graph representation through assigning various importance weights to
neighboring nodes. Finally, accurate scene graphs are produced by the relation
inference module to recognize all entities and the corresponding relations. We
evaluate our proposed method on the widely-adopted Visual Genome Dataset, and
the results demonstrate the effectiveness and superiority of our model.
Sequence Alignment with Dirichlet Process Mixtures
Authors: Ieva Kazlauskaite, Ivan Ustyuzhaninov, Carl Henrik Ek, Neill D. F. Campbell
Date: 2018-11-26 00:00:00
Tasks: Gaussian Processes

We present a probabilistic model for unsupervised alignment of
high-dimensional time-warped sequences based on the Dirichlet Process Mixture
Model (DPMM). We follow the approach introduced in (Kazlauskaite, 2018) of
simultaneously representing each data sequence as a composition of a true
underlying function and a time-warping, both of which are modelled using
Gaussian processes (GPs) (Rasmussen, 2005), and aligning the underlying
functions using an unsupervised alignment method. In (Kazlauskaite, 2018) the
alignment is performed using the GP latent variable model (GP-LVM) (Lawrence,
2005) as a model of sequences, while our main contribution is extending this
approach to using DPMM, which allows us to align the sequences temporally and
cluster them at the same time. We show that the DPMM achieves competitive
results in comparison to the GP-LVM on synthetic and real-world data sets, and
discuss the different properties of the estimated underlying functions and the
time-warps favoured by these models.
Beyond "How may I help you?": Assisting Customer Service Agents with Proactive Responses
Authors: Mengting Wan, Xin Chen
Date: 2018-11-26 00:00:00

We study the problem of providing recommended responses to customer service
agents in live-chat dialogue systems. Smart-reply systems have been widely
applied in real-world applications (e.g. Gmail, LinkedIn Messaging), where most
of them can successfully recommend reactive responses. However, we observe a
major limitation of current methods is that they generally have difficulties in
suggesting proactive investigation act (e.g. "Do you perhaps have another
account with us?") due to the lack of long-term context information, which
indeed act as critical steps for customer service agents to collect information
and resolve customers' issues. Thus in this work, we propose an end-to-end
method with special focus on suggesting proactive investigative questions to
customer agents in Airbnb's customer service live-chat system. Effectiveness of
our proposed method can be validated through qualitative and quantitative
results.
Matching Features without Descriptors: Implicitly Matched Interest Points
Authors: Titus Cieslewski, Michael Bloesch, Davide Scaramuzza
Date: 2018-11-26 00:00:00
Tasks: Pose Estimation, valid

The extraction and matching of interest points is a prerequisite for many geometric computer vision problems. Traditionally, matching has been achieved by assigning descriptors to interest points and matching points that have similar descriptors. In this paper, we propose a method by which interest points are instead already implicitly matched at detection time. With this, descriptors do not need to be calculated, stored, communicated, or matched any more. This is achieved by a convolutional neural network with multiple output channels and can be thought of as a collection of a variety of detectors, each specialized to specific visual features. This paper describes how to design and train such a network in a way that results in successful relative pose estimation performance despite the limitation on interest point count. While the overall matching score is slightly lower than with traditional methods, the approach is descriptor free and thus enables localization systems with a significantly smaller memory footprint and multi-agent localization systems with lower bandwidth requirements. The network also outputs the confidence for a specific interest point resulting in a valid match. We evaluate performance relative to state-of-the-art alternatives.
Adversarial Video Compression Guided by Soft Edge Detection
Authors: Sungsoo Kim, Jin Soo Park, Christos G. Bampis, Jaeseong Lee, Mia K. Markey, Alexandros G. Dimakis, Alan C. Bovik
Date: 2018-11-26 00:00:00
Tasks: Decoder, Edge Detection, Video Compression

We propose a video compression framework using conditional Generative
Adversarial Networks (GANs). We rely on two encoders: one that deploys a
standard video codec and another which generates low-level maps via a pipeline
of down-sampling, a newly devised soft edge detector, and a novel lossless
compression scheme. For decoding, we use a standard video decoder as well as a
neural network based one, which is trained using a conditional GAN. Recent
"deep" approaches to video compression require multiple videos to pre-train
generative networks to conduct interpolation. In contrast to this prior work,
our scheme trains a generative decoder on pairs of a very limited number of key
frames taken from a single video and corresponding low-level maps. The trained
decoder produces reconstructed frames relying on a guidance of low-level maps,
without any interpolation. Experiments on a diverse set of 131 videos
demonstrate that our proposed GAN-based compression engine achieves much higher
quality reconstructions at very low bitrates than prevailing standard codecs
such as H.264 or HEVC.