pwc-archive/papers-with-abstracts huggingface dataset:
Trajectory-based Learning for Ball-in-Maze Games
Authors: Sujoy Paul, Jeroen van Baar
Date: 2018-11-28 00:00:00
Tasks: Deep Reinforcement Learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep Reinforcement Learning has shown tremendous success in solving several
games and tasks in robotics. However, unlike humans, it generally requires a
lot of training instances. Trajectories imitating to solve the task at hand can
help to increase sample-efficiency of deep RL methods. In this paper, we
present a simple approach to use such trajectories, applied to the challenging
Ball-in-Maze Games, recently introduced in the literature. We show that in
spite of not using human-generated trajectories and just using the simulator as
a model to generate a limited number of trajectories, we can get a speed-up of
about 2-3x in the learning process. We also discuss some challenges we observed
while using trajectory-based learning for very sparse reward functions.
Neural Sign Language Translation based on Human Keypoint Estimation
Authors: Sang-Ki Ko, Chang Jo Kim, Hyedong Jung, Choongsang Cho
Date: 2018-11-28 00:00:00
Tasks: Keypoint Estimation, Sign Language Translation, Translation

We propose a sign language translation system based on human keypoint estimation. It is well-known that many problems in the field of computer vision require a massive amount of dataset to train deep neural network models. The situation is even worse when it comes to the sign language translation problem as it is far more difficult to collect high-quality training data. In this paper, we introduce the KETI (short for Korea Electronics Technology Institute) sign language dataset which consists of 14,672 videos of high resolution and quality. Considering the fact that each country has a different and unique sign language, the KETI sign language dataset can be the starting line for further research on the Korean sign language translation. Using the KETI sign language dataset, we develop a neural network model for translating sign videos into natural language sentences by utilizing the human keypoints extracted from a face, hands, and body parts. The obtained human keypoint vector is normalized by the mean and standard deviation of the keypoints and used as input to our translation model based on the sequence-to-sequence architecture. As a result, we show that our approach is robust even when the size of the training data is not sufficient. Our translation model achieves 93.28% (55.28%, respectively) translation accuracy on the validation set (test set, respectively) for 105 sentences that can be used in emergency situations. We compare several types of our neural sign translation models based on different attention mechanisms in terms of classical metrics for measuring the translation performance.
Partial Evaluation of Logic Programs in Vector Spaces
Authors: Chiaki Sakama, Hien D. Nguyen, Taisuke Sato, Katsumi Inoue
Date: 2018-11-28 00:00:00

In this paper, we introduce methods of encoding propositional logic programs
in vector spaces. Interpretations are represented by vectors and programs are
represented by matrices. The least model of a definite program is computed by
multiplying an interpretation vector and a program matrix. To optimize
computation in vector spaces, we provide a method of partial evaluation of
programs using linear algebra. Partial evaluation is done by unfolding rules in
a program, and it is realized in a vector space by multiplying program
matrices. We perform experiments using randomly generated programs and show
that partial evaluation has potential for realizing efficient computation in
huge scale of programs.
Beyond Pham's algorithm for joint diagonalization
Authors: Pierre Ablin, Jean-François Cardoso, Alexandre Gramfort
Date: 2018-11-28 00:00:00

The approximate joint diagonalization of a set of matrices consists in
finding a basis in which these matrices are as diagonal as possible. This
problem naturally appears in several statistical learning tasks such as blind
signal separation. We consider the diagonalization criterion studied in a
seminal paper by Pham (2001), and propose a new quasi-Newton method for its
optimization. Through numerical experiments on simulated and real datasets, we
show that the proposed method outper-forms Pham's algorithm. An open source
Python package is released.
ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Network
Authors: Sachin Mehta, Mohammad Rastegari, Linda Shapiro, Hannaneh Hajishirzi
Date: 2018-11-28 00:00:00
Proceeding: CVPR 2019 6
Tasks: General Classification, image-classification, Image Classification, Language Modeling, Language Modelling, Object, object-detection, Object Detection, Real-Time Object Detection, Real-Time Semantic Segmentation, Semantic Segmentation

We introduce a light-weight, power efficient, and general purpose
convolutional neural network, ESPNetv2, for modeling visual and sequential
data. Our network uses group point-wise and depth-wise dilated separable
convolutions to learn representations from a large effective receptive field
with fewer FLOPs and parameters. The performance of our network is evaluated on
four different tasks: (1) object classification, (2) semantic segmentation, (3)
object detection, and (4) language modeling. Experiments on these tasks,
including image classification on the ImageNet and language modeling on the
PenTree bank dataset, demonstrate the superior performance of our method over
the state-of-the-art methods. Our network outperforms ESPNet by 4-5% and has
2-4x fewer FLOPs on the PASCAL VOC and the Cityscapes dataset. Compared to
YOLOv2 on the MS-COCO object detection, ESPNetv2 delivers 4.4% higher accuracy
with 6x fewer FLOPs. Our experiments show that ESPNetv2 is much more power
efficient than existing state-of-the-art efficient methods including
ShuffleNets and MobileNets. Our code is open-source and available at
https://github.com/sacmehta/ESPNetv2
Context-Aware Dialog Re-Ranking for Task-Oriented Dialog Systems
Authors: Junki Ohmura, Maxine Eskenazi
Date: 2018-11-28 00:00:00
Tasks: Re-Ranking, speech-recognition, Speech Recognition

Dialog response ranking is used to rank response candidates by considering
their relation to the dialog history. Although researchers have addressed this
concept for open-domain dialogs, little attention has been focused on
task-oriented dialogs. Furthermore, no previous studies have analyzed whether
response ranking can improve the performance of existing dialog systems in real
human-computer dialogs with speech recognition errors. In this paper, we
propose a context-aware dialog response re-ranking system. Our system reranks
responses in two steps: (1) it calculates matching scores for each candidate
response and the current dialog context; (2) it combines the matching scores
and a probability distribution of the candidates from an existing dialog system
for response re-ranking. By using neural word embedding-based models and
handcrafted or logistic regression-based ensemble models, we have improved the
performance of a recently proposed end-to-end task-oriented dialog system on
real dialogs with speech recognition errors.
Deep Collective Matrix Factorization for Augmented Multi-View Learning
Authors: Ragunathan Mariappan, Vaibhav Rajan
Date: 2018-11-28 00:00:00
Tasks: Bayesian Optimization, Matrix Completion, MULTI-VIEW LEARNING

Learning by integrating multiple heterogeneous data sources is a common
requirement in many tasks. Collective Matrix Factorization (CMF) is a technique
to learn shared latent representations from arbitrary collections of matrices.
It can be used to simultaneously complete one or more matrices, for predicting
the unknown entries. Classical CMF methods assume linearity in the interaction
of latent factors which can be restrictive and fails to capture complex
non-linear interactions. In this paper, we develop the first deep-learning
based method, called dCMF, for unsupervised learning of multiple shared
representations, that can model such non-linear interactions, from an arbitrary
collection of matrices. We address optimization challenges that arise due to
dependencies between shared representations through Multi-Task Bayesian
Optimization and design an acquisition function adapted for collective learning
of hyperparameters. Our experiments show that dCMF significantly outperforms
previous CMF algorithms in integrating heterogeneous data for predictive
modeling. Further, on two tasks - recommendation and prediction of gene-disease
association - dCMF outperforms state-of-the-art matrix completion algorithms
that can utilize auxiliary sources of information.
Semi-supervised learning with Bidirectional GANs
Authors: Maciej Zamorski, Maciej Zięba
Date: 2018-11-28 00:00:00
Tasks: General Classification, Image Retrieval, Retrieval, Triplet

In this work we introduce a novel approach to train Bidirectional Generative
Adversarial Model (BiGAN) in a semi-supervised manner. The presented method
utilizes triplet loss function as an additional component of the objective
function used to train discriminative data representation in the latent space
of the BiGAN model. This representation can be further used as a seed for
generating artificial images, but also as a good feature embedding for
classification and image retrieval tasks. We evaluate the quality of the
proposed method in the two mentioned challenging tasks using two benchmark
datasets: CIFAR10 and SVHN.
MeshNet: Mesh Neural Network for 3D Shape Representation
Authors: Yutong Feng, Yifan Feng, Haoxuan You, Xibin Zhao, Yue Gao
Date: 2018-11-28 00:00:00
Tasks: 3D Shape Classification, 3D Shape Representation, General Classification, Retrieval

Mesh is an important and powerful type of data for 3D shapes and widely
studied in the field of computer vision and computer graphics. Regarding the
task of 3D shape representation, there have been extensive research efforts
concentrating on how to represent 3D shapes well using volumetric grid,
multi-view and point cloud. However, there is little effort on using mesh data
in recent years, due to the complexity and irregularity of mesh data. In this
paper, we propose a mesh neural network, named MeshNet, to learn 3D shape
representation from mesh data. In this method, face-unit and feature splitting
are introduced, and a general architecture with available and effective blocks
are proposed. In this way, MeshNet is able to solve the complexity and
irregularity problem of mesh and conduct 3D shape representation well. We have
applied the proposed MeshNet method in the applications of 3D shape
classification and retrieval. Experimental results and comparisons with the
state-of-the-art methods demonstrate that the proposed MeshNet can achieve
satisfying 3D shape classification and retrieval performance, which indicates
the effectiveness of the proposed method on 3D shape representation.
Mixture Martingales Revisited with Applications to Sequential Tests and Confidence Intervals
Authors: Emilie Kaufmann, Wouter Koolen
Date: 2018-11-28 00:00:00
Tasks: valid

This paper presents new deviation inequalities that are valid uniformly in time under adaptive sampling in a multi-armed bandit model. The deviations are measured using the Kullback-Leibler divergence in a given one-dimensional exponential family, and may take into account several arms at a time. They are obtained by constructing for each arm a mixture martingale based on a hierarchical prior, and by multiplying those martingales. Our deviation inequalities allow us to analyze stopping rules based on generalized likelihood ratios for a large class of sequential identification problems, and to construct tight confidence intervals for some functions of the means of the arms.
Spectral Feature Transformation for Person Re-identification
Authors: Chuanchen Luo, Yuntao Chen, Naiyan Wang, Zhao-Xiang Zhang
Date: 2018-11-28 00:00:00
Proceeding: ICCV 2019 10
Tasks: Clustering, Deep Learning, Person Re-Identification

With the surge of deep learning techniques, the field of person
re-identification has witnessed rapid progress in recent years. Deep learning
based methods focus on learning a feature space where samples are clustered
compactly according to their corresponding identities. Most existing methods
rely on powerful CNNs to transform the samples individually. In contrast, we
propose to consider the sample relations in the transformation. To achieve this
goal, we incorporate spectral clustering technique into CNN. We derive a novel
module named Spectral Feature Transformation and seamlessly integrate it into
existing CNN pipeline with negligible cost,which makes our method enjoy the
best of two worlds. Empirical studies show that the proposed approach
outperforms previous state-of-the-art methods on four public benchmarks by a
considerable margin without bells and whistles.
Adversarial Machine Learning And Speech Emotion Recognition: Utilizing Generative Adversarial Networks For Robustness
Authors: Siddique Latif, Rajib Rana, Junaid Qadir
Date: 2018-11-28 00:00:00
Tasks: Adversarial Attack, BIG-bench Machine Learning, Emotion Recognition, Generative Adversarial Network, Speech Emotion Recognition

Deep learning has undoubtedly offered tremendous improvements in the
performance of state-of-the-art speech emotion recognition (SER) systems.
However, recent research on adversarial examples poses enormous challenges on
the robustness of SER systems by showing the susceptibility of deep neural
networks to adversarial examples as they rely only on small and imperceptible
perturbations. In this study, we evaluate how adversarial examples can be used
to attack SER systems and propose the first black-box adversarial attack on SER
systems. We also explore potential defenses including adversarial training and
generative adversarial network (GAN) to enhance robustness. Experimental
evaluations suggest various interesting aspects of the effective utilization of
adversarial examples useful for achieving robustness for SER systems opening up
opportunities for researchers to further innovate in this space.
FADL:Federated-Autonomous Deep Learning for Distributed Electronic Health Record
Authors: Dianbo Liu, Timothy Miller, Raheel Sayeed, Kenneth D. Mandl
Date: 2018-11-28 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Federated Learning

Electronic health record (EHR) data is collected by individual institutions
and often stored across locations in silos. Getting access to these data is
difficult and slow due to security, privacy, regulatory, and operational
issues. We show, using ICU data from 58 different hospitals, that machine
learning models to predict patient mortality can be trained efficiently without
moving health data out of their silos using a distributed machine learning
strategy. We propose a new method, called Federated-Autonomous Deep Learning
(FADL) that trains part of the model using all data sources in a distributed
manner and other parts using data from specific data sources. We observed that
FADL outperforms traditional federated learning strategy and conclude that
balance between global and local training is an important factor to consider
when design distributed machine learning methods , especially in healthcare.
DeepMapping: Unsupervised Map Estimation From Multiple Point Clouds
Authors: Li Ding, Chen Feng
Date: 2018-11-28 00:00:00
Proceeding: CVPR 2019 6
Tasks: Point Cloud Registration

We propose DeepMapping, a novel registration framework using deep neural
networks (DNNs) as auxiliary functions to align multiple point clouds from
scratch to a globally consistent frame. We use DNNs to model the highly
non-convex mapping process that traditionally involves hand-crafted data
association, sensor pose initialization, and global refinement. Our key novelty
is that "training" these DNNs with properly defined unsupervised losses is
equivalent to solving the underlying registration problem, but less sensitive
to good initialization than ICP. Our framework contains two DNNs: a
localization network that estimates the poses for input point clouds, and a map
network that models the scene structure by estimating the occupancy status of
global coordinates. This allows us to convert the registration problem to a
binary occupancy classification, which can be solved efficiently using
gradient-based optimization. We further show that DeepMapping can be readily
extended to address the problem of Lidar SLAM by imposing geometric constraints
between consecutive point clouds. Experiments are conducted on both simulated
and real datasets. Qualitative and quantitative comparisons demonstrate that
DeepMapping often enables more robust and accurate global registration of
multiple point clouds than existing techniques. Our code is available at
https://ai4ce.github.io/DeepMapping/.
Composable Probabilistic Inference Networks Using MRAM-based Stochastic Neurons
Authors: Ramtin Zand, Kerem Y. Camsari, Supriyo Datta, Ronald F. DeMara
Date: 2018-11-28 00:00:00

Magnetoresistive random access memory (MRAM) technologies with thermally
unstable nanomagnets are leveraged to develop an intrinsic stochastic neuron as
a building block for restricted Boltzmann machines (RBMs) to form deep belief
networks (DBNs). The embedded MRAM-based neuron is modeled using precise
physics equations. The simulation results exhibit the desired sigmoidal
relation between the input voltages and probability of the output state. A
probabilistic inference network simulator (PIN-Sim) is developed to realize a
circuit-level model of an RBM utilizing resistive crossbar arrays along with
differential amplifiers to implement the positive and negative weight values.
The PIN-Sim is composed of five main blocks to train a DBN, evaluate its
accuracy, and measure its power consumption. The MNIST dataset is leveraged to
investigate the energy and accuracy tradeoffs of seven distinct network
topologies in SPICE using the 14nm HP-FinFET technology library with the
nominal voltage of 0.8V, in which an MRAM-based neuron is used as the
activation function. The software and hardware level simulations indicate that
a $784\times200\times10$ topology can achieve less than 5% error rates with
$\sim400 pJ$ energy consumption. The error rates can be reduced to 2.5% by
using a $784\times500\times500\times500\times10$ DBN at the cost of
$\sim10\times$ higher energy consumption and significant area overhead.
Finally, the effects of specific hardware-level parameters on power dissipation
and accuracy tradeoffs are identified via the developed PIN-Sim framework.
Image Generation from Layout
Authors: Bo Zhao, Lili Meng, Weidong Yin, Leonid Sigal
Date: 2018-11-28 00:00:00
Proceeding: CVPR 2019 6
Tasks: Layout-to-Image Generation, Object

Despite significant recent progress on generative models, controlled generation of images depicting multiple and complex object layouts is still a difficult problem. Among the core challenges are the diversity of appearance a given object may possess and, as a result, exponential set of images consistent with a specified layout. To address these challenges, we propose a novel approach for layout-based image generation; we call it Layout2Im. Given the coarse spatial layout (bounding boxes + object categories), our model can generate a set of realistic images which have the correct objects in the desired locations. The representation of each object is disentangled into a specified/certain part (category) and an unspecified/uncertain part (appearance). The category is encoded using a word embedding and the appearance is distilled into a low-dimensional vector sampled from a normal distribution. Individual object representations are composed together using convolutional LSTM, to obtain an encoding of the complete layout, and then decoded to an image. Several loss terms are introduced to encourage accurate and diverse generation. The proposed Layout2Im model significantly outperforms the previous state of the art, boosting the best reported inception score by 24.66% and 28.57% on the very challenging COCO-Stuff and Visual Genome datasets, respectively. Extensive experiments also demonstrate our method's ability to generate complex and diverse images with multiple objects.
Self-Supervised Spatiotemporal Feature Learning via Video Rotation Prediction
Authors: Longlong Jing, Xiaodong Yang, Jingen Liu, YingLi Tian
Date: 2018-11-28 00:00:00
Tasks: Action Recognition, Prediction, Self-Supervised Action Recognition, Temporal Action Localization, Video Understanding

The success of deep neural networks generally requires a vast amount of
training data to be labeled, which is expensive and unfeasible in scale,
especially for video collections. To alleviate this problem, in this paper, we
propose 3DRotNet: a fully self-supervised approach to learn spatiotemporal
features from unlabeled videos. A set of rotations are applied to all videos,
and a pretext task is defined as prediction of these rotations. When
accomplishing this task, 3DRotNet is actually trained to understand the
semantic concepts and motions in videos. In other words, it learns a
spatiotemporal video representation, which can be transferred to improve video
understanding tasks in small datasets. Our extensive experiments successfully
demonstrate the effectiveness of the proposed framework on action recognition,
leading to significant improvements over the state-of-the-art self-supervised
methods. With the self-supervised pre-trained 3DRotNet from large datasets, the
recognition accuracy is boosted up by 20.4% on UCF101 and 16.7% on HMDB51
respectively, compared to the models trained from scratch.
Instance-level Sketch-based Retrieval by Deep Triplet Classification Siamese Network
Authors: Peng Lu, Hangyu Lin, Yanwei Fu, Shaogang Gong, Yu-Gang Jiang, xiangyang xue
Date: 2018-11-28 00:00:00
Tasks: General Classification, Image Retrieval, Retrieval, Sketch-Based Image Retrieval, Sketch Recognition, Triplet

Sketch has been employed as an effective communicative tool to express the abstract and intuitive meanings of object. Recognizing the free-hand sketch drawing is extremely useful in many real-world applications. While content-based sketch recognition has been studied for several decades, the instance-level Sketch-Based Image Retrieval (SBIR) tasks have attracted significant research attention recently. The existing datasets such as QMUL-Chair and QMUL-Shoe, focus on the retrieval tasks of chairs and shoes. However, there are several key limitations in previous instance-level SBIR works. The state-of-the-art works have to heavily rely on the pre-training process, quality of edge maps, multi-cropping testing strategy, and augmenting sketch images. To efficiently solve the instance-level SBIR, we propose a new Deep Triplet Classification Siamese Network (DeepTCNet) which employs DenseNet-169 as the basic feature extractor and is optimized by the triplet loss and classification loss. Critically, our proposed DeepTCNet can break the limitations existed in previous works. The extensive experiments on five benchmark sketch datasets validate the effectiveness of the proposed model. Additionally, to study the tasks of sketch-based hairstyle retrieval, this paper contributes a new instance-level photo-sketch dataset - Hairstyle Photo-Sketch dataset, which is composed of 3600 sketches and photos, and 2400 sketch-photo pairs.