pwc-archive/papers-with-abstracts huggingface dataset:
Flexible Attributed Network Embedding
Authors: Enya Shen, Zhidong Cao, Changqing Zou, Jian-Min Wang
Date: 2018-11-27 00:00:00
Tasks: General Classification, Network Embedding, Representation Learning

Network embedding aims to find a way to encode network by learning an
embedding vector for each node in the network. The network often has property
information which is highly informative with respect to the node's position and
role in the network. Most network embedding methods fail to utilize this
information during network representation learning. In this paper, we propose a
novel framework, FANE, to integrate structure and property information in the
network embedding process. In FANE, we design a network to unify heterogeneity
of the two information sources, and define a new random walking strategy to
leverage property information and make the two information compensate. FANE is
conceptually simple and empirically powerful. It improves over the
state-of-the-art methods on Cora dataset classification task by over 5%, more
than 10% on WebKB dataset classification task. Experiments also show that the
results improve more than the state-of-the-art methods as increasing training
size. Moreover, qualitative visualization show that our framework is helpful in
network property information exploration. In all, we present a new way for
efficiently learning state-of-the-art task-independent representations in
complex attributed networks. The source code and datasets of this paper can be
obtained from https://github.com/GraphWorld/FANE.
Reconstruction Loss Minimized FCN for Single Image Dehazing
Authors: Shirsendu Sukanta Halder, Sanchayan Santra, Bhabatosh Chanda
Date: 2018-11-27 00:00:00
Tasks: Image Dehazing, Single Image Dehazing

Haze and fog reduce the visibility of outdoor scenes as a veil like
semi-transparent layer appears over the objects. As a result, images captured
under such conditions lack contrast. Image dehazing methods try to alleviate
this problem by recovering a clear version of the image. In this paper, we
propose a Fully Convolutional Neural Network based model to recover the clear
scene radiance by estimating the environmental illumination and the scene
transmittance jointly from a hazy image. The method uses a relaxed haze imaging
model to allow for the situations with non-uniform illumination. We have
trained the network by minimizing a custom-defined loss that measures the error
of reconstructing the hazy image in three different ways. Additionally, we use
a multilevel approach to determine the scene transmittance and the
environmental illumination in order to reduce the dependence of the estimate on
image scale. Evaluations show that our model performs well compared to the
existing state-of-the-art methods. It also verifies the potential of our model
in diverse situations and various lighting conditions.
Unsupervised Image Captioning
Authors: Yang Feng, Lin Ma, Wei Liu, Jiebo Luo
Date: 2018-11-27 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Captioning, Image Description, Sentence

Deep neural networks have achieved great successes on the image captioning
task. However, most of the existing models depend heavily on paired
image-sentence datasets, which are very expensive to acquire. In this paper, we
make the first attempt to train an image captioning model in an unsupervised
manner. Instead of relying on manually labeled image-sentence pairs, our
proposed model merely requires an image set, a sentence corpus, and an existing
visual concept detector. The sentence corpus is used to teach the captioning
model how to generate plausible sentences. Meanwhile, the knowledge in the
visual concept detector is distilled into the captioning model to guide the
model to recognize the visual concepts in an image. In order to further
encourage the generated captions to be semantically consistent with the image,
the image and caption are projected into a common latent space so that they can
reconstruct each other. Given that the existing sentence corpora are mainly
designed for linguistic research and are thus with little reference to image
contents, we crawl a large-scale image description corpus of two million
natural sentences to facilitate the unsupervised image captioning scenario.
Experimental results show that our proposed model is able to produce quite
promising results without any caption annotations.
Tackling Early Sparse Gradients in Softmax Activation Using Leaky Squared Euclidean Distance
Authors: Wei Shen, Rujie Liu
Date: 2018-11-27 00:00:00
Tasks: One-Shot Learning

Softmax activation is commonly used to output the probability distribution
over categories based on certain distance metric. In scenarios like one-shot
learning, the distance metric is often chosen to be squared Euclidean distance
between the query sample and the category prototype. This practice works well
in most time. However, we find that choosing squared Euclidean distance may
cause distance explosion leading gradients to be extremely sparse in the early
stage of back propagation. We term this phenomena as the early sparse gradients
problem. Though it doesn't deteriorate the convergence of the model, it may set
up a barrier to further model improvement. To tackle this problem, we propose
to use leaky squared Euclidean distance to impose a restriction on distances.
In this way, we can avoid distance explosion and increase the magnitude of
gradients. Extensive experiments are conducted on Omniglot and miniImageNet
datasets. We show that using leaky squared Euclidean distance can improve
one-shot classification accuracy on both datasets.
Joint Representation Learning of Cross-lingual Words and Entities via Attentive Distant Supervision
Authors: Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu, Chengjiang Li, Xu Chen, Tiansi Dong
Date: 2018-11-27 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Cross-Lingual Entity Linking, Entity Linking, Representation Learning, Translation, Word Translation

Joint representation learning of words and entities benefits many NLP tasks,
but has not been well explored in cross-lingual settings. In this paper, we
propose a novel method for joint representation learning of cross-lingual words
and entities. It captures mutually complementary knowledge, and enables
cross-lingual inferences among knowledge bases and texts. Our method does not
require parallel corpora, and automatically generates comparable data via
distant supervision using multi-lingual knowledge bases. We utilize two types
of regularizers to align cross-lingual words and entities, and design knowledge
attention and cross-lingual attention to further reduce noises. We conducted a
series of experiments on three tasks: word translation, entity relatedness, and
cross-lingual entity linking. The results, both qualitatively and
quantitatively, demonstrate the significance of our method.
Verb Argument Structure Alternations in Word and Sentence Embeddings
Authors: Katharina Kann, Alex Warstadt, Adina Williams, Samuel R. Bowman
Date: 2018-11-27 00:00:00
Proceeding: WS 2019 1
Tasks: Sentence, Sentence Embedding, Sentence-Embedding, Sentence Embeddings, Word Embeddings

Verbs occur in different syntactic environments, or frames. We investigate
whether artificial neural networks encode grammatical distinctions necessary
for inferring the idiosyncratic frame-selectional properties of verbs. We
introduce five datasets, collectively called FAVA, containing in aggregate
nearly 10k sentences labeled for grammatical acceptability, illustrating
different verbal argument structure alternations. We then test whether models
can distinguish acceptable English verb-frame combinations from unacceptable
ones using a sentence embedding alone. For converging evidence, we further
construct LaVA, a corresponding word-level dataset, and investigate whether the
same syntactic features can be extracted from word embeddings. Our models
perform reliable classifications for some verbal alternations but not others,
suggesting that while these representations do encode fine-grained lexical
information, it is incomplete or can be hard to extract. Further, differences
between the word- and sentence-level models show that some information present
in word embeddings is not passed on to the down-stream sentence embeddings.
Event-Based Structured Light for Depth Reconstruction using Frequency Tagged Light Patterns
Authors: T. Leroux, S. -H. Ieng, R. Benosman
Date: 2018-11-27 00:00:00
Tasks: 3D Depth Estimation, Depth Estimation

This paper presents a new method for 3D depth estimation using the output of
an asynchronous time driven image sensor. In association with a high speed
Digital Light Processing projection system, our method achieves real-time
reconstruction of 3D points cloud, up to several hundreds of hertz. Unlike
state of the art methodology, we introduce a method that relies on the use of
frequency tagged light pattern that make use of the high temporal resolution of
event based sensors. This approch eases matching as each pattern unique
frequency allow for any easy matching between displayed patterns and the event
based sensor. Results are show on real scenes.
Generating Attention from Classifier Activations for Fine-grained Recognition
Authors: Wei Shen, Rujie Liu
Date: 2018-11-27 00:00:00
Tasks: Semantic Segmentation

Recent advances in fine-grained recognition utilize attention maps to
localize objects of interest. Although there are many ways to generate
attention maps, most of them rely on sophisticated loss functions or complex
training processes. In this work, we propose a simple and straightforward
attention generation model based on the output activations of classifiers. The
advantage of our model is that it can be easily trained with image level labels
and softmax loss functions. More specifically, multiple linear local
classifiers are firstly adopted to perform fine-grained classification at each
location of high level CNN feature maps. The attention map is generated by
aggregating and max-pooling the output activations. Then the attention map
serves as a surrogate target object mask to train those local classifiers,
similar to training models for semantic segmentation. Our model achieves
state-of-the-art results on three heavily benchmarked datasets, i.e. 87.9% on
CUB-200-2011 dataset, 94.1% on Stanford Cars dataset and 92.1% on FGVC-Aircraft
dataset, demonstrating its effectiveness on fine-grained recognition tasks.
Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
Authors: Jacques Kaiser, Hesham Mostafa, Emre Neftci
Date: 2018-11-27 00:00:00

A growing body of work underlines striking similarities between biological neural networks and recurrent, binary neural networks. A relatively smaller body of work, however, discusses similarities between learning dynamics employed in deep artificial neural networks and synaptic plasticity in spiking neural networks. The challenge preventing this is largely caused by the discrepancy between the dynamical properties of synaptic plasticity and the requirements for gradient backpropagation. Learning algorithms that approximate gradient backpropagation using locally synthesized gradients can overcome this challenge. Here, we show that synthetic gradients enable the derivation of Deep Continuous Local Learning (DECOLLE) in spiking neural networks. DECOLLE is capable of learning deep spatio-temporal representations from spikes relying solely on local information. Synaptic plasticity rules are derived systematically from user-defined cost functions and neural dynamics by leveraging existing autodifferentiation methods of machine learning frameworks. We benchmark our approach on the MNIST and the event-based neuromorphic DvsGesture dataset, on which DECOLLE performs comparably to the state-of-the-art. DECOLLE networks provide continuously learning machines that are relevant to biology and supportive of event-based, low-power computer vision architectures matching the accuracies of conventional computers on tasks where temporal precision and speed are essential.
Quality-Aware Multimodal Saliency Detection via Deep Reinforcement Learning
Authors: Xiao Wang, Tao Sun, Rui Yang, Chenglong Li, Bin Luo, Jin Tang
Date: 2018-11-27 00:00:00
Tasks: Decision Making, Decoder, Deep Reinforcement Learning, object-detection, Object Detection, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), RGB Salient Object Detection, Saliency Detection, Salient Object Detection

Incorporating various modes of information into the machine learning
procedure is becoming a new trend. And data from various source can provide
more information than single one no matter they are heterogeneous or
homogeneous. Existing deep learning based algorithms usually directly
concatenate features from each domain to represent the input data. Seldom of
them take the quality of data into consideration which is a key issue in
related multimodal problems. In this paper, we propose an efficient
quality-aware deep neural network to model the weight of data from each domain
using deep reinforcement learning (DRL). Specifically, we take the weighting of
each domain as a decision-making problem and teach an agent learn to interact
with the environment. The agent can tune the weight of each domain through
discrete action selection and obtain a positive reward if the saliency results
are improved. The target of the agent is to achieve maximum rewards after
finished its sequential action selection. We validate the proposed algorithms
on multimodal saliency detection in a coarse-to-fine way. The coarse saliency
maps are generated from an encoder-decoder framework which is trained with
content loss and adversarial loss. The final results can be obtained via
adaptive weighting of maps from each domain. Experiments conducted on two kinds
of salient object detection benchmarks validated the effectiveness of our
proposed quality-aware deep neural network.
A Coarse-to-fine Deep Convolutional Neural Network Framework for Frame Duplication Detection and Localization in Forged Videos
Authors: Chengjiang Long, Arslan Basharat, Anthony Hoogs
Date: 2018-11-27 00:00:00
Tasks: Frame Duplication Detection, Localization In Video Forgery

Videos can be manipulated by duplicating a sequence of consecutive frames with the goal of concealing or imitating a specific content in the same video. In this paper, we propose a novel coarse-to-fine framework based on deep Convolutional Neural Networks to automatically detect and localize such frame duplication. First, an I3D network finds coarse-level matches between candidate duplicated frame sequences and the corresponding selected original frame sequences. Then a Siamese network based on ResNet architecture identifies fine-level correspondences between an individual duplicated frame and the corresponding selected frame. We also propose a robust statistical approach to compute a video-level score indicating the likelihood of manipulation or forgery. Additionally, for providing manipulation localization information we develop an inconsistency detector based on the I3D network to distinguish the duplicated frames from the selected original frames. Quantified evaluation on two challenging video forgery datasets clearly demonstrates that this approach performs significantly better than four recent state-of-the-art methods.
Speaker Diarization With Lexical Information
Authors: Tae Jin Park, Kyu Han, Ian Lane, Panayiotis Georgiou
Date: 2018-11-27 00:00:00
Tasks: Clustering, speaker-diarization, Speaker Diarization

This work presents a novel approach to leverage lexical information for
speaker diarization. We introduce a speaker diarization system that can
directly integrate lexical as well as acoustic information into a speaker
clustering process. Thus, we propose an adjacency matrix integration technique
to integrate word level speaker turn probabilities with speaker embeddings in a
comprehensive way. Our proposed method works without any reference transcript.
Words, and word boundary information are provided by an ASR system. We show
that our proposed method improves a baseline speaker diarization system solely
based on speaker embeddings, achieving a meaningful improvement on the CALLHOME
American English Speech dataset.
Towards Long-Term Memory for Social Robots: Proposing a New Challenge for the RoboCup@Home League
Authors: Matías Pavez, Javier Ruiz del Solar, Victoria Amo, Felix Meyer zu Driehausen
Date: 2018-11-27 00:00:00

Long-term memory is essential to feel like a continuous being, and to be able
to interact/communicate coherently. Social robots need long-term memories in
order to establish long-term relationships with humans and other robots, and do
not act just for the moment. In this paper this challenge is highlighted, open
questions are identified, the need of addressing this challenge in the
RoboCup@Home League with new tests is motivated, and a new test is proposed.
A Unified Coded Deep Neural Network Training Strategy Based on Generalized PolyDot Codes for Matrix Multiplication
Authors: Sanghamitra Dutta, Ziqian Bai, Haewon Jeong, Tze Meng Low, Pulkit Grover
Date: 2018-11-27 00:00:00

This paper has two contributions. First, we propose a novel coded matrix
multiplication technique called Generalized PolyDot codes that advances on
existing methods for coded matrix multiplication under storage and
communication constraints. This technique uses "garbage alignment," i.e.,
aligning computations in coded computing that are not a part of the desired
output. Generalized PolyDot codes bridge between Polynomial codes and MatDot
codes, trading off between recovery threshold and communication costs. Second,
we demonstrate that Generalized PolyDot can be used for training large Deep
Neural Networks (DNNs) on unreliable nodes prone to soft-errors. This requires
us to address three additional challenges: (i) prohibitively large overhead of
coding the weight matrices in each layer of the DNN at each iteration; (ii)
nonlinear operations during training, which are incompatible with linear
coding; and (iii) not assuming presence of an error-free master node, requiring
us to architect a fully decentralized implementation without any "single point
of failure." We allow all primary DNN training steps, namely, matrix
multiplication, nonlinear activation, Hadamard product, and update steps as
well as the encoding/decoding to be error-prone. We consider the case of
mini-batch size $B=1$, as well as $B>1$, leveraging coded matrix-vector
products, and matrix-matrix products respectively. The problem of DNN training
under soft-errors also motivates an interesting, probabilistic error model
under which a real number $(P,Q)$ MDS code is shown to correct $P-Q-1$ errors
with probability $1$ as compared to $\lfloor \frac{P-Q}{2} \rfloor$ for the
more conventional, adversarial error model. We also demonstrate that our
proposed strategy can provide unbounded gains in error tolerance over a
competing replication strategy and a preliminary MDS-code-based strategy for
both these error models.
MATCH-Net: Dynamic Prediction in Survival Analysis using Convolutional Neural Networks
Authors: Daniel Jarrett, Jinsung Yoon, Mihaela van der Schaar
Date: 2018-11-26 00:00:00
Tasks: Prognosis, Survival Analysis

Accurate prediction of disease trajectories is critical for early
identification and timely treatment of patients at risk. Conventional methods
in survival analysis are often constrained by strong parametric assumptions and
limited in their ability to learn from high-dimensional data, while existing
neural network models are not readily-adapted to the longitudinal setting. This
paper develops a novel convolutional approach that addresses these drawbacks.
We present MATCH-Net: a Missingness-Aware Temporal Convolutional Hitting-time
Network, designed to capture temporal dependencies and heterogeneous
interactions in covariate trajectories and patterns of missingness. To the best
of our knowledge, this is the first investigation of temporal convolutions in
the context of dynamic prediction for personalized risk prognosis. Using
real-world data from the Alzheimer's Disease Neuroimaging Initiative, we
demonstrate state-of-the-art performance without making any assumptions
regarding underlying longitudinal or time-to-event processes attesting to the
model's potential utility in clinical decision support.
ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies
Authors: Bao Wang, Binjie Yuan, Zuoqiang Shi, Stanley J. Osher
Date: 2018-11-26 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Adversarial Attack, Adversarial Defense

Empirical adversarial risk minimization (EARM) is a widely used mathematical framework to robustly train deep neural nets (DNNs) that are resistant to adversarial attacks. However, both natural and robust accuracies, in classifying clean and adversarial images, respectively, of the trained robust models are far from satisfactory. In this work, we unify the theory of optimal control of transport equations with the practice of training and testing of ResNets. Based on this unified viewpoint, we propose a simple yet effective ResNets ensemble algorithm to boost the accuracy of the robustly trained model on both clean and adversarial images. The proposed algorithm consists of two components: First, we modify the base ResNets by injecting a variance specified Gaussian noise to the output of each residual mapping. Second, we average over the production of multiple jointly trained modified ResNets to get the final prediction. These two steps give an approximation to the Feynman-Kac formula for representing the solution of a transport equation with viscosity, or a convection-diffusion equation. For the CIFAR10 benchmark, this simple algorithm leads to a robust model with a natural accuracy of {\bf 85.62}\% on clean images and a robust accuracy of ${\bf 57.94 \%}$ under the 20 iterations of the IFGSM attack, which outperforms the current state-of-the-art in defending against IFGSM attack on the CIFAR10. Both natural and robust accuracies of the proposed ResNets ensemble can be improved dynamically as the building block ResNet advances. The code is available at: \url{https://github.com/BaoWangMath/EnResNet}.
Joint Monocular 3D Vehicle Detection and Tracking
Authors: Hou-Ning Hu, Qi-Zhi Cai, Dequan Wang, Ji Lin, Min Sun, Philipp Krähenbühl, Trevor Darrell, Fisher Yu
Date: 2018-11-26 00:00:00
Proceeding: ICCV 2019 10
Tasks: 3D Object Detection, 3D Pose Estimation, Autonomous Vehicles, Multiple Object Tracking, Object Tracking, Online Multi-Object Tracking, Pose Estimation, Trajectory Prediction, vehicle detection

Vehicle 3D extents and trajectories are critical cues for predicting the future location of vehicles and planning future agent ego-motion based on those predictions. In this paper, we propose a novel online framework for 3D vehicle detection and tracking from monocular videos. The framework can not only associate detections of vehicles in motion over time, but also estimate their complete 3D bounding box information from a sequence of 2D images captured on a moving platform. Our method leverages 3D box depth-ordering matching for robust instance association and utilizes 3D trajectory prediction for re-identification of occluded vehicles. We also design a motion learning module based on an LSTM for more accurate long-term motion extrapolation. Our experiments on simulation, KITTI, and Argoverse datasets show that our 3D tracking pipeline offers robust data association and tracking. On Argoverse, our image-based method is significantly better for tracking 3D vehicles within 30 meters than the LiDAR-centric baseline methods.