pwc-archive/papers-with-abstracts huggingface dataset:
Not All Words are Equal: Video-specific Information Loss for Video Captioning
Authors: Jiarong Dong, Ke Gao, Xiaokai Chen, Junbo Guo, Juan Cao, Yongdong Zhang
Date: 2019-01-01 00:00:00
Tasks: All, Video Captioning

An ideal description for a given video should fix its gaze on salient and
representative content, which is capable of distinguishing this video from
others. However, the distribution of different words is unbalanced in video
captioning datasets, where distinctive words for describing video-specific
salient objects are far less than common words such as 'a' 'the' and 'person'.
The dataset bias often results in recognition error or detail deficiency of
salient but unusual objects. To address this issue, we propose a novel learning
strategy called Information Loss, which focuses on the relationship between the
video-specific visual content and corresponding representative words. Moreover,
a framework with hierarchical visual representations and an optimized
hierarchical attention mechanism is established to capture the most salient
spatial-temporal visual information, which fully exploits the potential
strength of the proposed learning strategy. Extensive experiments demonstrate
that the ingenious guidance strategy together with the optimized architecture
outperforms state-of-the-art video captioning methods on MSVD with CIDEr score
87.5, and achieves superior CIDEr score 47.7 on MSR-VTT. We also show that our
Information Loss is generic which improves various models by significant
margins.
Proceedings 3rd Workshop on formal reasoning about Causation, Responsibility, and Explanations in Science and Technology
Authors: Bernd Finkbeiner, Samantha Kleinberg
Date: 2019-01-01 00:00:00
Tasks: Philosophy

The CREST 2018 workshop is the third in a series of workshops addressing
formal approaches to reasoning about causation in systems engineering. The
topic of formally identifying the cause(s) of specific events - usually some
form of failures -, and explaining why they occurred, are increasingly in the
focus of several, disjoint communities. The main objective of CREST is to bring
together researchers and practitioners from industry and academia in order to
enable discussions how explicit and implicit reasoning about causation is
performed. A further objective is to link to the foundations of causal
reasoning in the philosophy of sciences and to causal reasoning performed in
other areas of computer science, engineering, and beyond.
Exploring spectro-temporal features in end-to-end convolutional neural networks
Authors: Sean Robertson, Gerald Penn, Yingxue Wang
Date: 2019-01-01 00:00:00
Tasks: speech-recognition, Speech Recognition

Triangular, overlapping Mel-scaled filters ("f-banks") are the current
standard input for acoustic models that exploit their input's time-frequency
geometry, because they provide a psycho-acoustically motivated time-frequency
geometry for a speech signal. F-bank coefficients are provably robust to small
deformations in the scale. In this paper, we explore two ways in which filter
banks can be adjusted for the purposes of speech recognition. First, triangular
filters can be replaced with Gabor filters, a compactly supported filter that
better localizes events in time, or Gammatone filters, a
psychoacoustically-motivated filter. Second, by rearranging the order of
operations in computing filter bank features, features can be integrated over
smaller time scales while simultaneously providing better frequency resolution.
We make all feature implementations available online through open-source
repositories. Initial experimentation with a modern end-to-end CNN phone
recognizer yielded no significant improvements to phone error rate due to
either modification. The result, and its ramifications with respect to learned
filter banks, is discussed.
Recurrent Neural Networks for Time Series Forecasting
Authors: Gábor Petneházi
Date: 2019-01-01 00:00:00
Tasks: Feature Engineering, Time Series, Time Series Analysis, Time Series Forecasting

Time series forecasting is difficult. It is difficult even for recurrent
neural networks with their inherent ability to learn sequentiality. This
article presents a recurrent neural network based time series forecasting
framework covering feature engineering, feature importances, point and interval
predictions, and forecast evaluation. The description of the method is followed
by an empirical study using both LSTM and GRU networks.
Improving Tree-LSTM with Tree Attention
Authors: Mahtab Ahmed, Muhammad Rifayat Samee, Robert E. Mercer
Date: 2019-01-01 00:00:00
Tasks: Sentence

In Natural Language Processing (NLP), we often need to extract information
from tree topology. Sentence structure can be represented via a dependency tree
or a constituency tree structure. For this reason, a variant of LSTMs, named
Tree-LSTM, was proposed to work on tree topology. In this paper, we design a
generalized attention framework for both dependency and constituency trees by
encoding variants of decomposable attention inside a Tree-LSTM cell. We
evaluated our models on a semantic relatedness task and achieved notable
results compared to Tree-LSTM based methods with no attention as well as other
neural and non-neural methods and good results compared to Tree-LSTM based
methods with attention.
Impossibility and Uncertainty Theorems in AI Value Alignment (or why your AGI should not have a utility function)
Authors: Peter Eckersley
Date: 2018-12-31 00:00:00
Tasks: BIG-bench Machine Learning

Utility functions or their equivalents (value functions, objective functions,
loss functions, reward functions, preference orderings) are a central tool in
most current machine learning systems. These mechanisms for defining goals and
guiding optimization run into practical and conceptual difficulty when there
are independent, multi-dimensional objectives that need to be pursued
simultaneously and cannot be reduced to each other. Ethicists have proved
several impossibility theorems that stem from this origin; those results appear
to show that there is no way of formally specifying what it means for an
outcome to be good for a population without violating strong human ethical
intuitions (in such cases, the objective function is a social welfare
function). We argue that this is a practical problem for any machine learning
system (such as medical decision support systems or autonomous weapons) or
rigidly rule-based bureaucracy that will make high stakes decisions about human
lives: such systems should not use objective functions in the strict
mathematical sense.
  We explore the alternative of using uncertain objectives, represented for
instance as partially ordered preferences, or as probability distributions over
total orders. We show that previously known impossibility theorems can be
transformed into uncertainty theorems in both of those settings, and prove
lower bounds on how much uncertainty is implied by the impossibility results.
We close by proposing two conjectures about the relationship between
uncertainty in objectives and severe unintended consequences from AI systems.
Extreme Relative Pose Estimation for RGB-D Scans via Scene Completion
Authors: Zhenpei Yang, Jeffrey Z. Pan, Linjie Luo, Xiaowei Zhou, Kristen Grauman, Qi-Xing Huang
Date: 2018-12-31 00:00:00
Proceeding: CVPR 2019 6
Tasks: Pose Estimation

Estimating the relative rigid pose between two RGB-D scans of the same
underlying environment is a fundamental problem in computer vision, robotics,
and computer graphics. Most existing approaches allow only limited maximum
relative pose changes since they require considerable overlap between the input
scans. We introduce a novel deep neural network that extends the scope to
extreme relative poses, with little or even no overlap between the input scans.
The key idea is to infer more complete scene information about the underlying
environment and match on the completed scans. In particular, instead of only
performing scene completion from each individual scan, our approach alternates
between relative pose estimation and scene completion. This allows us to
perform scene completion by utilizing information from both input scans at late
iterations, resulting in better results for both scene completion and relative
pose estimation. Experimental results on benchmark datasets show that our
approach leads to considerable improvements over state-of-the-art approaches
for relative pose estimation. In particular, our approach provides encouraging
relative pose estimates even between non-overlapping scans.
Deep Frame Prediction for Video Coding
Authors: Hyomin Choi, Ivan V. Bajic
Date: 2018-12-31 00:00:00
Tasks: Decoder, Prediction

We propose a novel frame prediction method using a deep neural network (DNN), with the goal of improving video coding efficiency. The proposed DNN makes use of decoded frames, at both encoder and decoder, to predict textures of the current coding block. Unlike conventional inter-prediction, the proposed method does not require any motion information to be transferred between the encoder and the decoder. Still, both uni-directional and bi-directional prediction are possible using the proposed DNN, which is enabled by the use of the temporal index channel, in addition to color channels. In this study, we developed a jointly trained DNN for both uni- and bi- directional prediction, as well as separate networks for uni- and bi-directional prediction, and compared the efficacy of both approaches. The proposed DNNs were compared with the conventional motion-compensated prediction in the latest video coding standard, HEVC, in terms of BD-Bitrate. The experiments show that the proposed joint DNN (for both uni- and bi-directional prediction) reduces the luminance bitrate by about 4.4%, 2.4%, and 2.3% in the Low delay P, Low delay, and Random access configurations, respectively. In addition, using the separately trained DNNs brings further bit savings of about 0.3%-0.5%.
Determining Principal Component Cardinality through the Principle of Minimum Description Length
Authors: Ami Tavory
Date: 2018-12-31 00:00:00
Tasks: Dimensionality Reduction, Model Selection, regression

PCA (Principal Component Analysis) and its variants areubiquitous techniques for matrix dimension reduction and reduced-dimensionlatent-factor extraction. One significant challenge in using PCA, is thechoice of the number of principal components. The information-theoreticMDL (Minimum Description Length) principle gives objective compression-based criteria for model selection, but it is difficult to analytically applyits modern definition - NML (Normalized Maximum Likelihood) - to theproblem of PCA. This work shows a general reduction of NML prob-lems to lower-dimension problems. Applying this reduction, it boundsthe NML of PCA, by terms of the NML of linear regression, which areknown.
Entity Synonym Discovery via Multipiece Bilateral Context Matching
Authors: Chenwei Zhang, Yaliang Li, Nan Du, Wei Fan, Philip S. Yu
Date: 2018-12-31 00:00:00
Tasks: Entity Disambiguation

Being able to automatically discover synonymous entities in an open-world setting benefits various tasks such as entity disambiguation or knowledge graph canonicalization. Existing works either only utilize entity features, or rely on structured annotations from a single piece of context where the entity is mentioned. To leverage diverse contexts where entities are mentioned, in this paper, we generalize the distributional hypothesis to a multi-context setting and propose a synonym discovery framework that detects entity synonyms from free-text corpora with considerations on effectiveness and robustness. As one of the key components in synonym discovery, we introduce a neural network model SYNONYMNET to determine whether or not two given entities are synonym with each other. Instead of using entities features, SYNONYMNET makes use of multiple pieces of contexts in which the entity is mentioned, and compares the context-level similarity via a bilateral matching schema. Experimental results demonstrate that the proposed model is able to detect synonym sets that are not observed during training on both generic and domain-specific datasets: Wiki+Freebase, PubMed+UMLS, and MedBook+MKG, with up to 4.16% improvement in terms of Area Under the Curve and 3.19% in terms of Mean Average Precision compared to the best baseline method.
SiCloPe: Silhouette-Based Clothed People
Authors: Ryota Natsume, Shunsuke Saito, Zeng Huang, Weikai Chen, Chongyang Ma, Hao Li, Shigeo Morishima
Date: 2018-12-31 00:00:00
Proceeding: CVPR 2019 6
Tasks: Generative Adversarial Network, Image-to-Image Translation

We introduce a new silhouette-based representation for modeling clothed human
bodies using deep generative models. Our method can reconstruct a complete and
textured 3D model of a person wearing clothes from a single input picture.
Inspired by the visual hull algorithm, our implicit representation uses 2D
silhouettes and 3D joints of a body pose to describe the immense shape
complexity and variations of clothed people. Given a segmented 2D silhouette of
a person and its inferred 3D joints from the input picture, we first synthesize
consistent silhouettes from novel view points around the subject. The
synthesized silhouettes which are the most consistent with the input
segmentation are fed into a deep visual hull algorithm for robust 3D shape
prediction. We then infer the texture of the subject's back view using the
frontal image and segmentation mask as input to a conditional generative
adversarial network. Our experiments demonstrate that our silhouette-based
model is an effective representation and the appearance of the back view can be
predicted reliably using an image-to-image translation network. While classic
methods based on parametric models often fail for single-view images of
subjects with challenging clothing, our approach can still produce successful
results, which are comparable to those obtained from multi-view input.
Deep Information Theoretic Registration
Authors: Alireza Sedghi, Jie Luo, Alireza Mehrtash, Steve Pieper, Clare M. Tempany, Tina Kapur, Parvin Mousavi, William M. Wells III
Date: 2018-12-31 00:00:00
Tasks: Image Registration

This paper establishes an information theoretic framework for deep metric
based image registration techniques. We show an exact equivalence between
maximum profile likelihood and minimization of joint entropy, an important
early information theoretic registration method. We further derive deep
classifier-based metrics that can be used with iterated maximum likelihood to
achieve Deep Information Theoretic Registration on patches rather than pixels.
This alleviates a major shortcoming of previous information theoretic
registration approaches, namely the implicit pixel-wise independence
assumptions. Our proposed approach does not require well-registered training
data; this brings previous fully supervised deep metric registration approaches
to the realm of weak supervision. We evaluate our approach on several image
registration tasks and show significantly better performance compared to mutual
information, specifically when images have substantially different contrasts.
This work enables general-purpose registration in applications where current
methods are not successful.
Convex Relaxations of Convolutional Neural Nets
Authors: Burak Bartan, Mert Pilanci
Date: 2018-12-31 00:00:00

We propose convex relaxations for convolutional neural nets with one hidden
layer where the output weights are fixed. For convex activation functions such
as rectified linear units, the relaxations are convex second order cone
programs which can be solved very efficiently. We prove that the relaxation
recovers the global minimum under a planted model assumption, given
sufficiently many training samples from a Gaussian distribution. We also
identify a phase transition phenomenon in recovering the global minimum for the
relaxation.
Inorganic Materials Synthesis Planning with Literature-Trained Neural Networks
Authors: Edward Kim, Zach Jensen, Alexander van Grootel, Kevin Huang, Matthew Staib, Sheshera Mysore, Haw-Shiuan Chang, Emma Strubell, Andrew McCallum, Stefanie Jegelka, Elsa Olivetti
Date: 2018-12-31 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Word Embeddings

Leveraging new data sources is a key step in accelerating the pace of
materials design and discovery. To complement the strides in synthesis planning
driven by historical, experimental, and computed data, we present an automated
method for connecting scientific literature to synthesis insights. Starting
from natural language text, we apply word embeddings from language models,
which are fed into a named entity recognition model, upon which a conditional
variational autoencoder is trained to generate syntheses for arbitrary
materials. We show the potential of this technique by predicting precursors for
two perovskite materials, using only training data published over a decade
prior to their first reported syntheses. We demonstrate that the model learns
representations of materials corresponding to synthesis-related properties, and
that the model's behavior complements existing thermodynamic knowledge.
Finally, we apply the model to perform synthesizability screening for proposed
novel perovskite compounds.
Interest Point Detection based on Adaptive Ternary Coding
Authors: Zhenwei Miao, Kim-Hui Yap, Xudong Jiang
Date: 2018-12-31 00:00:00
Tasks: Face Recognition, Interest Point Detection, Quantization

In this paper, an adaptive pixel ternary coding mechanism is proposed and a
contrast invariant and noise resistant interest point detector is developed on
the basis of this mechanism. Every pixel in a local region is adaptively
encoded into one of the three statuses: bright, uncertain and dark. The blob
significance of the local region is measured by the spatial distribution of the
bright and dark pixels. Interest points are extracted from this blob
significance measurement. By labeling the statuses of ternary bright,
uncertain, and dark, the proposed detector shows more robustness to image noise
and quantization errors. Moreover, the adaptive strategy for the ternary
cording, which relies on two thresholds that automatically converge to the
median of the local region in measurement, enables this coding to be
insensitive to the image local contrast. As a result, the proposed detector is
invariant to illumination changes. The state-of-the-art results are achieved on
the standard datasets, and also in the face recognition application.
DCI: Discriminative and Contrast Invertible Descriptor
Authors: Zhenwei Miao, Kim-Hui Yap, Xudong Jiang, Subbhuraam Sinduja, Zhenhua Wang
Date: 2018-12-31 00:00:00
Tasks: Object, Object Recognition, Retrieval

Local feature descriptors have been widely used in fine-grained visual object
search thanks to their robustness in scale and rotation variation and cluttered
background. However, the performance of such descriptors drops under severe
illumination changes. In this paper, we proposed a Discriminative and Contrast
Invertible (DCI) local feature descriptor. In order to increase the
discriminative ability of the descriptor under illumination changes, a Laplace
gradient based histogram is proposed. A robust contrast flipping estimate is
proposed based on the divergence of a local region. Experiments on fine-grained
object recognition and retrieval applications demonstrate the superior
performance of DCI descriptor to others.
Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies
Authors: Alexander Sax, Bradley Emi, Amir R. Zamir, Leonidas Guibas, Silvio Savarese, Jitendra Malik
Date: 2018-12-31 00:00:00
Tasks: Object Detection, Reinforcement Learning

How much does having visual priors about the world (e.g. the fact that the
world is 3D) assist in learning to perform downstream motor tasks (e.g.
delivering a package)? We study this question by integrating a generic
perceptual skill set (e.g. a distance estimator, an edge detector, etc.) within
a reinforcement learning framework--see Figure 1. This skill set (hereafter
mid-level perception) provides the policy with a more processed state of the
world compared to raw images.
  We find that using a mid-level perception confers significant advantages over
training end-to-end from scratch (i.e. not leveraging priors) in
navigation-oriented tasks. Agents are able to generalize to situations where
the from-scratch approach fails and training becomes significantly more sample
efficient. However, we show that realizing these gains requires careful
selection of the mid-level perceptual skills. Therefore, we refine our findings
into an efficient max-coverage feature set that can be adopted in lieu of raw
images. We perform our study in completely separate buildings for training and
testing and compare against visually blind baseline policies and
state-of-the-art feature learning methods.
The role of visual saliency in the automation of seismic interpretation
Authors: Muhammad Amir Shafiq, Tariq Alshawi, Zhiling Long, Ghassan AlRegib
Date: 2018-12-31 00:00:00
Tasks: Attribute, Seismic Interpretation

In this paper, we propose a workflow based on SalSi for the detection and
delineation of geological structures such as salt domes. SalSi is a seismic
attribute designed based on the modeling of human visual system that detects
the salient features and captures the spatial correlation within seismic
volumes for delineating seismic structures. Using SalSi, we can not only
highlight the neighboring regions of salt domes to assist a seismic interpreter
but also delineate such structures using a region growing method and
post-processing. The proposed delineation workflow detects the salt-dome
boundary with very good precision and accuracy. Experimental results show the
effectiveness of the proposed workflow on a real seismic dataset acquired from
the North Sea, F3 block. For the subjective evaluation of the results of
different salt-dome delineation algorithms, we have used a reference salt-dome
boundary interpreted by a geophysicist. For the objective evaluation of
results, we have used five different metrics based on pixels, shape, and
curvedness to establish the effectiveness of the proposed workflow. The
proposed workflow is not only fast but also yields better results as compared
to other salt-dome delineation algorithms and shows a promising potential in
seismic interpretation.
Gray-box Adversarial Testing for Control Systems with Machine Learning Component
Authors: Shakiba Yaghoubi, Georgios Fainekos
Date: 2018-12-31 00:00:00
Tasks: BIG-bench Machine Learning

Neural Networks (NN) have been proposed in the past as an effective means for
both modeling and control of systems with very complex dynamics. However,
despite the extensive research, NN-based controllers have not been adopted by
the industry for safety critical systems. The primary reason is that systems
with learning based controllers are notoriously hard to test and verify. Even
harder is the analysis of such systems against system-level specifications. In
this paper, we provide a gradient based method for searching the input space of
a closed-loop control system in order to find adversarial samples against some
system-level requirements. Our experimental results show that combined with
randomized search, our method outperforms Simulated Annealing optimization.