pwc-archive/papers-with-abstracts huggingface dataset:
Learning Representations for Automatic Colorization
Authors: Gustav Larsson, Michael Maire, Gregory Shakhnarovich
Date: 2016-03-22 00:00:00
Tasks: Colorization, Image Colorization, Representation Learning

We develop a fully automatic image colorization system. Our approach
leverages recent advances in deep networks, exploiting both low-level and
semantic representations. As many scene elements naturally appear according to
multimodal color distributions, we train our model to predict per-pixel color
histograms. This intermediate output can be used to automatically generate a
color image, or further manipulated prior to image formation. On both fully and
partially automatic colorization tasks, we outperform existing methods. We also
explore colorization as a vehicle for self-supervised visual representation
learning.
Colorization as a Proxy Task for Visual Understanding
Authors: Gustav Larsson, Michael Maire, Gregory Shakhnarovich
Date: 2017-03-11 00:00:00
Proceeding: CVPR 2017 7
Tasks: Colorization

We investigate and improve self-supervision as a drop-in replacement for
ImageNet pretraining, focusing on automatic colorization as the proxy task.
Self-supervised training has been shown to be more promising for utilizing
unlabeled data than other, traditional unsupervised learning methods. We build
on this success and evaluate the ability of our self-supervised network in
several contexts. On VOC segmentation and classification tasks, we present
results that are state-of-the-art among methods not using ImageNet labels for
pretraining representations.
  Moreover, we present the first in-depth analysis of self-supervision via
colorization, concluding that formulation of the loss, training details and
network architecture play important roles in its effectiveness. This
investigation is further expanded by revisiting the ImageNet pretraining
paradigm, asking questions such as: How much training data is needed? How many
labels are needed? How much do features change when fine-tuned? We relate these
questions back to self-supervision by showing that colorization provides a
similarly powerful supervisory signal as various flavors of ImageNet
pretraining.
Leveraging Sparse and Dense Feature Combinations for Sentiment Classification
Authors: Tao Yu, Christopher Hidey, Owen Rambow, Kathleen McKeown
Date: 2017-08-13 00:00:00
Tasks: BIG-bench Machine Learning, Classification, Deep Learning, General Classification, Sentiment Analysis, Sentiment Classification

Neural networks are one of the most popular approaches for many natural
language processing tasks such as sentiment analysis. They often outperform
traditional machine learning models and achieve the state-of-art results on
most tasks. However, many existing deep learning models are complex, difficult
to train and provide a limited improvement over simpler methods. We propose a
simple, robust and powerful model for sentiment classification. This model
outperforms many deep learning models and achieves comparable results to other
deep learning models with complex architectures on sentiment analysis datasets.
We publish the code online.
Reprogramming Matter, Life, and Purpose
Authors: Hector Zenil
Date: 2017-04-02 00:00:00

Reprogramming matter may sound far-fetched, but we have been doing it with
increasing power and staggering efficiency for at least 60 years, and for
centuries we have been paving the way toward the ultimate reprogrammed fate of
the universe, the vessel of all programs. How will we be doing it in 60 years'
time and how will it impact life and the purpose both of machines and of
humans?
On Unifying Multi-View Self-Representations for Clustering by Tensor Multi-Rank Minimization
Authors: Yuan Xie, DaCheng Tao, Wensheng Zhang, Lei Zhang, Yan Liu, Yanyun Qu
Date: 2016-10-23 00:00:00
Tasks: Clustering, Multi-view Subspace Clustering

In this paper, we address the multi-view subspace clustering problem. Our
method utilizes the circulant algebra for tensor, which is constructed by
stacking the subspace representation matrices of different views and then
rotating, to capture the low rank tensor subspace so that the refinement of the
view-specific subspaces can be achieved, as well as the high order correlations
underlying multi-view data can be explored.} By introducing a recently proposed
tensor factorization, namely tensor-Singular Value Decomposition (t-SVD)
\cite{kilmer13}, we can impose a new type of low-rank tensor constraint on the
rotated tensor to capture the complementary information from multiple views.
Different from traditional unfolding based tensor norm, this low-rank tensor
constraint has optimality properties similar to that of matrix rank derived
from SVD, so the complementary information among views can be explored more
efficiently and thoroughly. The established model, called t-SVD based
Multi-view Subspace Clustering (t-SVD-MSC), falls into the applicable scope of
augmented Lagrangian method, and its minimization problem can be efficiently
solved with theoretical convergence guarantee and relatively low computational
complexity. Extensive experimental testing on eight challenging image dataset
shows that the proposed method has achieved highly competent objective
performance compared to several state-of-the-art multi-view clustering methods.
Model-Based Multiple Instance Learning
Authors: Ba-Ngu Vo, Dinh Phung, Quang N. Tran, Ba-Tuong Vo
Date: 2017-03-07 00:00:00
Tasks: Clustering, Decision Making, General Classification, model, Multiple Instance Learning, Novelty Detection

While Multiple Instance (MI) data are point patterns -- sets or multi-sets of
unordered points -- appropriate statistical point pattern models have not been
used in MI learning. This article proposes a framework for model-based MI
learning using point process theory. Likelihood functions for point pattern
data derived from point process theory enable principled yet conceptually
transparent extensions of learning tasks, such as classification, novelty
detection and clustering, to point pattern data. Furthermore, tractable point
pattern models as well as solutions for learning and decision making from point
pattern data are developed.
Visual Graph Mining
Authors: Quanshi Zhang, Xuan Song, Ryosuke Shibasaki
Date: 2017-08-13 00:00:00
Tasks: Graph Mining

In this study, we formulate the concept of "mining maximal-size frequent
subgraphs" in the challenging domain of visual data (images and videos). In
general, visual knowledge can usually be modeled as attributed relational
graphs (ARGs) with local attributes representing local parts and pairwise
attributes describing the spatial relationship between parts. Thus, from a
practical perspective, such mining of maximal-size subgraphs can be regarded as
a general platform for discovering and modeling the common objects within
cluttered and unlabeled visual data. Then, from a theoretical perspective,
visual graph mining should encode and overcome the great fuzziness of messy
data collected from complex real-world situations, which conflicts with the
conventional theoretical basis of graph mining designed for tabular data.
Common subgraphs hidden in these ARGs usually have soft attributes, with
considerable inter-graph variation. More importantly, we should also discover
the latent pattern space, including similarity metrics for the pattern and
hidden node relations, during the mining process. In this study, we redefine
the visual subgraph pattern that encodes all of these challenges in a general
way, and propose an approximate but efficient solution to graph mining. We
conduct five experiments to evaluate our method with different kinds of visual
data, including videos and RGB/RGB-D images. These experiments demonstrate the
generality of the proposed method.
Towards Speech Emotion Recognition "in the wild" using Aggregated Corpora and Deep Multi-Task Learning
Authors: Jaebok Kim, Gwenn Englebienne, Khiet P. Truong, Vanessa Evers
Date: 2017-08-13 00:00:00
Tasks: Cross-corpus, Emotion Recognition, Multi-Task Learning, Speech Emotion Recognition

One of the challenges in Speech Emotion Recognition (SER) "in the wild" is
the large mismatch between training and test data (e.g. speakers and tasks). In
order to improve the generalisation capabilities of the emotion models, we
propose to use Multi-Task Learning (MTL) and use gender and naturalness as
auxiliary tasks in deep neural networks. This method was evaluated in
within-corpus and various cross-corpus classification experiments that simulate
conditions "in the wild". In comparison to Single-Task Learning (STL) based
state of the art methods, we found that our MTL method proposed improved
performance significantly. Particularly, models using both gender and
naturalness achieved more gains than those using either gender or naturalness
separately. This benefit was also found in the high-level representations of
the feature space, obtained from our method proposed, where discriminative
emotional clusters could be observed.
Learning Deep Neural Networks for Vehicle Re-ID with Visual-spatio-temporal Path Proposals
Authors: Yantao Shen, Tong Xiao, Hongsheng Li, Shuai Yi, Xiaogang Wang
Date: 2017-08-13 00:00:00
Proceeding: ICCV 2017 10
Tasks: Person Re-Identification, Vehicle Re-Identification

Vehicle re-identification is an important problem and has many applications
in video surveillance and intelligent transportation. It gains increasing
attention because of the recent advances of person re-identification
techniques. However, unlike person re-identification, the visual differences
between pairs of vehicle images are usually subtle and even challenging for
humans to distinguish. Incorporating additional spatio-temporal information is
vital for solving the challenging re-identification task. Existing vehicle
re-identification methods ignored or used over-simplified models for the
spatio-temporal relations between vehicle images. In this paper, we propose a
two-stage framework that incorporates complex spatio-temporal information for
effectively regularizing the re-identification results. Given a pair of vehicle
images with their spatio-temporal information, a candidate
visual-spatio-temporal path is first generated by a chain MRF model with a
deeply learned potential function, where each visual-spatio-temporal state
corresponds to an actual vehicle image with its spatio-temporal information. A
Siamese-CNN+Path-LSTM model takes the candidate path as well as the pairwise
queries to generate their similarity score. Extensive experiments and analysis
show the effectiveness of our proposed method and individual components.
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Authors: Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, Joel Emer
Date: 2017-03-27 00:00:00
Tasks: Benchmarking, speech-recognition, Speech Recognition, Survey

Deep neural networks (DNNs) are currently widely used for many artificial
intelligence (AI) applications including computer vision, speech recognition,
and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it
comes at the cost of high computational complexity. Accordingly, techniques
that enable efficient processing of DNNs to improve energy efficiency and
throughput without sacrificing application accuracy or increasing hardware cost
are critical to the wide deployment of DNNs in AI systems.
  This article aims to provide a comprehensive tutorial and survey about the
recent advances towards the goal of enabling efficient processing of DNNs.
Specifically, it will provide an overview of DNNs, discuss various hardware
platforms and architectures that support DNNs, and highlight key trends in
reducing the computation cost of DNNs either solely via hardware design changes
or via joint hardware design and DNN algorithm changes. It will also summarize
various development resources that enable researchers and practitioners to
quickly get started in this field, and highlight important benchmarking metrics
and design considerations that should be used for evaluating the rapidly
growing number of DNN hardware designs, optionally including algorithmic
co-designs, being proposed in academia and industry.
  The reader will take away the following concepts from this article:
understand the key design considerations for DNNs; be able to evaluate
different DNN hardware implementations with benchmarks and comparison metrics;
understand the trade-offs between various hardware architectures and platforms;
be able to evaluate the utility of various DNN design techniques for efficient
processing; and understand recent implementation trends and opportunities.
Mahalanonbis Distance Informed by Clustering
Authors: Almog Lahav, Ronen Talmon, Yuval Kluger
Date: 2017-08-13 00:00:00
Tasks: Clustering

A fundamental question in data analysis, machine learning and signal
processing is how to compare between data points. The choice of the distance
metric is specifically challenging for high-dimensional data sets, where the
problem of meaningfulness is more prominent (e.g. the Euclidean distance
between images). In this paper, we propose to exploit a property of
high-dimensional data that is usually ignored - which is the structure stemming
from the relationships between the coordinates. Specifically we show that
organizing similar coordinates in clusters can be exploited for the
construction of the Mahalanobis distance between samples. When the observable
samples are generated by a nonlinear transformation of hidden variables, the
Mahalanobis distance allows the recovery of the Euclidean distances in the
hidden space.We illustrate the advantage of our approach on a synthetic example
where the discovery of clusters of correlated coordinates improves the
estimation of the principal directions of the samples. Our method was applied
to real data of gene expression for lung adenocarcinomas (lung cancer). By
using the proposed metric we found a partition of subjects to risk groups with
a good separation between their Kaplan-Meier survival plot.
Mining Deep And-Or Object Structures via Cost-Sensitive Question-Answer-Based Active Annotations
Authors: Quanshi Zhang, Ying Nian Wu, Hao Zhang, Song-Chun Zhu
Date: 2017-08-13 00:00:00
Tasks: Question Answering

This paper presents a cost-sensitive active Question-Answering (QA) framework
for learning a nine-layer And-Or graph (AOG) from web images. The AOG
explicitly represents object categories, poses/viewpoints, parts, and detailed
structures within the parts in a compositional hierarchy. The QA framework is
designed to minimize an overall risk, which trades off the loss and query
costs. The loss is defined for nodes in all layers of the AOG, including the
generative loss (measuring the likelihood of the images) and the discriminative
loss (measuring the fitness to human answers). The cost comprises both the
human labor of answering questions and the computational cost of model
learning. The cost-sensitive QA framework iteratively selects different
storylines of questions to update different nodes in the AOG. Experiments
showed that our method required much less human supervision (e.g., labeling
parts on 3--10 training objects for each category) and achieved better
performance than baseline methods.
Semi-supervised emotion lexicon expansion with label propagation and specialized word embeddings
Authors: Mario Giulianelli
Date: 2017-08-13 00:00:00
Tasks: Word Embeddings

There exist two main approaches to automatically extract affective
orientation: lexicon-based and corpus-based. In this work, we argue that these
two methods are compatible and show that combining them can improve the
accuracy of emotion classifiers. In particular, we introduce a novel variant of
the Label Propagation algorithm that is tailored to distributed word
representations, we apply batch gradient descent to accelerate the optimization
of label propagation and to make the optimization feasible for large graphs,
and we propose a reproducible method for emotion lexicon expansion. We conclude
that label propagation can expand an emotion lexicon in a meaningful way and
that the expanded emotion lexicon can be leveraged to improve the accuracy of
an emotion classifier.
Belief Tree Search for Active Object Recognition
Authors: Mohsen Malmir, Garrison W. Cottrell
Date: 2017-08-13 00:00:00
Tasks: Object, Object Recognition, Reinforcement Learning, Transfer Learning

Active Object Recognition (AOR) has been approached as an unsupervised
learning problem, in which optimal trajectories for object inspection are not
known and are to be discovered by reducing label uncertainty measures or
training with reinforcement learning. Such approaches have no guarantees of the
quality of their solution. In this paper, we treat AOR as a Partially
Observable Markov Decision Process (POMDP) and find near-optimal policies on
training data using Belief Tree Search (BTS) on the corresponding belief Markov
Decision Process (MDP). AOR then reduces to the problem of knowledge transfer
from near-optimal policies on training set to the test set. We train a Long
Short Term Memory (LSTM) network to predict the best next action on the
training set rollouts. We sho that the proposed AOR method generalizes well to
novel views of familiar objects and also to novel objects. We compare this
supervised scheme against guided policy search, and find that the LSTM network
reaches higher recognition accuracy compared to the guided policy method. We
further look into optimizing the observation function to increase the total
collected reward of optimal policy. In AOR, the observation function is known
only approximately. We propose a gradient-based method update to this
approximate observation function to increase the total reward of any policy. We
show that by optimizing the observation function and retraining the supervised
LSTM network, the AOR performance on the test set improves significantly.
Drug-Drug Interaction Extraction from Biomedical Text Using Long Short Term Memory Network
Authors: Sunil Kumar Sahu, Ashish Anand
Date: 2017-01-28 00:00:00
Tasks: Drug–drug Interaction Extraction, Feature Engineering, Medical Relation Extraction, Sentence

Simultaneous administration of multiple drugs can have synergistic or
antagonistic effects as one drug can affect activities of other drugs.
Synergistic effects lead to improved therapeutic outcomes, whereas,
antagonistic effects can be life-threatening, may lead to increased healthcare
cost, or may even cause death. Thus identification of unknown drug-drug
interaction (DDI) is an important concern for efficient and effective
healthcare. Although multiple resources for DDI exist, they are often unable to
keep pace with rich amount of information available in fast growing biomedical
texts. Most existing methods model DDI extraction from text as a classification
problem and mainly rely on handcrafted features. Some of these features further
depend on domain specific tools. Recently neural network models using latent
features have been shown to give similar or better performance than the other
existing models dependent on handcrafted features. In this paper, we present
three models namely, {\it B-LSTM}, {\it AB-LSTM} and {\it Joint AB-LSTM} based
on long short-term memory (LSTM) network. All three models utilize word and
position embedding as latent features and thus do not rely on explicit feature
engineering. Further use of bidirectional long short-term memory (Bi-LSTM)
networks allow implicit feature extraction from the whole sentence. The two
models, {\it AB-LSTM} and {\it Joint AB-LSTM} also use attentive pooling in the
output of Bi-LSTM layer to assign weights to features. Our experimental results
on the SemEval-2013 DDI extraction dataset show that the {\it Joint AB-LSTM}
model outperforms all the existing methods, including those relying on
handcrafted features. The other two proposed LSTM models also perform
competitively with state-of-the-art methods.
Chessboard and chess piece recognition with the support of neural networks
Authors: Maciej A. Czyzewski, Artur Laskowski, Szymon Wasik
Date: 2017-08-13 00:00:00
Tasks: Object Recognition

Chessboard and chess piece recognition is a computer vision problem that has not yet been efficiently solved. However, its solution is crucial for many experienced players who wish to compete against AI bots, but also prefer to make decisions based on the analysis of a physical chessboard. It is also important for organizers of chess tournaments who wish to digitize play for online broadcasting or ordinary players who wish to share their gameplay with friends. Typically, such digitization tasks are performed by humans or with the aid of specialized chessboards and pieces. However, neither solution is easy or convenient. To solve this problem, we propose a novel algorithm for digitizing chessboard configurations. We designed a method that is resistant to lighting conditions and the angle at which images are captured, and works correctly with numerous chessboard styles. The proposed algorithm processes pictures iteratively. During each iteration, it executes three major sub-processes: detecting straight lines, finding lattice points, and positioning the chessboard. Finally, we identify all chess pieces and generate a description of the board utilizing standard notation. For each of these steps, we designed our own algorithm that surpasses existing solutions. We support our algorithms by utilizing machine learning techniques whenever possible. The described method performs extraordinarily well and achieves an accuracy over $99.5\%$ for detecting chessboard lattice points (compared to the $74\%$ for the best alternative), $95\%$ (compared to $60\%$ for the best alternative) for positioning the chessboard in an image, and almost $95\%$ for chess piece recognition.
Cross-linguistic differences and similarities in image descriptions
Authors: Emiel van Miltenburg, Desmond Elliott, Piek Vossen
Date: 2017-07-06 00:00:00
Proceeding: WS 2017 9
Tasks: Image Description, Specificity

Automatic image description systems are commonly trained and evaluated on
large image description datasets. Recently, researchers have started to collect
such datasets for languages other than English. An unexplored question is how
different these datasets are from English and, if there are any differences,
what causes them to differ. This paper provides a cross-linguistic comparison
of Dutch, English, and German image descriptions. We find that these
descriptions are similar in many respects, but the familiarity of crowd workers
with the subjects of the images has a noticeable influence on description
specificity.
Image Quality Assessment Guided Deep Neural Networks Training
Authors: Zhuo Chen, Weisi Lin, Shiqi Wang, Long Xu, Leida Li
Date: 2017-08-13 00:00:00
Tasks: Data Augmentation, image-classification, Image Classification, Image Quality Assessment

For many computer vision problems, the deep neural networks are trained and
validated based on the assumption that the input images are pristine (i.e.,
artifact-free). However, digital images are subject to a wide range of
distortions in real application scenarios, while the practical issues regarding
image quality in high level visual information understanding have been largely
ignored. In this paper, in view of the fact that most widely deployed deep
learning models are susceptible to various image distortions, the distorted
images are involved for data augmentation in the deep neural network training
process to learn a reliable model for practical applications. In particular, an
image quality assessment based label smoothing method, which aims at
regularizing the label distribution of training images, is further proposed to
tune the objective functions in learning the neural network. Experimental
results show that the proposed method is effective in dealing with both low and
high quality images in the typical image classification task.