pwc-archive/papers-with-abstracts huggingface dataset:
Semantic Foggy Scene Understanding with Synthetic Data
Authors: Christos Sakaridis, Dengxin Dai, Luc van Gool
Date: 2017-08-25 00:00:00
Tasks: Image Dehazing, object-detection, Object Detection, Scene Understanding, Semantic Segmentation

This work addresses the problem of semantic foggy scene understanding (SFSU). Although extensive research has been performed on image dehazing and on semantic scene understanding with clear-weather images, little attention has been paid to SFSU. Due to the difficulty of collecting and annotating foggy images, we choose to generate synthetic fog on real images that depict clear-weather outdoor scenes, and then leverage these partially synthetic data for SFSU by employing state-of-the-art convolutional neural networks (CNN). In particular, a complete pipeline to add synthetic fog to real, clear-weather images using incomplete depth information is developed. We apply our fog synthesis on the Cityscapes dataset and generate Foggy Cityscapes with 20550 images. SFSU is tackled in two ways: 1) with typical supervised learning, and 2) with a novel type of semi-supervised learning, which combines 1) with an unsupervised supervision transfer from clear-weather images to their synthetic foggy counterparts. In addition, we carefully study the usefulness of image dehazing for SFSU. For evaluation, we present Foggy Driving, a dataset with 101 real-world images depicting foggy driving scenes, which come with ground truth annotations for semantic segmentation and object detection. Extensive experiments show that 1) supervised learning with our synthetic data significantly improves the performance of state-of-the-art CNN for SFSU on Foggy Driving; 2) our semi-supervised learning strategy further improves performance; and 3) image dehazing marginally advances SFSU with our learning strategy. The datasets, models and code are made publicly available.
Photometric stereo for strong specular highlights
Authors: Maryam Khanian, Ali Sharifi Boroujerdi, Michael Breuß
Date: 2017-09-05 00:00:00

Photometric stereo (PS) is a fundamental technique in computer vision known
to produce 3-D shape with high accuracy. The setting of PS is defined by using
several input images of a static scene taken from one and the same camera
position but under varying illumination. The vast majority of studies in this
3-D reconstruction method assume orthographic projection for the camera model.
In addition, they mainly consider the Lambertian reflectance model as the way
that light scatters at surfaces. So, providing reliable PS results from real
world objects still remains a challenging task. We address 3-D reconstruction
by PS using a more realistic set of assumptions combining for the first time
the complete Blinn-Phong reflectance model and perspective projection. To this
end, we will compare two different methods of incorporating the perspective
projection into our model. Experiments are performed on both synthetic and real
world images. Note that our real-world experiments do not benefit from
laboratory conditions. The results show the high potential of our method even
for complex real world applications such as medical endoscopy images which may
include high amounts of specular highlights.
Visualizing and Improving Scattering Networks
Authors: Fergal Cotter, Nick Kingsbury
Date: 2017-09-05 00:00:00
Tasks: General Classification, image-classification, Image Classification, Texture Classification

Scattering Transforms (or ScatterNets) introduced by Mallat are a promising
start into creating a well-defined feature extractor to use for pattern
recognition and image classification tasks. They are of particular interest due
to their architectural similarity to Convolutional Neural Networks (CNNs),
while requiring no parameter learning and still performing very well
(particularly in constrained classification tasks).
  In this paper we visualize what the deeper layers of a ScatterNet are
sensitive to using a 'DeScatterNet'. We show that the higher orders of
ScatterNets are sensitive to complex, edge-like patterns (checker-boards and
rippled edges). These complex patterns may be useful for texture
classification, but are quite dissimilar from the patterns visualized in second
and third layers of Convolutional Neural Networks (CNNs) - the current state of
the art Image Classifiers. We propose that this may be the source of the
current gaps in performance between ScatterNets and CNNs (83% vs 93% on
CIFAR-10 for ScatterNet+SVM vs ResNet). We then use these visualization tools
to propose possible enhancements to the ScatterNet design, which show they have
the power to extract features more closely resembling CNNs, while still being
well-defined and having the invariance properties fundamental to ScatterNets.
Learning Non-Metric Visual Similarity for Image Retrieval
Authors: Noa Garcia, George Vogiatzis
Date: 2017-09-05 00:00:00
Proceeding: ICLR 2018 1
Tasks: Content-Based Image Retrieval, Image Retrieval, Instance Search, Retrieval

Measuring visual similarity between two or more instances within a data
distribution is a fundamental task in image retrieval. Theoretically,
non-metric distances are able to generate a more complex and accurate
similarity model than metric distances, provided that the non-linear data
distribution is precisely captured by the system. In this work, we explore
neural networks models for learning a non-metric similarity function for
instance search. We argue that non-metric similarity functions based on neural
networks can build a better model of human visual perception than standard
metric distances. As our proposed similarity function is differentiable, we
explore a real end-to-end trainable approach for image retrieval, i.e. we learn
the weights from the input image pixels to the final similarity score.
Experimental evaluation shows that non-metric similarity networks are able to
learn visual similarities between images and improve performance on top of
state-of-the-art image representations, boosting results in standard image
retrieval datasets with respect standard metric distances.
Complexity Classification in Infinite-Domain Constraint Satisfaction
Authors: Manuel Bodirsky
Date: 2012-01-04 00:00:00
Tasks: Classification, General Classification, Spatial Reasoning

A constraint satisfaction problem (CSP) is a computational problem where the
input consists of a finite set of variables and a finite set of constraints,
and where the task is to decide whether there exists a satisfying assignment of
values to the variables. Depending on the type of constraints that we allow in
the input, a CSP might be tractable, or computationally hard. In recent years,
general criteria have been discovered that imply that a CSP is polynomial-time
tractable, or that it is NP-hard. Finite-domain CSPs have become a major common
research focus of graph theory, artificial intelligence, and finite model
theory. It turned out that the key questions for complexity classification of
CSPs are closely linked to central questions in universal algebra.
  This thesis studies CSPs where the variables can take values from an infinite
domain. This generalization enhances dramatically the range of computational
problems that can be modeled as a CSP. Many problems from areas that have so
far seen no interaction with constraint satisfaction theory can be formulated
using infinite domains, e.g. problems from temporal and spatial reasoning,
phylogenetic reconstruction, and operations research.
  It turns out that the universal-algebraic approach can also be applied to
study large classes of infinite-domain CSPs, yielding elegant complexity
classification results. A new tool in this thesis that becomes relevant
particularly for infinite domains is Ramsey theory. We demonstrate the
feasibility of our approach with two complete complexity classification
results: one on CSPs in temporal reasoning, the other on a generalization of
Schaefer's theorem for propositional logic to logic over graphs. We also study
the limits of complexity classification, and present classes of computational
problems provably do not exhibit a complexity dichotomy into hard and easy
problems.
Reinforcement Learning-based Thermal Comfort Control for Vehicle Cabins
Authors: James Brusey, Diana Hintea, Elena Gaura, Neil Beloe
Date: 2017-04-25 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Vehicle climate control systems aim to keep passengers thermally comfortable.
However, current systems control temperature rather than thermal comfort and
tend to be energy hungry, which is of particular concern when considering
electric vehicles. This paper poses energy-efficient vehicle comfort control as
a Markov Decision Process, which is then solved numerically using
Sarsa({\lambda}) and an empirically validated, single-zone, 1D thermal model of
the cabin. The resulting controller was tested in simulation using 200 randomly
selected scenarios and found to exceed the performance of bang-bang,
proportional, simple fuzzy logic, and commercial controllers with 23%, 43%,
40%, 56% increase, respectively. Compared to the next best performing
controller, energy consumption is reduced by 13% while the proportion of time
spent thermally comfortable is increased by 23%. These results indicate that
this is a viable approach that promises to translate into substantial comfort
and energy improvements in the car.
Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model
Authors: Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, Rita Cucchiara
Date: 2016-11-29 00:00:00
Tasks: Saliency Prediction

Data-driven saliency has recently gained a lot of attention thanks to the use
of Convolutional Neural Networks for predicting gaze fixations. In this paper
we go beyond standard approaches to saliency prediction, in which gaze maps are
computed with a feed-forward network, and present a novel model which can
predict accurate saliency maps by incorporating neural attentive mechanisms.
The core of our solution is a Convolutional LSTM that focuses on the most
salient regions of the input image to iteratively refine the predicted saliency
map. Additionally, to tackle the center bias typical of human eye fixations,
our model can learn a set of prior maps generated with Gaussian functions. We
show, through an extensive evaluation, that the proposed architecture
outperforms the current state of the art on public saliency prediction
datasets. We further study the contribution of each key component to
demonstrate their robustness on different scenarios.
Modeling Target-Side Inflection in Neural Machine Translation
Authors: Aleš Tamchyna, Marion Weller-Di Marco, Alexander Fraser
Date: 2017-07-19 00:00:00
Proceeding: WS 2017 9
Tasks: LEMMA, Machine Translation, NMT, POS, TAG, Translation

NMT systems have problems with large vocabulary sizes. Byte-pair encoding
(BPE) is a popular approach to solving this problem, but while BPE allows the
system to generate any target-side word, it does not enable effective
generalization over the rich vocabulary in morphologically rich languages with
strong inflectional phenomena. We introduce a simple approach to overcome this
problem by training a system to produce the lemma of a word and its
morphologically rich POS tag, which is then followed by a deterministic
generation step. We apply this strategy for English-Czech and English-German
translation scenarios, obtaining improvements in both settings. We furthermore
show that the improvement is not due to only adding explicit morphological
information.
SketchParse : Towards Rich Descriptions for Poorly Drawn Sketches using Multi-Task Hierarchical Deep Networks
Authors: Ravi Kiran Sarvadevabhatla, Isht Dwivedi, Abhijat Biswas, Sahil Manocha, R. Venkatesh Babu
Date: 2017-09-05 00:00:00
Tasks: Image Retrieval, Object, Pose Prediction, Retrieval, Sketch-Based Image Retrieval

The ability to semantically interpret hand-drawn line sketches, although very
challenging, can pave way for novel applications in multimedia. We propose
SketchParse, the first deep-network architecture for fully automatic parsing of
freehand object sketches. SketchParse is configured as a two-level fully
convolutional network. The first level contains shared layers common to all
object categories. The second level contains a number of expert sub-networks.
Each expert specializes in parsing sketches from object categories which
contain structurally similar parts. Effectively, the two-level configuration
enables our architecture to scale up efficiently as additional categories are
added. We introduce a router layer which (i) relays sketch features from shared
layers to the correct expert (ii) eliminates the need to manually specify
object category during inference. To bypass laborious part-level annotation, we
sketchify photos from semantic object-part image datasets and use them for
training. Our architecture also incorporates object pose prediction as a novel
auxiliary task which boosts overall performance while providing supplementary
information regarding the sketch. We demonstrate SketchParse's abilities (i) on
two challenging large-scale sketch datasets (ii) in parsing unseen,
semantically related object categories (iii) in improving fine-grained
sketch-based image retrieval. As a novel application, we also outline how
SketchParse's output can be used to generate caption-style descriptions for
hand-drawn sketches.
Fast Adaptation in Generative Models with Generative Matching Networks
Authors: Sergey Bartunov, Dmitry P. Vetrov
Date: 2016-12-07 00:00:00
Tasks: Diversity, One-Shot Learning

Despite recent advances, the remaining bottlenecks in deep generative models
are necessity of extensive training and difficulties with generalization from
small number of training examples. We develop a new generative model called
Generative Matching Network which is inspired by the recently proposed matching
networks for one-shot learning in discriminative tasks. By conditioning on the
additional input dataset, our model can instantly learn new concepts that were
not available in the training data but conform to a similar generative process.
The proposed framework does not explicitly restrict diversity of the
conditioning data and also does not require an extensive inference procedure
for training or adaptation. Our experiments on the Omniglot dataset demonstrate
that Generative Matching Networks significantly improve predictive performance
on the fly as more additional data is available and outperform existing state
of the art conditional generative models.
Newton-type Methods for Inference in Higher-Order Markov Random Fields
Authors: Hariprasad Kannan, Nikos Komodakis, Nikos Paragios
Date: 2017-09-05 00:00:00
Proceeding: CVPR 2017 7
Tasks: Vocal Bursts Type Prediction

Linear programming relaxations are central to {\sc map} inference in discrete
Markov Random Fields. The ability to properly solve the Lagrangian dual is a
critical component of such methods. In this paper, we study the benefit of
using Newton-type methods to solve the Lagrangian dual of a smooth version of
the problem. We investigate their ability to achieve superior convergence
behavior and to better handle the ill-conditioned nature of the formulation, as
compared to first order methods. We show that it is indeed possible to
efficiently apply a trust region Newton method for a broad range of {\sc map}
inference problems. In this paper we propose a provably convergent and
efficient framework that includes (i) excellent compromise between
computational complexity and precision concerning the Hessian matrix
construction, (ii) a damping strategy that aids efficient optimization, (iii) a
truncation strategy coupled with a generic pre-conditioner for Conjugate
Gradients, (iv) efficient sum-product computation for sparse clique potentials.
Results for higher-order Markov Random Fields demonstrate the potential of this
approach.
Discriminative Similarity for Clustering and Semi-Supervised Learning
Authors: Yingzhen Yang, Feng Liang, Nebojsa Jojic, Shuicheng Yan, Jiashi Feng, Thomas S. Huang
Date: 2017-09-05 00:00:00
Tasks: Clustering

Similarity-based clustering and semi-supervised learning methods separate the
data into clusters or classes according to the pairwise similarity between the
data, and the pairwise similarity is crucial for their performance. In this
paper, we propose a novel discriminative similarity learning framework which
learns discriminative similarity for either data clustering or semi-supervised
learning. The proposed framework learns classifier from each hypothetical
labeling, and searches for the optimal labeling by minimizing the
generalization error of the learned classifiers associated with the
hypothetical labeling. Kernel classifier is employed in our framework. By
generalization analysis via Rademacher complexity, the generalization error
bound for the kernel classifier learned from hypothetical labeling is expressed
as the sum of pairwise similarity between the data from different classes,
parameterized by the weights of the kernel classifier. Such pairwise similarity
serves as the discriminative similarity for the purpose of clustering and
semi-supervised learning, and discriminative similarity with similar form can
also be induced by the integrated squared error bound for kernel density
classification. Based on the discriminative similarity induced by the kernel
classifier, we propose new clustering and semi-supervised learning methods.
On the Suboptimality of Proximal Gradient Descent for $\ell^{0}$ Sparse Approximation
Authors: Yingzhen Yang, Jiashi Feng, Nebojsa Jojic, Jianchao Yang, Thomas S. Huang
Date: 2017-09-05 00:00:00
Tasks: Compressive Sensing, Dimensionality Reduction

We study the proximal gradient descent (PGD) method for $\ell^{0}$ sparse
approximation problem as well as its accelerated optimization with randomized
algorithms in this paper. We first offer theoretical analysis of PGD showing
the bounded gap between the sub-optimal solution by PGD and the globally
optimal solution for the $\ell^{0}$ sparse approximation problem under
conditions weaker than Restricted Isometry Property widely used in compressive
sensing literature. Moreover, we propose randomized algorithms to accelerate
the optimization by PGD using randomized low rank matrix approximation
(PGD-RMA) and randomized dimension reduction (PGD-RDR). Our randomized
algorithms substantially reduces the computation cost of the original PGD for
the $\ell^{0}$ sparse approximation problem, and the resultant sub-optimal
solution still enjoys provable suboptimality, namely, the sub-optimal solution
to the reduced problem still has bounded gap to the globally optimal solution
to the original problem.
Learning Local Shape Descriptors from Part Correspondences With Multi-view Convolutional Networks
Authors: Haibin Huang, Evangelos Kalogerakis, Siddhartha Chaudhuri, Duygu Ceylan, Vladimir G. Kim, Ersin Yumer
Date: 2017-06-14 00:00:00
Tasks: Semantic Segmentation

We present a new local descriptor for 3D shapes, directly applicable to a
wide range of shape analysis problems such as point correspondences, semantic
segmentation, affordance prediction, and shape-to-scan matching. The descriptor
is produced by a convolutional network that is trained to embed geometrically
and semantically similar points close to one another in descriptor space. The
network processes surface neighborhoods around points on a shape that are
captured at multiple scales by a succession of progressively zoomed out views,
taken from carefully selected camera positions. We leverage two extremely large
sources of data to train our network. First, since our network processes
rendered views in the form of 2D images, we repurpose architectures pre-trained
on massive image datasets. Second, we automatically generate a synthetic dense
point correspondence dataset by non-rigid alignment of corresponding shape
parts in a large collection of segmented 3D models. As a result of these design
choices, our network effectively encodes multi-scale local context and
fine-grained surface detail. Our network can be trained to produce either
category-specific descriptors or more generic descriptors by learning from
multiple shape categories. Once trained, at test time, the network extracts
local descriptors for shapes without requiring any part segmentation as input.
Our method can produce effective local descriptors even for shapes whose
category is unknown or different from the ones used while training. We
demonstrate through several experiments that our learned local descriptors are
more discriminative compared to state of the art alternatives, and are
effective in a variety of shape analysis applications.
Multi-Modal Multi-Scale Deep Learning for Large-Scale Image Annotation
Authors: Yulei Niu, Zhiwu Lu, Ji-Rong Wen, Tao Xiang, Shih-Fu Chang
Date: 2017-09-05 00:00:00
Tasks: Deep Learning

Image annotation aims to annotate a given image with a variable number of
class labels corresponding to diverse visual concepts. In this paper, we
address two main issues in large-scale image annotation: 1) how to learn a rich
feature representation suitable for predicting a diverse set of visual concepts
ranging from object, scene to abstract concept; 2) how to annotate an image
with the optimal number of class labels. To address the first issue, we propose
a novel multi-scale deep model for extracting rich and discriminative features
capable of representing a wide range of visual concepts. Specifically, a novel
two-branch deep neural network architecture is proposed which comprises a very
deep main network branch and a companion feature fusion network branch designed
for fusing the multi-scale features computed from the main branch. The deep
model is also made multi-modal by taking noisy user-provided tags as model
input to complement the image input. For tackling the second issue, we
introduce a label quantity prediction auxiliary task to the main label
prediction task to explicitly estimate the optimal label number for a given
image. Extensive experiments are carried out on two large-scale image
annotation benchmark datasets and the results show that our method
significantly outperforms the state-of-the-art.
Language Modeling by Clustering with Word Embeddings for Text Readability Assessment
Authors: Miriam Cha, Youngjune Gwon, H. T. Kung
Date: 2017-09-05 00:00:00
Tasks: Clustering, Language Modeling, Language Modelling, Sentence, Word Embeddings

We present a clustering-based language model using word embeddings for text
readability prediction. Presumably, an Euclidean semantic space hypothesis
holds true for word embeddings whose training is done by observing word
co-occurrences. We argue that clustering with word embeddings in the metric
space should yield feature representations in a higher semantic space
appropriate for text regression. Also, by representing features in terms of
histograms, our approach can naturally address documents of varying lengths. An
empirical evaluation using the Common Core Standards corpus reveals that the
features formed on our clustering-based language model significantly improve
the previously known results for the same corpus in readability prediction. We
also evaluate the task of sentence matching based on semantic relatedness using
the Wiki-SimpleWiki corpus and find that our features lead to superior matching
performance.