pwc-archive/papers-with-abstracts huggingface dataset:
How Much Chemistry Does a Deep Neural Network Need to Know to Make Accurate Predictions?
Authors: Garrett B. Goh, Charles Siegel, Abhinav Vishnu, Nathan O. Hodas, Nathan Baker
Date: 2017-10-05 00:00:00
Tasks: Representation Learning

The meteoric rise of deep learning models in computer vision research, having
achieved human-level accuracy in image recognition tasks is firm evidence of
the impact of representation learning of deep neural networks. In the chemistry
domain, recent advances have also led to the development of similar CNN models,
such as Chemception, that is trained to predict chemical properties using
images of molecular drawings. In this work, we investigate the effects of
systematically removing and adding localized domain-specific information to the
image channels of the training data. By augmenting images with only 3
additional basic information, and without introducing any architectural
changes, we demonstrate that an augmented Chemception (AugChemception)
outperforms the original model in the prediction of toxicity, activity, and
solvation free energy. Then, by altering the information content in the images,
and examining the resulting model's performance, we also identify two distinct
learning patterns in predicting toxicity/activity as compared to solvation free
energy. These patterns suggest that Chemception is learning about its tasks in
the manner that is consistent with established knowledge. Thus, our work
demonstrates that advanced chemical knowledge is not a pre-requisite for deep
learning models to accurately predict complex chemical properties.
SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Predicting Chemical Properties
Authors: Garrett B. Goh, Nathan O. Hodas, Charles Siegel, Abhinav Vishnu
Date: 2017-12-06 00:00:00
Tasks: Bayesian Optimization, Feature Engineering

Chemical databases store information in text representations, and the SMILES
format is a universal standard used in many cheminformatics software. Encoded
in each SMILES string is structural information that can be used to predict
complex chemical properties. In this work, we develop SMILES2vec, a deep RNN
that automatically learns features from SMILES to predict chemical properties,
without the need for additional explicit feature engineering. Using Bayesian
optimization methods to tune the network architecture, we show that an
optimized SMILES2vec model can serve as a general-purpose neural network for
predicting distinct chemical properties including toxicity, activity,
solubility and solvation energy, while also outperforming contemporary MLP
neural networks that uses engineered features. Furthermore, we demonstrate
proof-of-concept of interpretability by developing an explanation mask that
localizes on the most important characters used in making a prediction. When
tested on the solubility dataset, it identified specific parts of a chemical
that is consistent with established first-principles knowledge with an accuracy
of 88%. Our work demonstrates that neural networks can learn technically
accurate chemical concept and provide state-of-the-art accuracy, making
interpretable deep neural networks a useful tool of relevance to the chemical
industry.
Using Rule-Based Labels for Weak Supervised Learning: A ChemNet for Transferable Chemical Property Prediction
Authors: Garrett B. Goh, Charles Siegel, Abhinav Vishnu, Nathan O. Hodas
Date: 2017-12-07 00:00:00
Tasks: Property Prediction, speech-recognition, Speech Recognition, Transfer Learning

With access to large datasets, deep neural networks (DNN) have achieved
human-level accuracy in image and speech recognition tasks. However, in
chemistry, data is inherently small and fragmented. In this work, we develop an
approach of using rule-based knowledge for training ChemNet, a transferable and
generalizable deep neural network for chemical property prediction that learns
in a weak-supervised manner from large unlabeled chemical databases. When
coupled with transfer learning approaches to predict other smaller datasets for
chemical properties that it was not originally trained on, we show that
ChemNet's accuracy outperforms contemporary DNN models that were trained using
conventional supervised learning. Furthermore, we demonstrate that the ChemNet
pre-training approach is equally effective on both CNN (Chemception) and RNN
(SMILES2vec) models, indicating that this approach is network architecture
agnostic and is effective across multiple data modalities. Our results indicate
a pre-trained ChemNet that incorporates chemistry domain knowledge, enables the
development of generalizable neural networks for more accurate prediction of
novel chemical properties.
Quasi-homography warps in image stitching
Authors: Nan Li, Yifang Xu, Chao Wang
Date: 2017-01-27 00:00:00
Tasks: Image Stitching

The naturalness of warps is gaining extensive attentions in image stitching.
Recent warps such as SPHP and AANAP, use global similarity warps to mitigate
projective distortion (which enlarges regions), however, they necessarily bring
in perspective distortion (which generates inconsistencies). In this paper, we
propose a novel quasi-homography warp, which effectively balances the
perspective distortion against the projective distortion in the non-overlapping
region to create a more natural-looking panorama. Our approach formulates the
warp as the solution of a bivariate system, where perspective distortion and
projective distortion are characterized as slope preservation and scale
linearization respectively. Because our proposed warp only relies on a global
homography, thus it is totally parameter-free. A comprehensive experiment shows
that a quasi-homography warp outperforms some state-of-the-art warps in urban
scenes, including homography, AutoStitch and SPHP. A user study demonstrates
that it wins most users' favor, comparing to homography and SPHP.
Land use mapping in the Three Gorges Reservoir Area based on semantic segmentation deep learning method
Authors: Xin Zhang, Bingfang Wu, Liang Zhu, Fuyou Tian, Miao Zhang, Yuanzeng
Date: 2018-03-18 00:00:00
Tasks: Semantic Segmentation

The Three Gorges Dam, a massive cross-century project spans the Yangtze River
by the town of Sandouping, located in Yichang, Hubei province, China, was built
to provide great power, improve the River shipping, control floods in the upper
reaches of the Yangtze River, and increase the dry season flow in the middle
and lower reaches of the Yangtze River. Benefits are enormous and
comprehensive. However, the social and environmental impacts are also immense
and far-reaching to its surrounding areas. Mapping land use /land cover changed
(LUCC) is critical for tracking the impacts. Remote sensing has been proved to
be an effective way to map and monitor land use change in real time and in
large areas such as the Three Gorges Reservoir Area(TGRA) by using pixel based
or oriented based classifier in different resolution. In this paper, we first
test the state of the art semantic segmentation deep learning classifiers for
LUCC mapping with 7 categories in the TGRA area with rapideye 5m resolution
data. The topographic information was also added for better accuracy in
mountain area. By compared with the pixel-based classifier, the semantic
segmentation deep learning method has better accuracy and robustness at 5m
resolution level.
Sdf-GAN: Semi-supervised Depth Fusion with Multi-scale Adversarial Networks
Authors: Can Pu, Runzi Song, Radim Tylecek, Nanbo Li, Robert B. Fisher
Date: 2018-03-18 00:00:00
Tasks: Setting-1/2

Refining raw disparity maps from different algorithms to exploit their complementary advantages is still challenging. Uncertainty estimation and complex disparity relationships among pixels limit the accuracy and robustness of existing methods and there is no standard method for fusion of different kinds of depth data. In this paper, we introduce a new method to fuse disparity maps from different sources, while incorporating supplementary information (intensity, gradient, etc.) into a refiner network to better refine raw disparity inputs. A discriminator network classifies disparities at different receptive fields and scales. Assuming a Markov Random Field for the refined disparity map produces better estimates of the true disparity distribution. Both fully supervised and semi-supervised versions of the algorithm are proposed. The approach includes a more robust loss function to inpaint invalid disparity values and requires much less labeled data to train in the semi-supervised learning mode. The algorithm can be generalized to fuse depths from different kinds of depth sources. Experiments explored different fusion opportunities: stereo-monocular fusion, stereo-ToF fusion and stereo-stereo fusion. The experiments show the superiority of the proposed algorithm compared with the most recent algorithms on public synthetic datasets (Scene Flow, SYNTH3, our synthetic garden dataset) and real datasets (Kitti2015 dataset and Trimbot2020 Garden dataset).
Ratio-Preserving Half-Cylindrical Warps for Natural Image Stitching
Authors: Yifang Xu, Jing Chen, Tianli Liao
Date: 2018-03-18 00:00:00
Tasks: Image Stitching

A novel warp for natural image stitching is proposed that utilizes the
property of cylindrical warp and a horizontal pixel selection strategy. The
proposed ratio-preserving half-cylindrical warp is a combination of homography
and cylindrical warps which guarantees alignment by homography and possesses
less projective distortion by cylindrical warp. Unlike previous approaches
applying cylindrical warp before homography, we use partition lines to divide
the image into different parts and apply homography in the overlapping region
while a composition of homography and cylindrical warps in the non-overlapping
region. The pixel selection strategy then samples the points in horizontal and
reconstructs the image via interpolation to further reduce horizontal
distortion by maintaining the ratio as similarity. With applying
half-cylindrical warp and horizontal pixel selection, the projective distortion
in vertical and horizontal is mitigated simultaneously. Experiments show that
our warp is efficient and produces a more natural-looking stitched result than
previous methods.
DUGMA: Dynamic Uncertainty-Based Gaussian Mixture Alignment
Authors: Can Pu, Nanbo Li, Radim Tylecek, Robert B. Fisher
Date: 2018-03-18 00:00:00

Registering accurately point clouds from a cheap low-resolution sensor is a
challenging task. Existing rigid registration methods failed to use the
physical 3D uncertainty distribution of each point from a real sensor in the
dynamic alignment process mainly because the uncertainty model for a point is
static and invariant and it is hard to describe the change of these physical
uncertainty models in the registration process. Additionally, the existing
Gaussian mixture alignment architecture cannot be efficiently implement these
dynamic changes.
  This paper proposes a simple architecture combining error estimation from
sample covariances and dual dynamic global probability alignment using the
convolution of uncertainty-based Gaussian Mixture Models (GMM) from point
clouds. Firstly, we propose an efficient way to describe the change of each 3D
uncertainty model, which represents the structure of the point cloud much
better. Unlike the invariant GMM (representing a fixed point cloud) in
traditional Gaussian mixture alignment, we use two uncertainty-based GMMs that
change and interact with each other in each iteration. In order to have a wider
basin of convergence than other local algorithms, we design a more robust
energy function by convolving efficiently the two GMMs over the whole 3D space.
  Tens of thousands of trials have been conducted on hundreds of models from
multiple datasets to demonstrate the proposed method's superior performance
compared with the current state-of-the-art methods. The new dataset and code is
available from https://github.com/Canpu999
Line Artist: A Multiple Style Sketch to Painting Synthesis Scheme
Authors: Jinning Li, Si-Qi Liu, Mengyao Cao
Date: 2018-03-18 00:00:00
Tasks: Generative Adversarial Network, Image Generation, Style Transfer

Drawing a beautiful painting is a dream of many people since childhood. In
this paper, we propose a novel scheme, Line Artist, to synthesize artistic
style paintings with freehand sketch images, leveraging the power of deep
learning and advanced algorithms. Our scheme includes three models. The Sketch
Image Extraction (SIE) model is applied to generate the training data. It
includes smoothing reality images and pencil sketch extraction. The Detailed
Image Synthesis (DIS) model trains a conditional generative adversarial network
to generate detailed real-world information. The Adaptively Weighted Artistic
Style Transfer (AWAST) model is capable to combine multiple style images with a
content with the VGG19 network and PageRank algorithm. The appealing artistic
images are then generated by optimization iterations. Experiments are operated
on the Kaggle Cats dataset and The Oxford Buildings Dataset. Our synthesis
results are proved to be artistic, beautiful and robust.
Computing and Testing Pareto Optimal Committees
Authors: Haris Aziz, Jerome Lang, Jerome Monnot
Date: 2018-03-18 00:00:00

Selecting a set of alternatives based on the preferences of agents is an
important problem in committee selection and beyond. Among the various criteria
put forth for the desirability of a committee, Pareto optimality is a minimal
and important requirement. As asking agents to specify their preferences over
exponentially many subsets of alternatives is practically infeasible, we assume
that each agent specifies a weak order on single alternatives, from which a
preference relation over subsets is derived using some preference extension. We
consider five prominent extensions (responsive, downward lexicographic, upward
lexicographic, best, and worst). For each of them, we consider the
corresponding Pareto optimality notion, and we study the complexity of
computing and verifying Pareto optimal outcomes. We also consider strategic
issues: for four of the set extensions, we present a linear-time, Pareto
optimal and strategyproof algorithm that even works for weak preferences.
The Web as a Knowledge-base for Answering Complex Questions
Authors: Alon Talmor, Jonathan Berant
Date: 2018-03-18 00:00:00
Proceeding: NAACL 2018 6
Tasks: Reading Comprehension

Answering complex questions is a time-consuming activity for humans that
requires reasoning and integration of information. Recent work on reading
comprehension made headway in answering simple questions, but tackling complex
questions is still an ongoing research challenge. Conversely, semantic parsers
have been successful at handling compositionality, but only when the
information resides in a target knowledge-base. In this paper, we present a
novel framework for answering broad and complex questions, assuming answering
simple questions is possible using a search engine and a reading comprehension
model. We propose to decompose complex questions into a sequence of simple
questions, and compute the final answer from the sequence of answers. To
illustrate the viability of our approach, we create a new dataset of complex
questions, ComplexWebQuestions, and present a model that decomposes questions
and interacts with the web to compute an answer. We empirically demonstrate
that question decomposition improves performance from 20.8 precision@1 to 27.5
precision@1 on this new dataset.
Zoom and Learn: Generalizing Deep Stereo Matching to Novel Domains
Authors: Jiahao Pang, Wenxiu Sun, Chengxi Yang, Jimmy Ren, Ruichao Xiao, Jin Zeng, Liang Lin
Date: 2018-03-18 00:00:00
Proceeding: CVPR 2018 6
Tasks: Stereo Matching, Stereo Matching Hand

Despite the recent success of stereo matching with convolutional neural
networks (CNNs), it remains arduous to generalize a pre-trained deep stereo
model to a novel domain. A major difficulty is to collect accurate ground-truth
disparities for stereo pairs in the target domain. In this work, we propose a
self-adaptation approach for CNN training, utilizing both synthetic training
data (with ground-truth disparities) and stereo pairs in the new domain
(without ground-truths). Our method is driven by two empirical observations. By
feeding real stereo pairs of different domains to stereo models pre-trained
with synthetic data, we see that: i) a pre-trained model does not generalize
well to the new domain, producing artifacts at boundaries and ill-posed
regions; however, ii) feeding an up-sampled stereo pair leads to a disparity
map with extra details. To avoid i) while exploiting ii), we formulate an
iterative optimization problem with graph Laplacian regularization. At each
iteration, the CNN adapts itself better to the new domain: we let the CNN learn
its own higher-resolution output; at the meanwhile, a graph Laplacian
regularization is imposed to discriminatively keep the desired edges while
smoothing out the artifacts. We demonstrate the effectiveness of our method in
two domains: daily scenes collected by smartphone cameras, and street views
captured in a driving car.
The Automatic Identification of Butterfly Species
Authors: Juanying Xie, Qi Hou, Yinghuan Shi, Lv Peng, Liping Jing, Fuzhen Zhuang, Junping Zhang, Xiaoyang Tang, Shengquan Xu
Date: 2018-03-18 00:00:00

The available butterfly data sets comprise a few limited species, and the
images in the data sets are always standard patterns without the images of
butterflies in their living environment. To overcome the aforementioned
limitations in the butterfly data sets, we build a butterfly data set composed
of all species of butterflies in China with 4270 standard pattern images of
1176 butterfly species, and 1425 images from living environment of 111 species.
We propose to use the deep learning technique Faster-Rcnn to train an automatic
butterfly identification system including butterfly position detection and
species recognition. We delete those species with only one living environment
image from data set, then partition the rest images from living environment
into two subsets, one used as test subset, the other as training subset
respectively combined with all standard pattern butterfly images or the
standard pattern butterfly images with the same species of the images from
living environment. In order to construct the training subset for FasterRcnn,
nine methods were adopted to amplifying the images in the training subset
including the turning of up and down, and left and right, rotation with
different angles, adding noises, blurring, and contrast ratio adjusting etc.
Three prediction models were trained. The mAP (Mean Average prediction)
criterion was used to evaluate the performance of the prediction model. The
experimental results demonstrate that our Faster-Rcnn based butterfly automatic
identification system performed well, and its worst mAP is up to 60%, and can
simultaneously detect the positions of more than one butterflies in one images
from living environment and recognize the species of those butterflies as well.
Visual Text Correction
Authors: Amir Mazaheri, Mubarak Shah
Date: 2018-01-06 00:00:00
Proceeding: ECCV 2018 9
Tasks: Grammatical Error Correction, Sentence, Visual Text Correction

Videos, images, and sentences are mediums that can express the same
semantics. One can imagine a picture by reading a sentence or can describe a
scene with some words. However, even small changes in a sentence can cause a
significant semantic inconsistency with the corresponding video/image. For
example, by changing the verb of a sentence, the meaning may drastically
change. There have been many efforts to encode a video/sentence and decode it
as a sentence/video. In this research, we study a new scenario in which both
the sentence and the video are given, but the sentence is inaccurate. A
semantic inconsistency between the sentence and the video or between the words
of a sentence can result in an inaccurate description. This paper introduces a
new problem, called Visual Text Correction (VTC), i.e., finding and replacing
an inaccurate word in the textual description of a video. We propose a deep
network that can simultaneously detect an inaccuracy in a sentence, and fix it
by replacing the inaccurate word(s). Our method leverages the semantic
interdependence of videos and words, as well as the short-term and long-term
relations of the words in a sentence. In our formulation, part of a visual
feature vector for every single word is dynamically selected through a gating
process. Furthermore, to train and evaluate our model, we propose an approach
to automatically construct a large dataset for VTC problem. Our experiments and
performance analysis demonstrates that the proposed method provides very good
results and also highlights the general challenges in solving the VTC problem.
To the best of our knowledge, this work is the first of its kind for the Visual
Text Correction task.
Learning recurrent dynamics in spiking networks
Authors: Christopher Kim, Carson Chow
Date: 2018-03-18 00:00:00

Spiking activity of neurons engaged in learning and performing a task show
complex spatiotemporal dynamics. While the output of recurrent network models
can learn to perform various tasks, the possible range of recurrent dynamics
that emerge after learning remains unknown. Here we show that modifying the
recurrent connectivity with a recursive least squares algorithm provides
sufficient flexibility for synaptic and spiking rate dynamics of spiking
networks to produce a wide range of spatiotemporal activity. We apply the
training method to learn arbitrary firing patterns, stabilize irregular spiking
activity of a balanced network, and reproduce the heterogeneous spiking rate
patterns of cortical neurons engaged in motor planning and movement. We
identify sufficient conditions for successful learning, characterize two types
of learning errors, and assess the network capacity. Our findings show that
synaptically-coupled recurrent spiking networks possess a vast computational
capability that can support the diverse activity patterns in the brain.
Trajectory-based Scene Understanding using Dirichlet Process Mixture Model
Authors: Santhosh Kelathodi Kumaran, Debi Prosad Dogra, Partha Pratim Roy, Bidyut Baran Chaudhuri
Date: 2018-03-18 00:00:00
Tasks: Clustering, Decision Making, Scene Understanding

Appropriate modeling of a surveillance scene is essential for detection of anomalies in road traffic. Learning usual paths can provide valuable insight into road traffic conditions and thus can help in identifying unusual routes taken by commuters/vehicles. If usual traffic paths are learned in a nonparametric way, manual interventions in road marking road can be avoided. In this paper, we propose an unsupervised and nonparametric method to learn frequently used paths from the tracks of moving objects in $\Theta(kn)$ time, where $k$ denotes the number of paths and $n$ represents the number of tracks. In the proposed method, temporal dependencies of the moving objects are considered to make the clustering meaningful using Temporally Incremental Gravity Model (TIGM). In addition, the distance-based scene learning makes it intuitive to estimate the model parameters. Further, we have extended TIGM hierarchically as Dynamically Evolving Model (DEM) to represent notable traffic dynamics of a scene. Experimental validation reveals that the proposed method can learn a scene quickly without prior knowledge about the number of paths ($k$). We have compared the results with various state-of-the-art methods. We have also highlighted the advantages of the proposed method over existing techniques popularly used for designing traffic monitoring applications. It can be used for administrative decision making to control traffic at junctions or crowded places and generate alarm signals, if necessary.