pwc-archive/papers-with-abstracts huggingface dataset:
Not just a matter of semantics: the relationship between visual similarity and semantic similarity
Authors: Clemens-Alexander Brust, Joachim Denzler
Date: 2018-11-17 00:00:00
Tasks: Image Retrieval, Retrieval, Semantic Similarity, Semantic Textual Similarity, Transfer Learning, Zero-Shot Learning

Knowledge transfer, zero-shot learning and semantic image retrieval are methods that aim at improving accuracy by utilizing semantic information, e.g. from WordNet. It is assumed that this information can augment or replace missing visual data in the form of labeled training images because semantic similarity correlates with visual similarity. This assumption may seem trivial, but is crucial for the application of such semantic methods. Any violation can cause mispredictions. Thus, it is important to examine the visual-semantic relationship for a certain target problem. In this paper, we use five different semantic and visual similarity measures each to thoroughly analyze the relationship without relying too much on any single definition. We postulate and verify three highly consequential hypotheses on the relationship. Our results show that it indeed exists and that WordNet semantic similarity carries more information about visual similarity than just the knowledge of "different classes look different". They suggest that classification is not the ideal application for semantic methods and that wrong semantic information is much worse than none.
Unsupervised Online Learning With Multiple Postsynaptic Neurons Based on Spike-Timing-Dependent Plasticity Using a TFT-Type NOR Flash Memory Array
Authors: Soochang Lee, Chul-Heung Kim, Seongbin Oh, Byung-Gook Park, Jong-Ho Lee
Date: 2018-11-17 00:00:00
Tasks: General Classification

We present a two-layer fully connected neuromorphic system based on a
thin-film transistor (TFT)-type NOR flash memory array with multiple
postsynaptic (POST) neurons. Unsupervised online learning by
spike-timing-dependent plasticity (STDP) on the binary MNIST handwritten
datasets is implemented, and its recognition result is determined by measuring
firing rate of POST neurons. Using a proposed learning scheme, we investigate
the impact of the number of POST neurons in terms of recognition rate. In this
neuromorphic system, lateral inhibition function and homeostatic property are
exploited for competitive learning of multiple POST neurons. The simulation
results demonstrate unsupervised online learning of the full black-and-white
MNIST handwritten digits by STDP, which indicates the performance of pattern
recognition and classification without preprocessing of input patterns.
Augmented LiDAR Simulator for Autonomous Driving
Authors: Jin Fang, Dingfu Zhou, Feilong Yan, Tongtong Zhao, Feihu Zhang, Yu Ma, Liang Wang, Ruigang Yang
Date: 2018-11-17 00:00:00
Tasks: Autonomous Driving

In Autonomous Driving (AD), detection and tracking of obstacles on the roads
is a critical task. Deep-learning based methods using annotated LiDAR data have
been the most widely adopted approach for this. Unfortunately, annotating 3D
point cloud is a very challenging, time- and money-consuming task. In this
paper, we propose a novel LiDAR simulator that augments real point cloud with
synthetic obstacles (e.g., cars, pedestrians, and other movable objects).
Unlike previous simulators that entirely rely on CG models and game engines,
our augmented simulator bypasses the requirement to create high-fidelity
background CAD models. Instead, we can simply deploy a vehicle with a LiDAR
scanner to sweep the street of interests to obtain the background point cloud,
based on which annotated point cloud can be automatically generated. This
unique "scan-and-simulate" capability makes our approach scalable and
practical, ready for large-scale industrial applications. In this paper, we
describe our simulator in detail, in particular the placement of obstacles that
is critical for performance enhancement. We show that detectors with our
simulated LiDAR point cloud alone can perform comparably (within two percentage
points) with these trained with real data. Mixing real and simulated data can
achieve over 95% accuracy.
Boosting the Robustness Verification of DNN by Identifying the Achilles's Heel
Authors: Chengdong Feng, Zhenbang Chen, Weijiang Hong, Hengbiao Yu, Wei Dong, Ji Wang
Date: 2018-11-17 00:00:00

Deep Neural Network (DNN) is a widely used deep learning technique. How to
ensure the safety of DNN-based system is a critical problem for the research
and application of DNN. Robustness is an important safety property of DNN.
However, existing work of verifying DNN's robustness is time-consuming and hard
to scale to large-scale DNNs. In this paper, we propose a boosting method for
DNN robustness verification, aiming to find counter-examples earlier. Our
observation is DNN's different inputs have different possibilities of existing
counter-examples around them, and the input with a small difference between the
largest output value and the second largest output value tends to be the
achilles's heel of the DNN. We have implemented our method and applied it on
Reluplex, a state-of-the-art DNN verification tool, and four DNN attacking
methods. The results of the extensive experiments on two benchmarks indicate
the effectiveness of our boosting method.
On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs
Authors: Sandipan Banerjee, Walter J. Scheirer, Kevin W. Bowyer, Patrick J. Flynn
Date: 2018-11-17 00:00:00
Tasks: Face Swapping, Facial Inpainting, Hallucination

We propose a multi-scale GAN model to hallucinate realistic context (forehead, hair, neck, clothes) and background pixels automatically from a single input face mask. Instead of swapping a face on to an existing picture, our model directly generates realistic context and background pixels based on the features of the provided face mask. Unlike face inpainting algorithms, it can generate realistic hallucinations even for a large number of missing pixels. Our model is composed of a cascaded network of GAN blocks, each tasked with hallucination of missing pixels at a particular resolution while guiding the synthesis process of the next GAN block. The hallucinated full face image is made photo-realistic by using a combination of reconstruction, perceptual, adversarial and identity preserving losses at each block of the network. With a set of extensive experiments, we demonstrate the effectiveness of our model in hallucinating context and background pixels from face masks varying in facial pose, expression and lighting, collected from multiple datasets subject disjoint with our training data. We also compare our method with two popular face swapping and face completion methods in terms of visual quality and recognition performance. Additionally, we analyze our cascaded pipeline and compare it with the recently proposed progressive growing of GANs.
Cross-modality deep learning brings bright-field microscopy contrast to holography
Authors: Yichen Wu, Yilin Luo, Gunvant Chaudhari, Yair Rivenson, Ayfer Calis, Kevin De Haan, Aydogan Ozcan
Date: 2018-11-17 00:00:00
Tasks: Deep Learning

Deep learning brings bright-field microscopy contrast to holographic images
of a sample volume, bridging the volumetric imaging capability of holography
with the speckle- and artifact-free image contrast of bright-field incoherent
microscopy.
RelationNet2: Deep Comparison Columns for Few-Shot Learning
Authors: Xueting Zhang, Yu-ting Qiang, Flood Sung, Yongxin Yang, Timothy M. Hospedales
Date: 2018-11-17 00:00:00
Tasks: Few-Shot Image Classification, Few-Shot Learning, Metric Learning

Few-shot deep learning is a topical challenge area for scaling visual recognition to open ended growth of unseen new classes with limited labeled examples. A promising approach is based on metric learning, which trains a deep embedding to support image similarity matching. Our insight is that effective general purpose matching requires non-linear comparison of features at multiple abstraction levels. We thus propose a new deep comparison network comprised of embedding and relation modules that learn multiple non-linear distance metrics based on different levels of features simultaneously. Furthermore, to reduce over-fitting and enable the use of deeper embeddings, we represent images as distributions rather than vectors via learning parameterized Gaussian noise regularization. The resulting network achieves excellent performance on both miniImageNet and tieredImageNet.
Sense Perception Common Sense Relationships
Authors: Ndapa Nakashole
Date: 2018-11-17 00:00:00
Tasks: Common Sense Reasoning

Often missing in existing knowledge bases of facts, are relationships that encode common sense knowledge about unnamed entities. In this paper, we propose to extract novel, common sense relationships pertaining to sense perception concepts such as sound and smell.
Learning to Address Health Inequality in the United States with a Bayesian Decision Network
Authors: Tavpritesh Sethi, Anant Mittal, Shubham Maheshwari, Samarth Chugh
Date: 2018-09-18 00:00:00

Life-expectancy is a complex outcome driven by genetic, socio-demographic,
environmental and geographic factors. Increasing socio-economic and health
disparities in the United States are propagating the longevity-gap, making it a
cause for concern. Earlier studies have probed individual factors but an
integrated picture to reveal quantifiable actions has been missing. There is a
growing concern about a further widening of healthcare inequality caused by
Artificial Intelligence (AI) due to differential access to AI-driven services.
Hence, it is imperative to explore and exploit the potential of AI for
illuminating biases and enabling transparent policy decisions for positive
social and health impact. In this work, we reveal actionable interventions for
decreasing the longevity-gap in the United States by analyzing a County-level
data resource containing healthcare, socio-economic, behavioral, education and
demographic features. We learn an ensemble-averaged structure, draw inferences
using the joint probability distribution and extend it to a Bayesian Decision
Network for identifying policy actions. We draw quantitative estimates for the
impact of diversity, preventive-care quality and stable-families within the
unified framework of our decision network. Finally, we make this analysis and
dashboard available as an interactive web-application for enabling users and
policy-makers to validate our reported findings and to explore the impact of
ones beyond reported in this work.
Unnamed Entity Recognition of Sense Mentions
Authors: Ndapa Nakashole
Date: 2018-11-17 00:00:00

We consider the problem of recognizing mentions of human senses in text. Our contribution is a method for acquiring labeled data, and a learning method that is trained on this data. Experiments show the effectiveness of our proposed data labeling approach and our learning model on the task of sense recognition in text.
Alternating Segmentation and Simulation for Contrast Adaptive Tissue Classification
Authors: Dzung L. Pham, Snehashis Roy
Date: 2018-11-17 00:00:00
Tasks: Anatomy, Classification, General Classification, Segmentation

A key feature of magnetic resonance (MR) imaging is its ability to manipulate
how the intrinsic tissue parameters of the anatomy ultimately contribute to the
contrast properties of the final, acquired image. This flexibility, however,
can lead to substantial challenges for segmentation algorithms, particularly
supervised methods. These methods require atlases or training data, which are
composed of MR image and labeled image pairs. In most cases, the training data
are obtained with a fixed acquisition protocol, leading to suboptimal
performance when an input data set that requires segmentation has differing
contrast properties. This drawback is increasingly significant with the recent
movement towards multi-center research studies involving multiple scanners and
acquisition protocols. In this work, we propose a new framework for supervised
segmentation approaches that is robust to contrast differences between the
training MR image and the input image. Our approach uses a generative
simulation model within the segmentation process to compensate for the contrast
differences. We allow the contrast of the MR image in the training data to vary
by simulating a new contrast from the corresponding label image. The model
parameters are optimized by a cost function measuring the consistency between
the input MR image and its simulation based on a current estimate of the
segmentation labels. We provide a proof of concept of this approach by
combining a supervised classifier with a simple simulation model, and apply the
resulting algorithm to synthetic images and actual MR images.
PydMobileNet: Improved Version of MobileNets with Pyramid Depthwise Separable Convolution
Authors: Van-Thanh Hoang, Kang-Hyun Jo
Date: 2018-11-17 00:00:00
Tasks: Object Recognition

Convolutional neural networks (CNNs) have shown remarkable performance in
various computer vision tasks in recent years. However, the increasing model
size has raised challenges in adopting them in real-time applications as well
as mobile and embedded vision applications. Many works try to build networks as
small as possible while still have acceptable performance. The state-of-the-art
architecture is MobileNets. They use Depthwise Separable Convolution
(DWConvolution) in place of standard Convolution to reduce the size of
networks. This paper describes an improved version of MobileNet, called Pyramid
Mobile Network. Instead of using just a $3\times 3$ kernel size for
DWConvolution like in MobileNet, the proposed network uses a pyramid kernel
size to capture more spatial information. The proposed architecture is
evaluated on two highly competitive object recognition benchmark datasets
(CIFAR-10, CIFAR-100). The experiments demonstrate that the proposed network
achieves better performance compared with MobileNet as well as other
state-of-the-art networks. Additionally, it is more flexible in fine-tuning the
trade-off between accuracy, latency and model size than MobileNets.
Skeleton-based Gesture Recognition Using Several Fully Connected Layers with Path Signature Features and Temporal Transformer Module
Authors: Chenyang Li, Xin Zhang, Lufan Liao, Lianwen Jin, Weixin Yang
Date: 2018-11-17 00:00:00
Tasks: Computational Efficiency, General Classification, Gesture Recognition

The skeleton based gesture recognition is gaining more popularity due to its
wide possible applications. The key issues are how to extract discriminative
features and how to design the classification model. In this paper, we first
leverage a robust feature descriptor, path signature (PS), and propose three PS
features to explicitly represent the spatial and temporal motion
characteristics, i.e., spatial PS (S_PS), temporal PS (T_PS) and temporal
spatial PS (T_S_PS). Considering the significance of fine hand movements in the
gesture, we propose an "attention on hand" (AOH) principle to define joint
pairs for the S_PS and select single joint for the T_PS. In addition, the
dyadic method is employed to extract the T_PS and T_S_PS features that encode
global and local temporal dynamics in the motion. Secondly, without the
recurrent strategy, the classification model still faces challenges on temporal
variation among different sequences. We propose a new temporal transformer
module (TTM) that can match the sequence key frames by learning the temporal
shifting parameter for each input. This is a learning-based module that can be
included into standard neural network architecture. Finally, we design a
multi-stream fully connected layer based network to treat spatial and temporal
features separately and fused them together for the final result. We have
tested our method on three benchmark gesture datasets, i.e., ChaLearn 2016,
ChaLearn 2013 and MSRC-12. Experimental results demonstrate that we achieve the
state-of-the-art performance on skeleton-based gesture recognition with high
computational efficiency.
Bilingual Dictionary Induction for Bantu Languages
Authors: Ndapa Nakashole
Date: 2018-11-17 00:00:00
Tasks: Translation

We present a method for learning bilingual translation dictionaries between English and Bantu languages. We show that exploiting the grammatical structure common to Bantu languages enables bilingual dictionary induction for languages where training data is unavailable.
An Affect-Rich Neural Conversational Model with Biased Attention and Weighted Cross-Entropy Loss
Authors: Peixiang Zhong, Di Wang, Chunyan Miao
Date: 2018-11-17 00:00:00

Affect conveys important implicit information in human communication. Having
the capability to correctly express affect during human-machine conversations
is one of the major milestones in artificial intelligence. In recent years,
extensive research on open-domain neural conversational models has been
conducted. However, embedding affect into such models is still under explored.
In this paper, we propose an end-to-end affect-rich open-domain neural
conversational model that produces responses not only appropriate in syntax and
semantics, but also with rich affect. Our model extends the Seq2Seq model and
adopts VAD (Valence, Arousal and Dominance) affective notations to embed each
word with affects. In addition, our model considers the effect of negators and
intensifiers via a novel affective attention mechanism, which biases attention
towards affect-rich words in input sentences. Lastly, we train our model with
an affect-incorporated objective function to encourage the generation of
affect-rich words in the output responses. Evaluations based on both perplexity
and human evaluations show that our model outperforms the state-of-the-art
baseline model of comparable size in producing natural and affect-rich
responses.
Semi-Supervised Semantic Image Segmentation with Self-correcting Networks
Authors: Mostafa S. Ibrahim, Arash Vahdat, Mani Ranjbar, William G. Macready
Date: 2018-11-17 00:00:00
Proceeding: CVPR 2020 6
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Building a large image dataset with high-quality object masks for semantic segmentation is costly and time consuming. In this paper, we introduce a principled semi-supervised framework that only uses a small set of fully supervised images (having semantic segmentation labels and box labels) and a set of images with only object bounding box labels (we call it the weak set). Our framework trains the primary segmentation model with the aid of an ancillary model that generates initial segmentation labels for the weak set and a self-correction module that improves the generated labels during training using the increasingly accurate primary model. We introduce two variants of the self-correction module using either linear or convolutional functions. Experiments on the PASCAL VOC 2012 and Cityscape datasets show that our models trained with a small fully supervised set perform similar to, or better than, models trained with a large fully supervised set while requiring ~7x less annotation effort.
DSCnet: Replicating Lidar Point Clouds with Deep Sensor Cloning
Authors: Paden Tomasello, Sammy Sidhu, Anting Shen, Matthew W. Moskewicz, Nobie Redmon, Gayatri Joshi, Romi Phadte, Paras Jain, Forrest Iandola
Date: 2018-11-17 00:00:00
Tasks: Autonomous Vehicles, Depth Estimation, Depth Prediction, image-classification, Image Classification, Image Segmentation, Semantic Segmentation

Convolutional neural networks (CNNs) have become increasingly popular for
solving a variety of computer vision tasks, ranging from image classification
to image segmentation. Recently, autonomous vehicles have created a demand for
depth information, which is often obtained using hardware sensors such as Light
detection and ranging (LIDAR). Although it can provide precise distance
measurements, most LIDARs are still far too expensive to sell in mass-produced
consumer vehicles, which has motivated methods to generate depth information
from commodity automotive sensors like cameras.
  In this paper, we propose an approach called Deep Sensor Cloning (DSC). The
idea is to use Convolutional Neural Networks in conjunction with inexpensive
sensors to replicate the 3D point-clouds that are created by expensive LIDARs.
To accomplish this, we develop a new dataset (DSDepth) and a new family of CNN
architectures (DSCnets). While previous tasks such as KITTI depth prediction
use an interpolated RGB-D images as ground-truth for training, we instead use
DSCnets to directly predict LIDAR point-clouds. When we compare the output of
our models to a $75,000 LIDAR, we find that our most accurate DSCnet achieves a
relative error of 5.77% using a single camera and 4.69% using stereo cameras.
Detecting Incongruity Between News Headline and Body Text via a Deep Hierarchical Encoder
Authors: Seunghyun Yoon, Kunwoo Park, Joongbo Shin, Hongjun Lim, Seungpil Won, Meeyoung Cha, Kyomin Jung
Date: 2018-11-17 00:00:00
Tasks: Articles, Data Augmentation, Fake News Detection, incongruity detection, Stance Detection

Some news headlines mislead readers with overrated or false information, and
identifying them in advance will better assist readers in choosing proper news
stories to consume. This research introduces million-scale pairs of news
headline and body text dataset with incongruity label, which can uniquely be
utilized for detecting news stories with misleading headlines. On this dataset,
we develop two neural networks with hierarchical architectures that model a
complex textual representation of news articles and measure the incongruity
between the headline and the body text. We also present a data augmentation
method that dramatically reduces the text input size a model handles by
independently investigating each paragraph of news stories, which further
boosts the performance. Our experiments and qualitative evaluations demonstrate
that the proposed methods outperform existing approaches and efficiently detect
news stories with misleading headlines in the real world.
The Full Spectrum of Deepnet Hessians at Scale: Dynamics with SGD Training and Sample Size
Authors: Vardan Papyan
Date: 2018-11-16 00:00:00

We apply state-of-the-art tools in modern high-dimensional numerical linear algebra to approximate efficiently the spectrum of the Hessian of modern deepnets, with tens of millions of parameters, trained on real data. Our results corroborate previous findings, based on small-scale networks, that the Hessian exhibits "spiked" behavior, with several outliers isolated from a continuous bulk. We decompose the Hessian into different components and study the dynamics with training and sample size of each term individually.
Using Sentiment Induction to Understand Variation in Gendered Online Communities
Authors: Li Lucy, Julia Mendelsohn
Date: 2018-11-16 00:00:00
Proceeding: WS 2019 1
Tasks: text similarity

We analyze gendered communities defined in three different ways: text, users,
and sentiment. Differences across these representations reveal facets of
communities' distinctive identities, such as social group, topic, and
attitudes. Two communities may have high text similarity but not user
similarity or vice versa, and word usage also does not vary according to a
clearcut, binary perspective of gender. Community-specific sentiment lexicons
demonstrate that sentiment can be a useful indicator of words' social meaning
and community values, especially in the context of discussion content and user
demographics. Our results show that social platforms such as Reddit are active
settings for different constructions of gender.