pwc-archive/papers-with-abstracts huggingface dataset:
The Use of Mutual Coherence to Prove $\ell^1/\ell^0$-Equivalence in Classification Problems
Authors: Chelsea Weaver, Naoki Saito
Date: 2019-01-09 00:00:00
Tasks: compressed sensing, General Classification, Sparse Representation-based Classification

We consider the decomposition of a signal over an overcomplete set of
vectors. Minimization of the $\ell^1$-norm of the coefficient vector can often
retrieve the sparsest solution (so-called "$\ell^1/\ell^0$-equivalence"), a
generally NP-hard task, and this fact has powered the field of compressed
sensing. Wright et al.'s sparse representation-based classification (SRC)
applies this relationship to machine learning, wherein the signal to be
decomposed represents the test sample and columns of the dictionary are
training samples. We investigate the relationships between
$\ell^1$-minimization, sparsity, and classification accuracy in SRC. After
proving that the tractable, deterministic approach to verifying
$\ell^1/\ell^0$-equivalence fundamentally conflicts with the high coherence
between same-class training samples, we demonstrate that $\ell^1$-minimization
can still recover the sparsest solution when the classes are well-separated.
Further, using a nonlinear transform so that sparse recovery conditions may be
satisfied, we demonstrate that approximate (not strict) equivalence is key to
the success of SRC.
Sentiment Analysis of Czech Texts: An Algorithmic Survey
Authors: Erion Çano, Ondřej Bojar
Date: 2019-01-09 00:00:00
Tasks: Sentiment Analysis, Survey

In the area of online communication, commerce and transactions, analyzing
sentiment polarity of texts written in various natural languages has become
crucial. While there have been a lot of contributions in resources and studies
for the English language, "smaller" languages like Czech have not received much
attention. In this survey, we explore the effectiveness of many existing
machine learning algorithms for sentiment analysis of Czech Facebook posts and
product reviews. We report the sets of optimal parameter values for each
algorithm and the scores in both datasets. We finally observe that support
vector machines are the best classifier and efforts to increase performance
even more with bagging, boosting or voting ensemble schemes fail to do so.
How Compact?: Assessing Compactness of Representations through Layer-Wise Pruning
Authors: Hyun-Joo Jung, Jaedeok Kim, Yoonsuck Choe
Date: 2019-01-09 00:00:00
Tasks: image-classification, Image Classification

Various forms of representations may arise in the many layers embedded in
deep neural networks (DNNs). Of these, where can we find the most compact
representation? We propose to use a pruning framework to answer this question:
How compact can each layer be compressed, without losing performance? Most of
the existing DNN compression methods do not consider the relative
compressibility of the individual layers. They uniformly apply a single target
sparsity to all layers or adapt layer sparsity using heuristics and additional
training. We propose a principled method that automatically determines the
sparsity of individual layers derived from the importance of each layer. To do
this, we consider a metric to measure the importance of each layer based on the
layer-wise capacity. Given the trained model and the total target sparsity, we
first evaluate the importance of each layer from the model. From the evaluated
importance, we compute the layer-wise sparsity of each layer. The proposed
method can be applied to any DNN architecture and can be combined with any
pruning method that takes the total target sparsity as a parameter. To validate
the proposed method, we carried out an image classification task with two types
of DNN architectures on two benchmark datasets and used three pruning methods
for compression. In case of VGG-16 model with weight pruning on the ImageNet
dataset, we achieved up to 75% (17.5% on average) better top-5 accuracy than
the baseline under the same total target sparsity. Furthermore, we analyzed
where the maximum compression can occur in the network. This kind of analysis
can help us identify the most compact representation within a deep neural
network.
Dirichlet Variational Autoencoder
Authors: Weonyoung Joo, Wonsung Lee, Sungrae Park, Il-Chul Moon
Date: 2019-01-09 00:00:00
Proceeding: ICLR 2019 5
Tasks: Decoder, General Classification, Topic Models

This paper proposes Dirichlet Variational Autoencoder (DirVAE) using a
Dirichlet prior for a continuous latent variable that exhibits the
characteristic of the categorical probabilities. To infer the parameters of
DirVAE, we utilize the stochastic gradient method by approximating the Gamma
distribution, which is a component of the Dirichlet distribution, with the
inverse Gamma CDF approximation. Additionally, we reshape the component
collapsing issue by investigating two problem sources, which are decoder weight
collapsing and latent value collapsing, and we show that DirVAE has no
component collapsing; while Gaussian VAE exhibits the decoder weight collapsing
and Stick-Breaking VAE shows the latent value collapsing. The experimental
results show that 1) DirVAE models the latent representation result with the
best log-likelihood compared to the baselines; and 2) DirVAE produces more
interpretable latent values with no collapsing issues which the baseline models
suffer from. Also, we show that the learned latent representation from the
DirVAE achieves the best classification accuracy in the semi-supervised and the
supervised classification tasks on MNIST, OMNIGLOT, and SVHN compared to the
baseline VAEs. Finally, we demonstrated that the DirVAE augmented topic models
show better performances in most cases.
Transfer Representation Learning with TSK Fuzzy System
Authors: Peng Xu, Zhaohong Deng, Jun Wang, Qun Zhang, Shitong Wang
Date: 2019-01-09 00:00:00
Tasks: Dimensionality Reduction, Representation Learning, Transfer Learning

Transfer learning can address the learning tasks of unlabeled data in the
target domain by leveraging plenty of labeled data from a different but related
source domain. A core issue in transfer learning is to learn a shared feature
space in where the distributions of the data from two domains are matched. This
learning process can be named as transfer representation learning (TRL). The
feature transformation methods are crucial to ensure the success of TRL. The
most commonly used feature transformation method in TRL is kernel-based
nonlinear mapping to the high-dimensional space followed by linear
dimensionality reduction. But the kernel functions are lack of interpretability
and are difficult to be selected. To this end, the TSK fuzzy system (TSK-FS) is
combined with transfer learning and a more intuitive and interpretable modeling
method, called transfer representation learning with TSK-FS (TRL-TSK-FS) is
proposed in this paper. Specifically, TRL-TSK-FS realizes TRL from two aspects.
On one hand, the data in the source and target domains are transformed into the
fuzzy feature space in which the distribution distance of the data between two
domains is min-imized. On the other hand, discriminant information and
geo-metric properties of the data are preserved by linear discriminant analysis
and principal component analysis. In addition, another advantage arises with
the proposed method, that is, the nonlinear transformation is realized by
constructing fuzzy mapping with the antecedent part of the TSK-FS instead of
kernel functions which are difficult to be selected. Extensive experiments are
conducted on the text and image datasets. The results obviously show the
superiority of the proposed method.
Guess What's on my Screen? Clustering Smartphone Screenshots with Active Learning
Authors: Agnese Chiatti, Dolzodmaa Davaasuren, Nilam Ram, Prasenjit Mitra, Byron Reeves, Thomas Robinson
Date: 2019-01-09 00:00:00
Tasks: Active Learning, Clustering

A significant proportion of individuals' daily activities is experienced
through digital devices. Smartphones in particular have become one of the
preferred interfaces for content consumption and social interaction.
Identifying the content embedded in frequently-captured smartphone screenshots
is thus a crucial prerequisite to studies of media behavior and health
intervention planning that analyze activity interplay and content switching
over time. Screenshot images can depict heterogeneous contents and
applications, making the a priori definition of adequate taxonomies a
cumbersome task, even for humans. Privacy protection of the sensitive data
captured on screens means the costs associated with manual annotation are
large, as the effort cannot be crowd-sourced. Thus, there is need to examine
utility of unsupervised and semi-supervised methods for digital screenshot
classification. This work introduces the implications of applying clustering on
large screenshot sets when only a limited amount of labels is available. In
this paper we develop a framework for combining K-Means clustering with Active
Learning for efficient leveraging of labeled and unlabeled samples, with the
goal of discovering latent classes and describing a large collection of
screenshot data. We tested whether SVM-embedded or XGBoost-embedded solutions
for class probability propagation provide for more well-formed cluster
configurations. Visual and textual vector representations of the screenshot
images are derived and combined to assess the relative contribution of
multi-modal features to the overall performance.
Image Recognition of Tea Leaf Diseases Based on Convolutional Neural Network
Authors: Xiaoxiao Sun, Shaomin Mu, Yongyu Xu, Zhihao Cao, Tingting Su
Date: 2019-01-09 00:00:00
Tasks: General Classification, Image Segmentation, Semantic Segmentation

In order to identify and prevent tea leaf diseases effectively, convolution
neural network (CNN) was used to realize the image recognition of tea disease
leaves. Firstly, image segmentation and data enhancement are used to preprocess
the images, and then these images were input into the network for training.
Secondly, to reach a higher recognition accuracy of CNN, the learning rate and
iteration numbers were adjusted frequently and the dropout was added properly
in the case of over-fitting. Finally, the experimental results show that the
recognition accuracy of CNN is 93.75%, while the accuracy of SVM and BP neural
network is 89.36% and 87.69% respectively. Therefore, the recognition algorithm
based on CNN is better in classification and can improve the recognition
efficiency of tea leaf diseases effectively.
Low-Cost Transfer Learning of Face Tasks
Authors: Thrupthi Ann John, Isha Dua, Vineeth N. Balasubramanian, C. V. Jawahar
Date: 2019-01-09 00:00:00
Tasks: Face Recognition, Transfer Learning

Do we know what the different filters of a face network represent? Can we use
this filter information to train other tasks without transfer learning? For
instance, can age, head pose, emotion and other face related tasks be learned
from face recognition network without transfer learning? Understanding the role
of these filters allows us to transfer knowledge across tasks and take
advantage of large data sets in related tasks. Given a pretrained network, we
can infer which tasks the network generalizes for and the best way to transfer
the information to a new task.
Is it Time to Swish? Comparing Deep Learning Activation Functions Across NLP tasks
Authors: Steffen Eger, Paul Youssef, Iryna Gurevych
Date: 2019-01-09 00:00:00
Proceeding: EMNLP 2018 10
Tasks: image-classification, Image Classification

Activation functions play a crucial role in neural networks because they are
the nonlinearities which have been attributed to the success story of deep
learning. One of the currently most popular activation functions is ReLU, but
several competitors have recently been proposed or 'discovered', including
LReLU functions and swish. While most works compare newly proposed activation
functions on few tasks (usually from image classification) and against few
competitors (usually ReLU), we perform the first large-scale comparison of 21
activation functions across eight different NLP tasks. We find that a largely
unknown activation function performs most stably across all tasks, the
so-called penalized tanh function. We also show that it can successfully
replace the sigmoid and tanh gates in LSTM cells, leading to a 2 percentage
point (pp) improvement over the standard choices on a challenging NLP task.
Deep Semantic Multimodal Hashing Network for Scalable Image-Text and Video-Text Retrievals
Authors: Lu Jin, Zechao Li, Jinhui Tang
Date: 2019-01-09 00:00:00
Tasks: Cross-Modal Retrieval, Deep Hashing, Image-text Retrieval, Representation Learning, Retrieval, Text Retrieval, Video-Text Retrieval

Hashing has been widely applied to multimodal retrieval on large-scale multimedia data due to its efficiency in computation and storage. In this article, we propose a novel deep semantic multimodal hashing network (DSMHN) for scalable image-text and video-text retrieval. The proposed deep hashing framework leverages 2-D convolutional neural networks (CNN) as the backbone network to capture the spatial information for image-text retrieval, while the 3-D CNN as the backbone network to capture the spatial and temporal information for video-text retrieval. In the DSMHN, two sets of modality-specific hash functions are jointly learned by explicitly preserving both intermodality similarities and intramodality semantic labels. Specifically, with the assumption that the learned hash codes should be optimal for the classification task, two stream networks are jointly trained to learn the hash functions by embedding the semantic labels on the resultant hash codes. Moreover, a unified deep multimodal hashing framework is proposed to learn compact and high-quality hash codes by exploiting the feature representation learning, intermodality similarity-preserving learning, semantic label-preserving learning, and hash function learning with different types of loss functions simultaneously. The proposed DSMHN method is a generic and scalable deep hashing framework for both image-text and video-text retrievals, which can be flexibly integrated with different types of loss functions. We conduct extensive experiments for both single modal- and cross-modal-retrieval tasks on four widely used multimodal-retrieval data sets. Experimental results on both image-text- and video-text-retrieval tasks demonstrate that the DSMHN significantly outperforms the state-of-the-art methods.
What do Language Representations Really Represent?
Authors: Johannes Bjerva, Robert Östling, Maria Han Veiga, Jörg Tiedemann, Isabelle Augenstein
Date: 2019-01-09 00:00:00
Proceeding: CL 2019 6
Tasks: Language Modeling, Language Modelling, Translation

A neural language model trained on a text corpus can be used to induce
distributed representations of words, such that similar words end up with
similar representations. If the corpus is multilingual, the same model can be
used to learn distributed representations of languages, such that similar
languages end up with similar representations. We show that this holds even
when the multilingual corpus has been translated into English, by picking up
the faint signal left by the source languages. However, just like it is a
thorny problem to separate semantic from syntactic similarity in word
representations, it is not obvious what type of similarity is captured by
language representations. We investigate correlations and causal relationships
between language representations learned from translations on one hand, and
genetic, geographical, and several levels of structural similarity between
languages on the other. Of these, structural similarity is found to correlate
most strongly with language representation similarity, while genetic
relationships---a convenient benchmark used for evaluation in previous
work---appears to be a confounding factor. Apart from implications about
translation effects, we see this more generally as a case where NLP and
linguistic typology can interact and benefit one another.
Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection
Authors: Lu Zhang, Xiangyu Zhu, Xiangyu Chen, Xu Yang, Zhen Lei, Zhi-Yong Liu
Date: 2019-01-09 00:00:00
Proceeding: ICCV 2019 10
Tasks: 2D Object Detection, Multispectral Object Detection, Position

Multispectral pedestrian detection has shown great advantages under poor illumination conditions, since the thermal modality provides complementary information for the color image. However, real multispectral data suffers from the position shift problem, i.e. the color-thermal image pairs are not strictly aligned, making one object has different positions in different modalities. In deep learning based methods, this problem makes it difficult to fuse the feature maps from both modalities and puzzles the CNN training. In this paper, we propose a novel Aligned Region CNN (AR-CNN) to handle the weakly aligned multispectral data in an end-to-end way. Firstly, we design a Region Feature Alignment (RFA) module to capture the position shift and adaptively align the region features of the two modalities. Secondly, we present a new multimodal fusion method, which performs feature re-weighting to select more reliable features and suppress the useless ones. Besides, we propose a novel RoI jitter strategy to improve the robustness to unexpected shift patterns of different devices and system settings. Finally, since our method depends on a new kind of labelling: bounding boxes that match each modality, we manually relabel the KAIST dataset by locating bounding boxes in both modalities and building their relationships, providing a new KAIST-Paired Annotation. Extensive experimental validations on existing datasets are performed, demonstrating the effectiveness and robustness of the proposed method. Code and data are available at https://github.com/luzhang16/AR-CNN.
MOANA: An Online Learned Adaptive Appearance Model for Robust Multiple Object Tracking in 3D
Authors: Zheng Tang, Jenq-Neng Hwang
Date: 2019-01-09 00:00:00
Tasks: 3D geometry, CPU, Multiple Object Tracking, Object Tracking

Multiple object tracking has been a challenging field, mainly due to noisy
detection sets and identity switch caused by occlusion and similar appearance
among nearby targets. Previous works rely on appearance models built on
individual or several selected frames for the comparison of features, but they
cannot encode long-term appearance changes caused by pose, viewing angle and
lighting conditions. In this work, we propose an adaptive model that learns
online a relatively long-term appearance change of each target. The proposed
model is compatible with any feature of fixed dimension or their combination,
whose learning rates are dynamically controlled by adaptive update and spatial
weighting schemes. To handle occlusion and nearby objects sharing similar
appearance, we also design cross-matching and re-identification schemes based
on the application of the proposed adaptive appearance models. Additionally,
the 3D geometry information is effectively incorporated in our formulation for
data association. The proposed method outperforms all the state-of-the-art on
the MOTChallenge 3D benchmark and achieves real-time computation with only a
standard desktop CPU. It has also shown superior performance over the
state-of-the-art on the 2D benchmark of MOTChallenge.
Fast CNN-Based Object Tracking Using Localization Layers and Deep Features Interpolation
Authors: Al-Hussein A. El-Shafie, Mohamed Zaki, Serag El-Din Habib
Date: 2019-01-09 00:00:00
Tasks: Object, Object Tracking

Object trackers based on Convolution Neural Network (CNN) have achieved
state-of-the-art performance on recent tracking benchmarks, while they suffer
from slow computational speed. The high computational load arises from the
extraction of the feature maps of the candidate and training patches in every
video frame. The candidate and training patches are typically placed randomly
around the previous target location and the estimated target location
respectively. In this paper, we propose novel schemes to speed-up the
processing of the CNN-based trackers. We input the whole region-of-interest
once to the CNN to eliminate the redundant computations of the random candidate
patches. In addition to classifying each candidate patch as an object or
background, we adapt the CNN to classify the target location inside the object
patches as a coarse localization step, and we employ bilinear interpolation for
the CNN feature maps as a fine localization step. Moreover, bilinear
interpolation is exploited to generate CNN feature maps of the training patches
without actually forwarding the training patches through the network which
achieves a significant reduction of the required computations. Our tracker does
not rely on offline video training. It achieves competitive performance results
on the OTB benchmark with 8x speed improvements compared to the equivalent
tracker.
Performance Analysis and Dynamic Evolution of Deep Convolutional Neural Network for Nonlinear Inverse Scattering
Authors: Lianlin Li, Long Gang Wang, Fernando L. Teixeira
Date: 2019-01-09 00:00:00
Tasks: SSIM

The solution of nonlinear electromagnetic (EM) inverse scattering problems is
typically hindered by several challenges such as ill-posedness, strong
nonlinearity, and high computational costs. Recently, deep learning has been
demonstrated to be a promising tool in addressing these challenges. In
particular, it is possible to establish a connection between a deep
convolutional neural network (CNN) and iterative solution methods of nonlinear
EM inverse scattering. This has led to the development of an efficient
CNN-based solution to nonlinear EM inverse problems, termed DeepNIS. It has
been shown that DeepNIS can outperform conventional nonlinear inverse
scattering methods in terms of both image quality and computational time. In
this work, we quantitatively evaluate the performance of DeepNIS as a function
of the number of layers using structure similarity measure (SSIM) and
mean-square error (MSE) metrics. In addition, we probe the dynamic evolution
behavior of DeepNIS by examining its near-isometry property. It is shown that
after a proper training stage the proposed CNN is near optimal in terms of the
stability and generalization ability.
Sequential Attention-based Network for Noetic End-to-End Response Selection
Authors: Qian Chen, Wen Wang
Date: 2019-01-09 00:00:00
Tasks: Conversational Response Selection, Goal-Oriented Dialog

The noetic end-to-end response selection challenge as one track in Dialog System Technology Challenges 7 (DSTC7) aims to push the state of the art of utterance classification for real world goal-oriented dialog systems, for which participants need to select the correct next utterances from a set of candidates for the multi-turn context. This paper describes our systems that are ranked the top on both datasets under this challenge, one focused and small (Advising) and the other more diverse and large (Ubuntu). Previous state-of-the-art models use hierarchy-based (utterance-level and token-level) neural networks to explicitly model the interactions among different turns' utterances for context modeling. In this paper, we investigate a sequential matching model based only on chain sequence for multi-turn response selection. Our results demonstrate that the potentials of sequential matching approaches have not yet been fully exploited in the past for multi-turn response selection. In addition to ranking the top in the challenge, the proposed model outperforms all previous models, including state-of-the-art hierarchy-based models, and achieves new state-of-the-art performances on two large-scale public multi-turn response selection benchmark datasets.