pwc-archive/papers-with-abstracts huggingface dataset:
Deep Convolutional Neural Network for Plant Seedlings Classification
Authors: Daniel K. Nkemelu, Daniel Omeiza, Nancy Lubalo
Date: 2018-11-20 00:00:00
Tasks: Classification, General Classification

Agriculture is vital for human survival and remains a major driver of several
economies around the world; more so in underdeveloped and developing economies.
With increasing demand for food and cash crops, due to a growing global
population and the challenges posed by climate change, there is a pressing need
to increase farm outputs while incurring minimal costs. Previous machine vision
technologies developed for selective weeding have faced the challenge of
reliable and accurate weed detection. We present approaches for plant seedlings
classification with a dataset that contains 4,275 images of approximately 960
unique plants belonging to 12 species at several growth stages. We compare the
performances of two traditional algorithms and a Convolutional Neural Network
(CNN), a deep learning technique widely applied to image recognition, for this
task. Our findings show that CNN-driven seedling classification applications
when used in farming automation has the potential to optimize crop yield and
improve productivity and efficiency when designed appropriately.
Single-Label Multi-Class Image Classification by Deep Logistic Regression
Authors: Qi Dong, Xiatian Zhu, Shaogang Gong
Date: 2018-11-20 00:00:00
Tasks: Attribute, Classification, General Classification, image-classification, Image Classification, Multi-class Classification, regression

The objective learning formulation is essential for the success of convolutional neural networks. In this work, we analyse thoroughly the standard learning objective functions for multi-class classification CNNs: softmax regression (SR) for single-label scenario and logistic regression (LR) for multi-label scenario. Our analyses lead to an inspiration of exploiting LR for single-label classification learning, and then the disclosing of the negative class distraction problem in LR. To address this problem, we develop two novel LR based objective functions that not only generalise the conventional LR but importantly turn out to be competitive alternatives to SR in single label classification. Extensive comparative evaluations demonstrate the model learning advantages of the proposed LR functions over the commonly adopted SR in single-label coarse-grained object categorisation and cross-class fine-grained person instance identification tasks. We also show the performance superiority of our method on clothing attribute classification in comparison to the vanilla LR function.
Shape-only Features for Plant Leaf Identification
Authors: Charlie Hewitt, Marwa Mahmoud
Date: 2018-11-20 00:00:00
Tasks: General Classification

This paper presents a novel feature set for shape-only leaf identification
motivated by real-world, mobile deployment. The feature set includes basic
shape features, as well as signal features extracted from local area integral
invariants (LAIIs), similar to curvature maps, at multiple scales. The proposed
methodology is evaluated on a number of publicly available leaf datasets with
comparable results to existing methods which make use of colour and texture
features in addition to shape. Over 90% classification accuracy is achieved on
most datasets, with top-four accuracy for these datasets reaching over 98%.
Rotation and scale invariance of the proposed features are demonstrated, along
with an evaluation of the generalisability of the approach for generic shape
matching.
Gen-Oja: A Two-time-scale approach for Streaming CCA
Authors: Kush Bhatia, Aldo Pacchiano, Nicolas Flammarion, Peter L. Bartlett, Michael. I. Jordan
Date: 2018-11-20 00:00:00
Tasks: Vocal Bursts Valence Prediction

In this paper, we study the problems of principal Generalized Eigenvector computation and Canonical Correlation Analysis in the stochastic setting. We propose a simple and efficient algorithm, Gen-Oja, for these problems. We prove the global convergence of our algorithm, borrowing ideas from the theory of fast-mixing Markov chains and two-time-scale stochastic approximation, showing that it achieves the optimal rate of convergence. In the process, we develop tools for understanding stochastic processes with Markovian noise which might be of independent interest.
Structured Pruning for Efficient ConvNets via Incremental Regularization
Authors: Huan Wang, Qiming Zhang, Yuehai Wang, Haoji Hu
Date: 2018-11-20 00:00:00
Proceeding: NIPS Workshop CDNNRIA 2018

Parameter pruning is a promising approach for CNN compression and
acceleration by eliminating redundant model parameters with tolerable
performance loss. Despite its effectiveness, existing regularization-based
parameter pruning methods usually drive weights towards zero with large and
constant regularization factors, which neglects the fact that the
expressiveness of CNNs is fragile and needs a more gentle way of regularization
for the networks to adapt during pruning. To solve this problem, we propose a
new regularization-based pruning method (named IncReg) to incrementally assign
different regularization factors to different weight groups based on their
relative importance, whose effectiveness is proved on popular CNNs compared
with state-of-the-art methods.
TSM: Temporal Shift Module for Efficient Video Understanding
Authors: Ji Lin, Chuang Gan, Song Han
Date: 2018-11-20 00:00:00
Proceeding: ICCV 2019 10
Tasks: 3D Action Recognition, Action Classification, Action Recognition, Micro-Action Recognition, object-detection, Object Detection, Video Object Detection, Video Object Tracking, Video Recognition, Video Understanding

The explosive growth in video streaming gives rise to challenges on performing video understanding at high accuracy and low computation cost. Conventional 2D CNNs are computationally cheap but cannot capture temporal relationships; 3D CNN based methods can achieve good performance but are computationally intensive, making it expensive to deploy. In this paper, we propose a generic and effective Temporal Shift Module (TSM) that enjoys both high efficiency and high performance. Specifically, it can achieve the performance of 3D CNN but maintain 2D CNN's complexity. TSM shifts part of the channels along the temporal dimension; thus facilitate information exchanged among neighboring frames. It can be inserted into 2D CNNs to achieve temporal modeling at zero computation and zero parameters. We also extended TSM to online setting, which enables real-time low-latency online video recognition and video object detection. TSM is accurate and efficient: it ranks the first place on the Something-Something leaderboard upon publication; on Jetson Nano and Galaxy Note8, it achieves a low latency of 13ms and 35ms for online video recognition. The code is available at: https://github.com/mit-han-lab/temporal-shift-module.
Locally Private Gaussian Estimation
Authors: Matthew Joseph, Janardhan Kulkarni, Jieming Mao, Zhiwei Steven Wu
Date: 2018-11-20 00:00:00
Proceeding: NeurIPS 2019 12

We study a basic private estimation problem: each of $n$ users draws a single i.i.d. sample from an unknown Gaussian distribution, and the goal is to estimate the mean of this Gaussian distribution while satisfying local differential privacy for each user. Informally, local differential privacy requires that each data point is individually and independently privatized before it is passed to a learning algorithm. Locally private Gaussian estimation is therefore difficult because the data domain is unbounded: users may draw arbitrarily different inputs, but local differential privacy nonetheless mandates that different users have (worst-case) similar privatized output distributions. We provide both adaptive two-round solutions and nonadaptive one-round solutions for locally private Gaussian estimation. We then partially match these upper bounds with an information-theoretic lower bound. This lower bound shows that our accuracy guarantees are tight up to logarithmic factors for all sequentially interactive $(\varepsilon,\delta)$-locally private protocols.
The Effect of Explicit Structure Encoding of Deep Neural Networks for Symbolic Music Generation
Authors: Ke Chen, Weilin Zhang, Shlomo Dubnov, Gus Xia, Wei Li
Date: 2018-11-20 00:00:00
Tasks: Music Generation

With recent breakthroughs in artificial neural networks, deep generative
models have become one of the leading techniques for computational creativity.
Despite very promising progress on image and short sequence generation,
symbolic music generation remains a challenging problem since the structure of
compositions are usually complicated. In this study, we attempt to solve the
melody generation problem constrained by the given chord progression. This
music meta-creation problem can also be incorporated into a plan recognition
system with user inputs and predictive structural outputs. In particular, we
explore the effect of explicit architectural encoding of musical structure via
comparing two sequential generative models: LSTM (a type of RNN) and WaveNet
(dilated temporal-CNN). As far as we know, this is the first study of applying
WaveNet to symbolic music generation, as well as the first systematic
comparison between temporal-CNN and RNN for music generation. We conduct a
survey for evaluation in our generations and implemented Variable Markov Oracle
in music pattern discovery. Experimental results show that to encode structure
more explicitly using a stack of dilated convolution layers improved the
performance significantly, and a global encoding of underlying chord
progression into the generation procedure gains even more.
A Gray Box Interpretable Visual Debugging Approach for Deep Sequence Learning Model
Authors: Md Mofijul Islam, Amar Debnath, Tahsin Al Sayeed, Jyotirmay Nag Setu, Md Mahmudur Rahman, Md Sadman Sakib, Md Abdur Razzaque, Md. Mosaddek Khan, Swakkhar Shatabda
Date: 2018-11-20 00:00:00
Tasks: Decision Making, Deep Learning

Deep Learning algorithms are often used as black box type learning and they
are too complex to understand. The widespread usability of Deep Learning
algorithms to solve various machine learning problems demands deep and
transparent understanding of the internal representation as well as decision
making. Moreover, the learning models, trained on sequential data, such as
audio and video data, have intricate internal reasoning process due to their
complex distribution of features. Thus, a visual simulator might be helpful to
trace the internal decision making mechanisms in response to adversarial input
data, and it would help to debug and design appropriate deep learning models.
However, interpreting the internal reasoning of deep learning model is not well
studied in the literature. In this work, we have developed a visual interactive
web application, namely d-DeVIS, which helps to visualize the internal
reasoning of the learning model which is trained on the audio data. The
proposed system allows to perceive the behavior as well as to debug the model
by interactively generating adversarial audio data point. The web application
of d-DeVIS is available at ddevis.herokuapp.com.
On a hypergraph probabilistic graphical model
Authors: Mohammad Ali Javidian, Linyuan Lu, Marco Valtorta, Zhiyu Wang
Date: 2018-11-20 00:00:00
Tasks: model

We propose a directed acyclic hypergraph framework for a probabilistic
graphical model that we call Bayesian hypergraphs. The space of directed
acyclic hypergraphs is much larger than the space of chain graphs. Hence
Bayesian hypergraphs can model much finer factorizations than Bayesian networks
or LWF chain graphs and provide simpler and more computationally efficient
procedures for factorizations and interventions. Bayesian hypergraphs also
allow a modeler to represent causal patterns of interaction such as Noisy-OR
graphically (without additional annotations). We introduce global, local and
pairwise Markov properties of Bayesian hypergraphs and prove under which
conditions they are equivalent. We define a projection operator, called shadow,
that maps Bayesian hypergraphs to chain graphs, and show that the Markov
properties of a Bayesian hypergraph are equivalent to those of its
corresponding chain graph. We extend the causal interpretation of LWF chain
graphs to Bayesian hypergraphs and provide corresponding formulas and a
graphical criterion for intervention.
Temporal Graph Offset Reconstruction: Towards Temporally Robust Graph Representation Learning
Authors: Stephen Bonner, John Brennan, Ibad Kureshi, Georgios Theodoropoulos, Andrew Stephen McGough, Boguslaw Obara
Date: 2018-11-20 00:00:00
Tasks: Graph Representation Learning, Link Prediction, Representation Learning

Graphs are a commonly used construct for representing relationships between
elements in complex high dimensional datasets. Many real-world phenomenon are
dynamic in nature, meaning that any graph used to represent them is inherently
temporal. However, many of the machine learning models designed to capture
knowledge about the structure of these graphs ignore this rich temporal
information when creating representations of the graph. This results in models
which do not perform well when used to make predictions about the future state
of the graph -- especially when the delta between time stamps is not small. In
this work, we explore a novel training procedure and an associated unsupervised
model which creates graph representations optimised to predict the future state
of the graph. We make use of graph convolutional neural networks to encode the
graph into a latent representation, which we then use to train our temporal
offset reconstruction method, inspired by auto-encoders, to predict a later
time point -- multiple time steps into the future. Using our method, we
demonstrate superior performance for the task of future link prediction
compared with none-temporal state-of-the-art baselines. We show our approach to
be capable of outperforming non-temporal baselines by 38% on a real world
dataset.
Multi-Task Learning of Generalizable Representations for Video Action Recognition
Authors: Zhiyu Yao, Yunbo Wang, Mingsheng Long, Jian-Min Wang, Philip S. Yu, Jiaguang Sun
Date: 2018-11-20 00:00:00
Tasks: Action Recognition, Multi-Task Learning, Optical Flow Estimation, Temporal Action Localization, Video Classification, Video Recognition

In classic video action recognition, labels may not contain enough information about the diverse video appearance and dynamics, thus, existing models that are trained under the standard supervised learning paradigm may extract less generalizable features. We evaluate these models under a cross-dataset experiment setting, as the above label bias problem in video analysis is even more prominent across different data sources. We find that using the optical flows as model inputs harms the generalization ability of most video recognition models. Based on these findings, we present a multi-task learning paradigm for video classification. Our key idea is to avoid label bias and improve the generalization ability by taking data as its own supervision or supervising constraints on the data. First, we take the optical flows and the RGB frames by taking them as auxiliary supervisions, and thus naming our model as Reversed Two-Stream Networks (Rev2Net). Further, we collaborate the auxiliary flow prediction task and the frame reconstruction task by introducing a new training objective to Rev2Net, named Decoding Discrepancy Penalty (DDP), which constraints the discrepancy of the multi-task features in a self-supervised manner. Rev2Net is shown to be effective on the classic action recognition task. It specifically shows a strong generalization ability in the cross-dataset experiments.
Strong mixed-integer programming formulations for trained neural networks
Authors: Ross Anderson, Joey Huchette, Christian Tjandraatmadja, Juan Pablo Vielma
Date: 2018-11-20 00:00:00

We present an ideal mixed-integer programming (MIP) formulation for a
rectified linear unit (ReLU) appearing in a trained neural network. Our
formulation requires a single binary variable and no additional continuous
variables beyond the input and output variables of the ReLU. We contrast it
with an ideal "extended" formulation with a linear number of additional
continuous variables, derived through standard techniques. An apparent drawback
of our formulation is that it requires an exponential number of inequality
constraints, but we provide a routine to separate the inequalities in linear
time. We also prove that these exponentially-many constraints are
facet-defining under mild conditions. Finally, we study network verification
problems and observe that dynamically separating from the exponential
inequalities 1) is much more computationally efficient and scalable than the
extended formulation, 2) decreases the solve time of a state-of-the-art MIP
solver by a factor of 7 on smaller instances, and 3) nearly matches the dual
bounds of a state-of-the-art MIP solver on harder instances, after just a few
rounds of separation and in orders of magnitude less time.
Learning deep kernels for exponential family densities
Authors: Li Wenliang, Danica J. Sutherland, Heiko Strathmann, Arthur Gretton
Date: 2018-11-20 00:00:00

The kernel exponential family is a rich class of distributions, which can be fit efficiently and with statistical guarantees by score matching. Being required to choose a priori a simple kernel such as the Gaussian, however, limits its practical applicability. We provide a scheme for learning a kernel parameterized by a deep network, which can find complex location-dependent local features of the data geometry. This gives a very rich class of density models, capable of fitting complex structures on moderate-dimensional problems. Compared to deep density models fit via maximum likelihood, our approach provides a complementary set of strengths and tradeoffs: in empirical studies, the former can yield higher likelihoods, whereas the latter gives better estimates of the gradient of the log density, the score, which describes the distribution's shape.
LGLG-WPCA: An Effective Texture-based Method for Face Recognition
Authors: Chaorong Li, Huang Wei, Huafu Chen
Date: 2018-11-20 00:00:00
Tasks: Face Recognition

In this paper, we proposed an effective face feature extraction method by Learning Gabor Log-Euclidean Gaussian with Whitening Principal Component Analysis (WPCA), called LGLG-WPCA. The proposed method learns face features from the embedded multivariate Gaussian in Gabor wavelet domain; it has the robust performance to adverse conditions such as varying poses, skin aging and uneven illumination. Because the space of Gaussian is a Riemannian manifold and it is difficult to incorporate learning mechanism in the model. To address this issue, we use L2EMG to map the multidimensional Gaussian model to the linear space, and then use WPCA to learn face features. We also implemented the key-point-based version of LGLG-WPCA, called LGLG(KP)-WPCA. Experiments show the proposed methods are effective and promising for face texture feature extraction and the combination of the feature of the proposed methods and the features of Deep Convolutional Network (DCNN) achieved the best recognition accuracies on FERET database compared to the state-of-the-art methods. In the next version of this paper, we will test the performance of the proposed methods on the large-varying pose databases.
Multi-layer Pruning Framework for Compressing Single Shot MultiBox Detector
Authors: Pravendra Singh, Manikandan. R, Neeraj Matiyali, Vinay P. Namboodiri
Date: 2018-11-20 00:00:00
Tasks: Traffic Sign Detection, Traffic Sign Recognition

We propose a framework for compressing state-of-the-art Single Shot MultiBox
Detector (SSD). The framework addresses compression in the following stages:
Sparsity Induction, Filter Selection, and Filter Pruning. In the Sparsity
Induction stage, the object detector model is sparsified via an improved global
threshold. In Filter Selection & Pruning stage, we select and remove filters
using sparsity statistics of filter weights in two consecutive convolutional
layers. This results in the model with the size smaller than most existing
compact architectures. We evaluate the performance of our framework with
multiple datasets and compare over multiple methods. Experimental results show
that our method achieves state-of-the-art compression of 6.7X and 4.9X on
PASCAL VOC dataset on models SSD300 and SSD512 respectively. We further show
that the method produces maximum compression of 26X with SSD512 on German
Traffic Sign Detection Benchmark (GTSDB). Additionally, we also empirically
show our method's adaptability for classification based architecture VGG16 on
datasets CIFAR and German Traffic Sign Recognition Benchmark (GTSRB) achieving
a compression rate of 125X and 200X with the reduction in flops by 90.50% and
96.6% respectively with no loss of accuracy. In addition to this, our method
does not require any special libraries or hardware support for the resulting
compressed models.
Causal Inference by String Diagram Surgery
Authors: Bart Jacobs, Aleks Kissinger, Fabio Zanasi
Date: 2018-11-20 00:00:00
Tasks: Causal Inference

Extracting causal relationships from observed correlations is a growing area in probabilistic reasoning, originating with the seminal work of Pearl and others from the early 1990s. This paper develops a new, categorically oriented view based on a clear distinction between syntax (string diagrams) and semantics (stochastic matrices), connected via interpretations as structure-preserving functors. A key notion in the identification of causal effects is that of an intervention, whereby a variable is forcefully set to a particular value independent of any prior propensities. We represent the effect of such an intervention as an endofunctor which performs `string diagram surgery' within the syntactic category of string diagrams. This diagram surgery in turn yields a new, interventional distribution via the interpretation functor. While in general there is no way to compute interventional distributions purely from observed data, we show that this is possible in certain special cases using a calculational tool called comb disintegration. We demonstrate the use of this technique on a well-known toy example, where we predict the causal effect of smoking on cancer in the presence of a confounding common cause. After developing this specific example, we show this technique provides simple sufficient conditions for computing interventions which apply to a wide variety of situations considered in the causal inference literature.
Black-Box Autoregressive Density Estimation for State-Space Models
Authors: Tom Ryder, Andrew Golighty, A. Stephen McGough, Dennis Prangle
Date: 2018-11-20 00:00:00
Tasks: Bayesian Inference, Deep Learning, Density Estimation, Econometrics, Epidemiology, State Space Models, Time Series, Time Series Analysis, Variational Inference

State-space models (SSMs) provide a flexible framework for modelling
time-series data. Consequently, SSMs are ubiquitously applied in areas such as
engineering, econometrics and epidemiology. In this paper we provide a fast
approach for approximate Bayesian inference in SSMs using the tools of deep
learning and variational inference.