pwc-archive/papers-with-abstracts huggingface dataset:
Learning with Inadequate and Incorrect Supervision
Authors: Chen Gong, Hengmin Zhang, Jian Yang, DaCheng Tao
Date: 2019-02-20 00:00:00
Tasks: image-classification, Image Classification, speech-recognition, Speech Recognition, Text Categorization

Practically, we are often in the dilemma that the labeled data at hand are
inadequate to train a reliable classifier, and more seriously, some of these
labeled data may be mistakenly labeled due to the various human factors.
Therefore, this paper proposes a novel semi-supervised learning paradigm that
can handle both label insufficiency and label inaccuracy. To address label
insufficiency, we use a graph to bridge the data points so that the label
information can be propagated from the scarce labeled examples to unlabeled
examples along the graph edges. To address label inaccuracy, Graph Trend
Filtering (GTF) and Smooth Eigenbase Pursuit (SEP) are adopted to filter out
the initial noisy labels. GTF penalizes the l_0 norm of label difference
between connected examples in the graph and exhibits better local adaptivity
than the traditional l_2 norm-based Laplacian smoother. SEP reconstructs the
correct labels by emphasizing the leading eigenvectors of Laplacian matrix
associated with small eigenvalues, as these eigenvectors reflect real label
smoothness and carry rich class separation cues. We term our algorithm as
`Semi-supervised learning under Inadequate and Incorrect Supervision' (SIIS).
Thorough experimental results on image classification, text categorization, and
speech recognition demonstrate that our SIIS is effective in label error
correction, leading to superior performance to the state-of-the-art methods in
the presence of label noise and label scarcity.
Mexican Hat Wavelet Kernel ELM for Multiclass Classification
Authors: Jie Wang, Yi-Fan Song, Tian-Lei Ma
Date: 2019-02-20 00:00:00
Tasks: Classification, General Classification

Kernel extreme learning machine (KELM) is a novel feedforward neural network,
which is widely used in classification problems. To some extent, it solves the
existing problems of the invalid nodes and the large computational complexity
in ELM. However, the traditional KELM classifier usually has a low test
accuracy when it faces multiclass classification problems. In order to solve
the above problem, a new classifier, Mexican Hat wavelet KELM classifier, is
proposed in this paper. The proposed classifier successfully improves the
training accuracy and reduces the training time in the multiclass
classification problems. Moreover, the validity of the Mexican Hat wavelet as a
kernel function of ELM is rigorously proved. Experimental results on different
data sets show that the performance of the proposed classifier is significantly
superior to the compared classifiers.
Learning Sparse Neural Networks via $\ell_0$ and T$\ell_1$ by a Relaxed Variable Splitting Method with Application to Multi-scale Curve Classification
Authors: Fanghui Xue, Jack Xin
Date: 2019-02-20 00:00:00
Tasks: General Classification

We study sparsification of convolutional neural networks (CNN) by a relaxed
variable splitting method of $\ell_0$ and transformed-$\ell_1$ (T$\ell_1$)
penalties, with application to complex curves such as texts written in
different fonts, and words written with trembling hands simulating those of
Parkinson's disease patients. The CNN contains 3 convolutional layers, each
followed by a maximum pooling, and finally a fully connected layer which
contains the largest number of network weights. With $\ell_0$ penalty, we
achieved over 99 \% test accuracy in distinguishing shaky vs. regular fonts or
hand writings with above 86 \% of the weights in the fully connected layer
being zero. Comparable sparsity and test accuracy are also reached with a
proper choice of T$\ell_1$ penalty.
A Novel Euler's Elastica based Segmentation Approach for Noisy Images via using the Progressive Hedging Algorithm
Authors: Lu Tan, Ling Li, Wanquan Liu, Jie Sun, Min Zhang
Date: 2019-02-20 00:00:00
Tasks: Segmentation

Euler's Elastica based unsupervised segmentation models have strong
capability of completing the missing boundaries for existing objects in a clean
image, but they are not working well for noisy images. This paper aims to
establish a Euler's Elastica based approach that properly deals with random
noises to improve the segmentation performance for noisy images. We solve the
corresponding optimization problem via using the progressive hedging algorithm
(PHA) with a step length suggested by the alternating direction method of
multipliers (ADMM). Technically, all the simplified convex versions of the
subproblems derived from the major framework of PHA can be obtained by using
the curvature weighted approach and the convex relaxation method. Then an
alternating optimization strategy is applied with the merits of using some
powerful accelerating techniques including the fast Fourier transform (FFT) and
generalized soft threshold formulas. Extensive experiments have been conducted
on both synthetic and real images, which validated some significant gains of
the proposed segmentation models and demonstrated the advantages of the
developed algorithm.
Deep Learning Based Video System for Accurate and Real-Time Parking Measurement
Authors: Bill Yang Cai, Ricardo Alvarez, Michelle Sit, Fábio Duarte, Carlo Ratti
Date: 2019-02-20 00:00:00
Tasks: Instance Segmentation, Semantic Segmentation

Parking spaces are costly to build, parking payments are difficult to
enforce, and drivers waste an excessive amount of time searching for empty
lots. Accurate quantification would inform developers and municipalities in
space allocation and design, while real-time measurements would provide drivers
and parking enforcement with information that saves time and resources. In this
paper, we propose an accurate and real-time video system for future Internet of
Things (IoT) and smart cities applications. Using recent developments in deep
convolutional neural networks (DCNNs) and a novel vehicle tracking filter, we
combine information across multiple image frames in a video sequence to remove
noise introduced by occlusions and detection failures. We demonstrate that our
system achieves higher accuracy than pure image-based instance segmentation,
and is comparable in performance to industry benchmark systems that utilize
more expensive sensors such as radar. Furthermore, our system shows significant
potential in its scalability to a city-wide scale and also in the richness of
its output that goes beyond traditional binary occupancy statistics.
LipschitzLR: Using theoretically computed adaptive learning rates for fast convergence
Authors: Rahul Yedida, Snehanshu Saha, Tejas Prashanth
Date: 2019-02-20 00:00:00
Tasks: Handwritten Digit Recognition, Object Detection

Optimizing deep neural networks is largely thought to be an empirical process, requiring manual tuning of several hyper-parameters, such as learning rate, weight decay, and dropout rate. Arguably, the learning rate is the most important of these to tune, and this has gained more attention in recent works. In this paper, we propose a novel method to compute the learning rate for training deep neural networks with stochastic gradient descent. We first derive a theoretical framework to compute learning rates dynamically based on the Lipschitz constant of the loss function. We then extend this framework to other commonly used optimization algorithms, such as gradient descent with momentum and Adam. We run an extensive set of experiments that demonstrate the efficacy of our approach on popular architectures and datasets, and show that commonly used learning rates are an order of magnitude smaller than the ideal value.
An Autoencoder-based Learned Image Compressor: Description of Challenge Proposal by NCTU
Authors: David Alexandre, Chih-Peng Chang, Wen-Hsiao Peng, Hsueh-Ming Hang
Date: 2019-02-20 00:00:00
Tasks: Image Compression, MS-SSIM, SSIM

We propose a lossy image compression system using the deep-learning
autoencoder structure to participate in the Challenge on Learned Image
Compression (CLIC) 2018. Our autoencoder uses the residual blocks with skip
connections to reduce the correlation among image pixels and condense the input
image into a set of feature maps, a compact representation of the original
image. The bit allocation and bitrate control are implemented by using the
importance maps and quantizer. The importance maps are generated by a separate
neural net in the encoder. The autoencoder and the importance net are trained
jointly based on minimizing a weighted sum of mean squared error, MS-SSIM, and
a rate estimate. Our aim is to produce reconstructed images with good
subjective quality subject to the 0.15 bits-per-pixel constraint.
Optimal Average-Case Reductions to Sparse PCA: From Weak Assumptions to Strong Hardness
Authors: Matthew Brennan, Guy Bresler
Date: 2019-02-20 00:00:00

In the past decade, sparse principal component analysis has emerged as an
archetypal problem for illustrating statistical-computational tradeoffs. This
trend has largely been driven by a line of research aiming to characterize the
average-case complexity of sparse PCA through reductions from the planted
clique (PC) conjecture - which conjectures that there is no polynomial-time
algorithm to detect a planted clique of size $K = o(N^{1/2})$ in
$\mathcal{G}(N, \frac{1}{2})$. All previous reductions to sparse PCA either
fail to show tight computational lower bounds matching existing algorithms or
show lower bounds for formulations of sparse PCA other than its canonical
generative model, the spiked covariance model. Also, these lower bounds all
quickly degrade with the exponent in the PC conjecture. Specifically, when only
given the PC conjecture up to $K = o(N^\alpha)$ where $\alpha < 1/2$, there is
no sparsity level $k$ at which these lower bounds remain tight. If $\alpha \le
1/3$ these reductions fail to even show the existence of a
statistical-computational tradeoff at any sparsity $k$. We give a reduction
from PC that yields the first full characterization of the computational
barrier in the spiked covariance model, providing tight lower bounds at all
sparsities $k$. We also show the surprising result that weaker forms of the PC
conjecture up to clique size $K = o(N^\alpha)$ for any given $\alpha \in (0,
1/2]$ imply tight computational lower bounds for sparse PCA at sparsities $k =
o(n^{\alpha/3})$. This shows that even a mild improvement in the signal
strength needed by the best known polynomial-time sparse PCA algorithms would
imply that the hardness threshold for PC is subpolynomial. This is the first
instance of a suboptimal hardness assumption implying optimal lower bounds for
another problem in unsupervised learning.
Automated Early Leaderboard Generation From Comparative Tables
Authors: Mayank Singh, Rajdeep Sarkar, Atharva Vyas, Pawan Goyal, Animesh Mukherjee, Soumen Chakrabarti
Date: 2018-02-13 00:00:00

A leaderboard is a tabular presentation of performance scores of the best
competing techniques that address a specific scientific problem. Manually
maintained leaderboards take time to emerge, which induces a latency in
performance discovery and meaningful comparison. This can delay dissemination
of best practices to non-experts and practitioners. Regarding papers as proxies
for techniques, we present a new system to automatically discover and maintain
leaderboards in the form of partial orders between papers, based on performance
reported therein. In principle, a leaderboard depends on the task, data set,
other experimental settings, and the choice of performance metrics. Often there
are also tradeoffs between different metrics. Thus, leaderboard discovery is
not just a matter of accurately extracting performance numbers and comparing
them. In fact, the levels of noise and uncertainty around performance
comparisons are so large that reliable traditional extraction is infeasible. We
mitigate these challenges by using relatively cleaner, structured parts of the
papers, e.g., performance tables. We propose a novel performance improvement
graph with papers as nodes, where edges encode noisy performance comparison
information extracted from tables. Every individual performance edge is
extracted from a table with citations to other papers. These extractions
resemble (noisy) outcomes of 'matches' in an incomplete tournament. We propose
several approaches to rank papers from these noisy 'match' outcomes. We show
that our ranking scheme can reproduce various manually curated leaderboards
very well. Using widely-used lists of state-of-the-art papers in 27 areas of
Computer Science, we demonstrate that our system produces very reliable
rankings.
Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting
Authors: Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, Deyu Meng
Date: 2019-02-20 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Image Classification, Meta-Learning

Current deep neural networks (DNNs) can easily overfit to biased training data with corrupted labels or class imbalance. Sample re-weighting strategy is commonly used to alleviate this issue by designing a weighting function mapping from training loss to sample weight, and then iterating between weight recalculating and classifier updating. Current approaches, however, need manually pre-specify the weighting function as well as its additional hyper-parameters. It makes them fairly hard to be generally applied in practice due to the significant variation of proper weighting schemes relying on the investigated problem and training data. To address this issue, we propose a method capable of adaptively learning an explicit weighting function directly from data. The weighting function is an MLP with one hidden layer, constituting a universal approximator to almost any continuous functions, making the method able to fit a wide range of weighting functions including those assumed in conventional research. Guided by a small amount of unbiased meta-data, the parameters of the weighting function can be finely updated simultaneously with the learning process of the classifiers. Synthetic and real experiments substantiate the capability of our method for achieving proper weighting functions in class imbalance and noisy label cases, fully complying with the common settings in traditional methods, and more complicated scenarios beyond conventional cases. This naturally leads to its better accuracy than other state-of-the-art methods.
Gaussian Process Priors for Dynamic Paired Comparison Modelling
Authors: Martin Ingram
Date: 2019-02-20 00:00:00
Tasks: Bayesian Inference, Bayesian Optimisation

Dynamic paired comparison models, such as Elo and Glicko, are frequently used
for sports prediction and ranking players or teams. We present an alternative
dynamic paired comparison model which uses a Gaussian Process (GP) as a prior
for the time dynamics rather than the Markovian dynamics usually assumed. In
addition, we show that the GP model can easily incorporate covariates. We
derive an efficient approximate Bayesian inference procedure based on the
Laplace Approximation and sparse linear algebra. We select hyperparameters by
maximising their marginal likelihood using Bayesian Optimisation, comparing the
results against random search. Finally, we fit and evaluate the model on the
2018 season of ATP tennis matches, where it performs competitively,
outperforming Elo and Glicko on log loss, particularly when surface covariates
are included.
Submodular Load Clustering with Robust Principal Component Analysis
Authors: Yishen Wang, Xiao Lu, Yiran Xu, Di Shi, Zhehan Yi, Jiajun Duan, Zhiwei Wang
Date: 2019-02-20 00:00:00
Tasks: Clustering, Load Forecasting

Traditional load analysis is facing challenges with the new electricity usage
patterns due to demand response as well as increasing deployment of distributed
generations, including photovoltaics (PV), electric vehicles (EV), and energy
storage systems (ESS). At the transmission system, despite of irregular load
behaviors at different areas, highly aggregated load shapes still share similar
characteristics. Load clustering is to discover such intrinsic patterns and
provide useful information to other load applications, such as load forecasting
and load modeling. This paper proposes an efficient submodular load clustering
method for transmission-level load areas. Robust principal component analysis
(R-PCA) firstly decomposes the annual load profiles into low-rank components
and sparse components to extract key features. A novel submodular cluster
center selection technique is then applied to determine the optimal cluster
centers through constructed similarity graph. Following the selection results,
load areas are efficiently assigned to different clusters for further load
analysis and applications. Numerical results obtained from PJM load demonstrate
the effectiveness of the proposed approach.
Utterance-level end-to-end language identification using attention-based CNN-BLSTM
Authors: Weicheng Cai, Danwei Cai, Shen Huang, Ming Li
Date: 2019-02-20 00:00:00
Tasks: Language Identification

In this paper, we present an end-to-end language identification framework,
the attention-based Convolutional Neural Network-Bidirectional Long-short Term
Memory (CNN-BLSTM). The model is performed on the utterance level, which means
the utterance-level decision can be directly obtained from the output of the
neural network. To handle speech utterances with entire arbitrary and
potentially long duration, we combine CNN-BLSTM model with a self-attentive
pooling layer together. The front-end CNN-BLSTM module plays a role as local
pattern extractor for the variable-length inputs, and the following
self-attentive pooling layer is built on top to get the fixed-dimensional
utterance-level representation. We conducted experiments on NIST LRE07
closed-set task, and the results reveal that the proposed attention-based
CNN-BLSTM model achieves comparable error reduction with other state-of-the-art
utterance-level neural network approaches for all 3 seconds, 10 seconds, 30
seconds duration tasks.
Learning Transferable Self-attentive Representations for Action Recognition in Untrimmed Videos with Weak Supervision
Authors: Xiao-Yu Zhang, Haichao Shi, Changsheng Li, Kai Zheng, Xiaobin Zhu, Lixin Duan
Date: 2019-02-20 00:00:00
Tasks: Action Recognition, Action Recognition In Videos, Temporal Action Localization

Action recognition in videos has attracted a lot of attention in the past
decade. In order to learn robust models, previous methods usually assume videos
are trimmed as short sequences and require ground-truth annotations of each
video frame/sequence, which is quite costly and time-consuming. In this paper,
given only video-level annotations, we propose a novel weakly supervised
framework to simultaneously locate action frames as well as recognize actions
in untrimmed videos. Our proposed framework consists of two major components.
First, for action frame localization, we take advantage of the self-attention
mechanism to weight each frame, such that the influence of background frames
can be effectively eliminated. Second, considering that there are trimmed
videos publicly available and also they contain useful information to leverage,
we present an additional module to transfer the knowledge from trimmed videos
for improving the classification performance in untrimmed ones. Extensive
experiments are conducted on two benchmark datasets (i.e., THUMOS14 and
ActivityNet1.3), and experimental results clearly corroborate the efficacy of
our method.
Human Motion Prediction via Learning Local Structure Representations and Temporal Dependencies
Authors: Xiao Guo, Jongmoo Choi
Date: 2019-02-20 00:00:00
Tasks: Human motion prediction, motion prediction

Human motion prediction from motion capture data is a classical problem in the computer vision, and conventional methods take the holistic human body as input. These methods ignore the fact that, in various human activities, different body components (limbs and the torso) have distinctive characteristics in terms of the moving pattern. In this paper, we argue local representations on different body components should be learned separately and, based on such idea, propose a network, Skeleton Network (SkelNet), for long-term human motion prediction. Specifically, at each time-step, local structure representations of input (human body) are obtained via SkelNet's branches of component-specific layers, then the shared layer uses local spatial representations to predict the future human pose. Our SkelNet is the first to use local structure representations for predicting the human motion. Then, for short-term human motion prediction, we propose the second network, named as Skeleton Temporal Network (Skel-TNet). Skel-TNet consists of three components: SkelNet and a Recurrent Neural Network, they have advantages in learning spatial and temporal dependencies for predicting human motion, respectively; a feed-forward network that outputs the final estimation. Our methods achieve promising results on the Human3.6M dataset and the CMU motion capture dataset.
Shallow Neural Networks for Fluid Flow Reconstruction with Limited Sensors
Authors: N. Benjamin Erichson, Lionel Mathelin, Zhewei Yao, Steven L. Brunton, Michael W. Mahoney, J. Nathan Kutz
Date: 2019-02-20 00:00:00

In many applications, it is important to reconstruct a fluid flow field, or some other high-dimensional state, from limited measurements and limited data. In this work, we propose a shallow neural network-based learning methodology for such fluid flow reconstruction. Our approach learns an end-to-end mapping between the sensor measurements and the high-dimensional fluid flow field, without any heavy preprocessing on the raw data. No prior knowledge is assumed to be available, and the estimation method is purely data-driven. We demonstrate the performance on three examples in fluid mechanics and oceanography, showing that this modern data-driven approach outperforms traditional modal approximation techniques which are commonly used for flow reconstruction. Not only does the proposed method show superior performance characteristics, it can also produce a comparable level of performance with traditional methods in the area, using significantly fewer sensors. Thus, the mathematical architecture is ideal for emerging global monitoring technologies where measurement data are often limited.
Video Face Recognition: Component-wise Feature Aggregation Network (C-FAN)
Authors: Sixue Gong, Yichun Shi, Anil K. Jain
Date: 2019-02-19 00:00:00
Tasks: Face Recognition

We propose a new approach to video face recognition. Our component-wise feature aggregation network (C-FAN) accepts a set of face images of a subject as an input, and outputs a single feature vector as the face representation of the set for the recognition task. The whole network is trained in two steps: (i) train a base CNN for still image face recognition; (ii) add an aggregation module to the base network to learn the quality value for each feature component, which adaptively aggregates deep feature vectors into a single vector to represent the face in a video. C-FAN automatically learns to retain salient face features with high quality scores while suppressing features with low quality scores. The experimental results on three benchmark datasets, YouTube Faces, IJB-A, and IJB-S show that the proposed C-FAN network is capable of generating a compact feature vector with 512 dimensions for a video sequence by efficiently aggregating feature vectors of all the video frames to achieve state of the art performance.