pwc-archive/papers-with-abstracts huggingface dataset:
On the convergence rate of the three operator splitting scheme
Authors: Fabian Pedregosa
Date: 2016-10-25 00:00:00

The three operator splitting scheme was recently proposed by [Davis and Yin, 2015] as a method to optimize composite objective functions with one convex smooth term and two convex (possibly non-smooth) terms for which we have access to their proximity operator. In this short note we provide an alternative proof for the sublinear rate of convergence of this method.
Robustness of Voice Conversion Techniques Under Mismatched Conditions
Authors: Monisankha Pal, Dipjyoti Paul, Md Sahidullah, Goutam Saha
Date: 2016-12-22 00:00:00
Tasks: Speech Enhancement, Voice Conversion

Most of the existing studies on voice conversion (VC) are conducted in
acoustically matched conditions between source and target signal. However, the
robustness of VC methods in presence of mismatch remains unknown. In this
paper, we report a comparative analysis of different VC techniques under
mismatched conditions. The extensive experiments with five different VC
techniques on CMU ARCTIC corpus suggest that performance of VC methods
substantially degrades in noisy conditions. We have found that bilinear
frequency warping with amplitude scaling (BLFWAS) outperforms other methods in
most of the noisy conditions. We further explore the suitability of different
speech enhancement techniques for robust conversion. The objective evaluation
results indicate that spectral subtraction and log minimum mean square error
(logMMSE) based speech enhancement techniques can be used to improve the
performance in specific noisy conditions.
Fast and Adaptive Sparse Precision Matrix Estimation in High Dimensions
Authors: Weidong Liu, Xi Luo
Date: 2012-03-17 00:00:00
Tasks: Vocal Bursts Intensity Prediction

This paper proposes a new method for estimating sparse precision matrices in
the high dimensional setting. It has been popular to study fast computation and
adaptive procedures for this problem. We propose a novel approach, called
Sparse Column-wise Inverse Operator, to address these two issues. We analyze an
adaptive procedure based on cross validation, and establish its convergence
rate under the Frobenius norm. The convergence rates under other matrix norms
are also established. This method also enjoys the advantage of fast computation
for large-scale problems, via a coordinate descent algorithm. Numerical merits
are illustrated using both simulated and real datasets. In particular, it
performs favorably on an HIV brain tissue dataset and an ADHD resting-state
fMRI dataset.
How to Train Your Deep Neural Network with Dictionary Learning
Authors: Vanika Singhal, Shikha Singh, Angshul Majumdar
Date: 2016-12-22 00:00:00
Tasks: Age And Gender Classification, Decoder, Dictionary Learning, Form, Gender Classification, General Classification

Currently there are two predominant ways to train deep neural networks. The
first one uses restricted Boltzmann machine (RBM) and the second one
autoencoders. RBMs are stacked in layers to form deep belief network (DBN); the
final representation layer is attached to the target to complete the deep
neural network. Autoencoders are nested one inside the other to form stacked
autoencoders; once the stcaked autoencoder is learnt the decoder portion is
detached and the target attached to the deepest layer of the encoder to form
the deep neural network. This work proposes a new approach to train deep neural
networks using dictionary learning as the basic building block; the idea is to
use the features from the shallower layer as inputs for training the next
deeper layer. One can use any type of dictionary learning (unsupervised,
supervised, discriminative etc.) as basic units till the pre-final layer. In
the final layer one needs to use the label consistent dictionary learning
formulation for classification. We compare our proposed framework with existing
state-of-the-art deep learning techniques on benchmark problems; we are always
within the top 10 results. In actual problems of age and gender classification,
we are better than the best known techniques.
Deep Blind Compressed Sensing
Authors: Shikha Singh, Vanika Singhal, Angshul Majumdar
Date: 2016-12-22 00:00:00
Tasks: compressed sensing

This work addresses the problem of extracting deeply learned features
directly from compressive measurements. There has been no work in this area.
Existing deep learning tools only give good results when applied on the full
signal, that too usually after preprocessing. These techniques require the
signal to be reconstructed first. In this work we show that by learning
directly from the compressed domain, considerably better results can be
obtained. This work extends the recently proposed framework of deep matrix
factorization in combination with blind compressed sensing; hence the term deep
blind compressed sensing. Simulation experiments have been carried out on
imaging via single pixel camera, under-sampled biomedical signals, arising in
wireless body area network and compressive hyperspectral imaging. In all cases,
the superiority of our proposed deep blind compressed sensing can be envisaged.
Cross-convolutional-layer Pooling for Image Recognition
Authors: Lingqiao Liu, Chunhua Shen, Anton Van Den Hengel
Date: 2015-10-04 00:00:00
Tasks: General Classification, image-classification, Image Classification

Recent studies have shown that a Deep Convolutional Neural Network (DCNN)
pretrained on a large image dataset can be used as a universal image
descriptor, and that doing so leads to impressive performance for a variety of
image classification tasks. Most of these studies adopt activations from a
single DCNN layer, usually the fully-connected layer, as the image
representation. In this paper, we proposed a novel way to extract image
representations from two consecutive convolutional layers: one layer is
utilized for local feature extraction and the other serves as guidance to pool
the extracted features. By taking different viewpoints of convolutional layers,
we further develop two schemes to realize this idea. The first one directly
uses convolutional layers from a DCNN. The second one applies the pretrained
CNN on densely sampled image regions and treats the fully-connected activations
of each image region as convolutional feature activations. We then train
another convolutional layer on top of that as the pooling-guidance
convolutional layer. By applying our method to three popular visual
classification tasks, we find our first scheme tends to perform better on the
applications which need strong discrimination on subtle object patterns within
small regions while the latter excels in the cases that require discrimination
on category-level patterns. Overall, the proposed method achieves superior
performance over existing ways of extracting image representations from a DCNN.
Probabilistic graphical model based approach for water mapping using GaoFen-2 (GF-2) high resolution imagery and Landsat 8 time series
Authors: Luyan Ji, Jie Wang, Xiurui Geng, Peng Gong
Date: 2016-12-22 00:00:00
Tasks: Time Series, Time Series Analysis

The objective of this paper is to evaluate the potential of Gaofen-2 (GF-2)
high resolution multispectral sensor (MS) and panchromatic (PAN) imagery on
water mapping. Difficulties of water mapping on high resolution data includes:
1) misclassification between water and shadows or other low-reflectance ground
objects, which is mostly caused by the spectral similarity within the given
band range; 2) small water bodies with size smaller than the spatial resolution
of MS image. To solve the confusion between water and low-reflectance objects,
the Landsat 8 time series with two shortwave infrared (SWIR) bands is added
because water has extremely strong absorption in SWIR. In order to integrate
the three multi-sensor, multi-resolution data sets, the probabilistic graphical
model (PGM) is utilized here with conditional probability distribution defined
mainly based on the size of each object. For comparison, results from the SVM
classifier on the PCA fused and MS data, thresholding method on the PAN image,
and water index method on the Landsat data are computed. The confusion matrices
are calculated for all the methods. The results demonstrate that the PGM method
can achieve the best performance with the highest overall accuracy. Moreover,
small rivers can also be extracted by adding weight on the PAN result in PGM.
Finally, the post-classification procedure is applied on the PGM result to
further exclude misclassification in shadow and water-land boundary regions.
Accordingly, the producer's, user's and overall accuracy are all increased,
indicating the effectiveness of our method.
Commonsense Knowledge Enhanced Embeddings for Solving Pronoun Disambiguation Problems in Winograd Schema Challenge
Authors: Quan Liu, Hui Jiang, Zhen-Hua Ling, Xiaodan Zhu, Si Wei, Yu Hu
Date: 2016-11-13 00:00:00
Tasks: coreference-resolution, Coreference Resolution

In this paper, we propose commonsense knowledge enhanced embeddings (KEE) for
solving the Pronoun Disambiguation Problems (PDP). The PDP task we investigate
in this paper is a complex coreference resolution task which requires the
utilization of commonsense knowledge. This task is a standard first round test
set in the 2016 Winograd Schema Challenge. In this task, traditional linguistic
features that are useful for coreference resolution, e.g. context and gender
information, are no longer effective anymore. Therefore, the KEE models are
proposed to provide a general framework to make use of commonsense knowledge
for solving the PDP problems. Since the PDP task doesn't have training data,
the KEE models would be used during the unsupervised feature extraction
process. To evaluate the effectiveness of the KEE models, we propose to
incorporate various commonsense knowledge bases, including ConceptNet, WordNet,
and CauseCom, into the KEE training process. We achieved the best performance
by applying the proposed methods to the 2016 Winograd Schema Challenge. In
addition, experiments conducted on the standard PDP task indicate that, the
proposed KEE models could solve the PDP problems by achieving 66.7% accuracy,
which is a new state-of-the-art performance.
Automatic Neuron Detection in Calcium Imaging Data Using Convolutional Networks
Authors: Noah J. Apthorpe, Alexander J. Riordan, Rob E. Aguilar, Jan Homann, Yi Gu, David W. Tank, H. Sebastian Seung
Date: 2016-06-23 00:00:00
Proceeding: NeurIPS 2016 12

Calcium imaging is an important technique for monitoring the activity of
thousands of neurons simultaneously. As calcium imaging datasets grow in size,
automated detection of individual neurons is becoming important. Here we apply
a supervised learning approach to this problem and show that convolutional
networks can achieve near-human accuracy and superhuman speed. Accuracy is
superior to the popular PCA/ICA method based on precision and recall relative
to ground truth annotation by a human expert. These results suggest that
convolutional networks are an efficient and flexible tool for the analysis of
large-scale calcium imaging data.
Detecting Unusual Input-Output Associations in Multivariate Conditional Data
Authors: Charmgil Hong, Milos Hauskrecht
Date: 2016-12-21 00:00:00
Tasks: Outlier Detection

Despite tremendous progress in outlier detection research in recent years,
the majority of existing methods are designed only to detect unconditional
outliers that correspond to unusual data patterns expressed in the joint space
of all data attributes. Such methods are not applicable when we seek to detect
conditional outliers that reflect unusual responses associated with a given
context or condition. This work focuses on multivariate conditional outlier
detection, a special type of the conditional outlier detection problem, where
data instances consist of multi-dimensional input (context) and output
(responses) pairs. We present a novel outlier detection framework that
identifies abnormal input-output associations in data with the help of a
decomposable conditional probabilistic model that is learned from all data
instances. Since components of this model can vary in their quality, we combine
them with the help of weights reflecting their reliability in assessment of
outliers. We study two ways of calculating the component weights: global that
relies on all data, and local that relies only on instances similar to the
target instance. Experimental results on data from various domains demonstrate
the ability of our framework to successfully identify multivariate conditional
outliers.
Distributed Dictionary Learning
Authors: Amir Daneshmand, Gesualdo Scutari, Francisco Facchinei
Date: 2016-12-21 00:00:00
Tasks: Dictionary Learning

The paper studies distributed Dictionary Learning (DL) problems where the
learning task is distributed over a multi-agent network with time-varying
(nonsymmetric) connectivity. This formulation is relevant, for instance, in
big-data scenarios where massive amounts of data are collected/stored in
different spatial locations and it is unfeasible to aggregate and/or process
all the data in a fusion center, due to resource limitations, communication
overhead or privacy considerations. We develop a general distributed
algorithmic framework for the (nonconvex) DL problem and establish its
asymptotic convergence. The new method hinges on Successive Convex
Approximation (SCA) techniques coupled with i) a gradient tracking mechanism
instrumental to locally estimate the missing global information; and ii) a
consensus step, as a mechanism to distribute the computations among the agents.
To the best of our knowledge, this is the first distributed algorithm with
provable convergence for the DL problem and, more in general, bi-convex
optimization problems over (time-varying) directed graphs.
Understanding Error Correction and its Role as Part of the Communication Channel in Environments composed of Self-Integrating Systems
Authors: Aleksander Lodwich
Date: 2016-12-21 00:00:00

The raise of complexity of technical systems also raises knowledge required
to set them up and to maintain them. The cost to evolve such systems can be
prohibitive. In the field of Autonomic Computing, technical systems should
therefore have various self-healing capabilities allowing system owners to
provide only partial, potentially inconsistent updates of the system. The
self-healing or self-integrating system shall find out the remaining changes to
communications and functionalities in order to accommodate change and yet still
restore function. This issue becomes even more interesting in context of
Internet of Things and Industrial Internet where previously unexpected device
combinations can be assembled in order to provide a surprising new function. In
order to pursue higher levels of self-integration capabilities I propose to
think of self-integration as sophisticated error correcting communications.
Therefore, this paper discusses an extended scope of error correction with the
purpose to emphasize error correction's role as an integrated element of
bi-directional communication channels in self-integrating, autonomic
communication scenarios.
Direct Feedback Alignment Provides Learning in Deep Neural Networks
Authors: Arild Nøkland
Date: 2016-09-06 00:00:00
Proceeding: NeurIPS 2016 12

Artificial neural networks are most commonly trained with the
back-propagation algorithm, where the gradient for learning is provided by
back-propagating the error, layer by layer, from the output layer to the hidden
layers. A recently discovered method called feedback-alignment shows that the
weights used for propagating the error backward don't have to be symmetric with
the weights used for propagation the activation forward. In fact, random
feedback weights work evenly well, because the network learns how to make the
feedback useful. In this work, the feedback alignment principle is used for
training hidden layers more independently from the rest of the network, and
from a zero initial condition. The error is propagated through fixed random
feedback connections directly from the output layer to each hidden layer. This
simple method is able to achieve zero training error even in convolutional
networks and very deep networks, completely without error back-propagation. The
method is a step towards biologically plausible machine learning because the
error signal is almost local, and no symmetric or reciprocal weights are
required. Experiments show that the test performance on MNIST and CIFAR is
almost as good as those obtained with back-propagation for fully connected
networks. If combined with dropout, the method achieves 1.45% error on the
permutation invariant MNIST task.
Revisiting Causality Inference in Memory-less Transition Networks
Authors: Abbas Shojaee, Isuru Ranasinghe, Alireza Ani
Date: 2016-08-08 00:00:00
Tasks: Time Series, Time Series Analysis

Several methods exist to infer causal networks from massive volumes of
observational data. However, almost all existing methods require a considerable
length of time series data to capture cause and effect relationships. In
contrast, memory-less transition networks or Markov Chain data, which refers to
one-step transitions to and from an event, have not been explored for causality
inference even though such data is widely available. We find that causal
network can be inferred from characteristics of four unique distribution zones
around each event. We call this Composition of Transitions and show that cause,
effect, and random events exhibit different behavior in their compositions. We
applied machine learning models to learn these different behaviors and to infer
causality. We name this new method Causality Inference using Composition of
Transitions (CICT). To evaluate CICT, we used an administrative inpatient
healthcare dataset to set up a network of patients transitions between
different diagnoses. We show that CICT is highly accurate in inferring whether
the transition between a pair of events is causal or random and performs well
in identifying the direction of causality in a bi-directional association.
Bayesian Decision Process for Cost-Efficient Dynamic Ranking via Crowdsourcing
Authors: Xi Chen, Kevin Jiao, Qihang Lin
Date: 2016-12-21 00:00:00

Rank aggregation based on pairwise comparisons over a set of items has a wide
range of applications. Although considerable research has been devoted to the
development of rank aggregation algorithms, one basic question is how to
efficiently collect a large amount of high-quality pairwise comparisons for the
ranking purpose. Because of the advent of many crowdsourcing services, a crowd
of workers are often hired to conduct pairwise comparisons with a small
monetary reward for each pair they compare. Since different workers have
different levels of reliability and different pairs have different levels of
ambiguity, it is desirable to wisely allocate the limited budget for
comparisons among the pairs of items and workers so that the global ranking can
be accurately inferred from the comparison results. To this end, we model the
active sampling problem in crowdsourced ranking as a Bayesian Markov decision
process, which dynamically selects item pairs and workers to improve the
ranking accuracy under a budget constraint. We further develop a
computationally efficient sampling policy based on knowledge gradient as well
as a moment matching technique for posterior approximation. Experimental
evaluations on both synthetic and real data show that the proposed policy
achieves high ranking accuracy with a lower labeling cost.
Image Segmentation Using Overlapping Group Sparsity
Authors: Shervin Minaee, Yao Wang
Date: 2016-11-23 00:00:00
Tasks: Clustering, Denoising, image-classification, Image Classification, Image Denoising, Image Segmentation, Segmentation, Semantic Segmentation

Sparse decomposition has been widely used for different applications, such as
source separation, image classification and image denoising. This paper
presents a new algorithm for segmentation of an image into background and
foreground text and graphics using sparse decomposition. First, the background
is represented using a suitable smooth model, which is a linear combination of
a few smoothly varying basis functions, and the foreground text and graphics
are modeled as a sparse component overlaid on the smooth background. Then the
background and foreground are separated using a sparse decomposition framework
and imposing some prior information, which promote the smoothness of
background, and the sparsity and connectivity of foreground pixels. This
algorithm has been tested on a dataset of images extracted from HEVC standard
test sequences for screen content coding, and is shown to outperform prior
methods, including least absolute deviation fitting, k-means clustering based
segmentation in DjVu, and shape primitive extraction and coding algorithm.
Trilaminar Multiway Reconstruction Tree for Efficient Large Scale Structure from Motion
Authors: Kun Sun, Wenbing Tao
Date: 2016-12-21 00:00:00

Accuracy and efficiency are two key problems in large scale incremental
Structure from Motion (SfM). In this paper, we propose a unified framework to
divide the image set into clusters suitable for reconstruction as well as find
multiple reliable and stable starting points. Image partitioning performs in
two steps. First, some small image groups are selected at places with high
image density, and then all the images are clustered according to their optimal
reconstruction paths to these image groups. This promises that the scene is
always reconstructed from dense places to sparse areas, which can reduce error
accumulation when images have weak overlap. To enable faster speed, images
outside the selected group in each cluster are further divided to achieve a
greater degree of parallelism. Experiments show that our method achieves
significant speedup, higher accuracy and better completeness.
Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling
Authors: Gábor Berend
Date: 2016-12-21 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Part-Of-Speech Tagging, POS, POS Tagging, Word Embeddings

In this paper we propose and carefully evaluate a sequence labeling framework
which solely utilizes sparse indicator features derived from dense distributed
word representations. The proposed model obtains (near) state-of-the art
performance for both part-of-speech tagging and named entity recognition for a
variety of languages. Our model relies only on a few thousand sparse
coding-derived features, without applying any modification of the word
representations employed for the different tasks. The proposed model has
favorable generalization properties as it retains over 89.8% of its average POS
tagging accuracy when trained at 1.2% of the total available training data,
i.e.~150 sentences per language.
Towards Wide Learning: Experiments in Healthcare
Authors: Snehasis Banerjee, Tanushyam Chattopadhyay, Swagata Biswas, Rohan Banerjee, Anirban Dutta Choudhury, Arpan Pal, Utpal Garain
Date: 2016-12-17 00:00:00
Tasks: Emotion Classification, Feature Engineering, General Classification

In this paper, a Wide Learning architecture is proposed that attempts to
automate the feature engineering portion of the machine learning (ML) pipeline.
Feature engineering is widely considered as the most time consuming and expert
knowledge demanding portion of any ML task. The proposed feature recommendation
approach is tested on 3 healthcare datasets: a) PhysioNet Challenge 2016
dataset of phonocardiogram (PCG) signals, b) MIMIC II blood pressure
classification dataset of photoplethysmogram (PPG) signals and c) an emotion
classification dataset of PPG signals. While the proposed method beats the
state of the art techniques for 2nd and 3rd dataset, it reaches 94.38% of the
accuracy level of the winner of PhysioNet Challenge 2016. In all cases, the
effort to reach a satisfactory performance was drastically less (a few days)
than manual feature engineering.