pwc-archive/papers-with-abstracts huggingface dataset:
Training Generative Adversarial Networks via Primal-Dual Subgradient Methods: A Lagrangian Perspective on GAN
Authors: Xu Chen, Jiang Wang, Hao Ge
Date: 2018-02-06 00:00:00
Proceeding: ICLR 2018 1

We relate the minimax game of generative adversarial networks (GANs) to
finding the saddle points of the Lagrangian function for a convex optimization
problem, where the discriminator outputs and the distribution of generator
outputs play the roles of primal variables and dual variables, respectively.
This formulation shows the connection between the standard GAN training process
and the primal-dual subgradient methods for convex optimization. The inherent
connection does not only provide a theoretical convergence proof for training
GANs in the function space, but also inspires a novel objective function for
training. The modified objective function forces the distribution of generator
outputs to be updated along the direction according to the primal-dual
subgradient methods. A toy example shows that the proposed method is able to
resolve mode collapse, which in this case cannot be avoided by the standard GAN
or Wasserstein GAN. Experiments on both Gaussian mixture synthetic data and
real-world image datasets demonstrate the performance of the proposed method on
generating diverse samples.
Recovering Structured Probability Matrices
Authors: Qingqing Huang, Sham M. Kakade, Weihao Kong, Gregory Valiant
Date: 2016-02-21 00:00:00
Tasks: Collaborative Filtering, Community Detection, Recommendation Systems, Topic Models, Word Embeddings

We consider the problem of accurately recovering a matrix B of size M by M ,
which represents a probability distribution over M2 outcomes, given access to
an observed matrix of "counts" generated by taking independent samples from the
distribution B. How can structural properties of the underlying matrix B be
leveraged to yield computationally efficient and information theoretically
optimal reconstruction algorithms? When can accurate reconstruction be
accomplished in the sparse data regime? This basic problem lies at the core of
a number of questions that are currently being considered by different
communities, including building recommendation systems and collaborative
filtering in the sparse data regime, community detection in sparse random
graphs, learning structured models such as topic models or hidden Markov
models, and the efforts from the natural language processing community to
compute "word embeddings".
  Our results apply to the setting where B has a low rank structure. For this
setting, we propose an efficient algorithm that accurately recovers the
underlying M by M matrix using Theta(M) samples. This result easily translates
to Theta(M) sample algorithms for learning topic models and learning hidden
Markov Models. These linear sample complexities are optimal, up to constant
factors, in an extremely strong sense: even testing basic properties of the
underlying matrix (such as whether it has rank 1 or 2) requires Omega(M)
samples. We provide an even stronger lower bound where distinguishing whether a
sequence of observations were drawn from the uniform distribution over M
observations versus being generated by an HMM with two hidden states requires
Omega(M) observations. This precludes sublinear-sample hypothesis tests for
basic properties, such as identity or uniformity, as well as sublinear sample
estimators for quantities such as the entropy rate of HMMs.
DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
Authors: Jingjing Xu, Xuancheng Ren, Junyang Lin, Xu sun
Date: 2018-02-05 00:00:00
Tasks: Dialogue Generation, Diversity, Generative Adversarial Network, Language Modeling, Language Modelling, Review Generation, Text Generation

Existing text generation methods tend to produce repeated and "boring"
expressions. To tackle this problem, we propose a new text generation model,
called Diversity-Promoting Generative Adversarial Network (DP-GAN). The
proposed model assigns low reward for repeatedly generated text and high reward
for "novel" and fluent text, encouraging the generator to produce diverse and
informative text. Moreover, we propose a novel language-model based
discriminator, which can better distinguish novel text from repeated text
without the saturation problem compared with existing classifier-based
discriminators. The experimental results on review generation and dialogue
generation tasks demonstrate that our model can generate substantially more
diverse and informative text than existing baselines. The code is available at
https://github.com/lancopku/DPGAN
Highly accurate model for prediction of lung nodule malignancy with CT scans
Authors: Jason Causey, Junyu Zhang, Shiqian Ma, Bo Jiang, Jake Qualls, David G. Politte, Fred Prior, Shuzhong Zhang, Xiuzhen Huang
Date: 2018-02-06 00:00:00
Tasks: Computed Tomography (CT)

Computed tomography (CT) examinations are commonly used to predict lung
nodule malignancy in patients, which are shown to improve noninvasive early
diagnosis of lung cancer. It remains challenging for computational approaches
to achieve performance comparable to experienced radiologists. Here we present
NoduleX, a systematic approach to predict lung nodule malignancy from CT data,
based on deep learning convolutional neural networks (CNN). For training and
validation, we analyze >1000 lung nodules in images from the LIDC/IDRI cohort.
All nodules were identified and classified by four experienced thoracic
radiologists who participated in the LIDC project. NoduleX achieves high
accuracy for nodule malignancy classification, with an AUC of ~0.99. This is
commensurate with the analysis of the dataset by experienced radiologists. Our
approach, NoduleX, provides an effective framework for highly accurate nodule
malignancy prediction with the model trained on a large patient population. Our
results are replicable with software available at
http://bioinformatics.astate.edu/NoduleX.
Hybrid Oracle: Making Use of Ambiguity in Transition-based Chinese Dependency Parsing
Authors: Xuancheng Ren, Xu sun
Date: 2017-11-28 00:00:00
Tasks: Chinese Dependency Parsing, Dependency Parsing, Sentence

In the training of transition-based dependency parsers, an oracle is used to
predict a transition sequence for a sentence and its gold tree. However, the
transition system may exhibit ambiguity, that is, there can be multiple correct
transition sequences that form the gold tree. We propose to make use of the
property in the training of neural dependency parsers, and present the Hybrid
Oracle. The new oracle gives all the correct transitions for a parsing state,
which are used in the cross entropy loss function to provide better supervisory
signal. It is also used to generate different transition sequences for a
sentence to better explore the training data and improve the generalization
ability of the parser. Evaluations show that the parsers trained using the
hybrid oracle outperform the parsers using the traditional oracle in Chinese
dependency parsing. We provide analysis from a linguistic view. The code is
available at https://github.com/lancopku/nndep .
Bootstrapping and Multiple Imputation Ensemble Approaches for Missing Data
Authors: Shehroz S. Khan, Amir Ahmad, Alex Mihailidis
Date: 2018-02-01 00:00:00
Tasks: Diversity, Imputation, Missing Values

Presence of missing values in a dataset can adversely affect the performance of a classifier. Single and Multiple Imputation are normally performed to fill in the missing values. In this paper, we present several variants of combining single and multiple imputation with bootstrapping to create ensembles that can model uncertainty and diversity in the data, and that are robust to high missingness in the data. We present three ensemble strategies: bootstrapping on incomplete data followed by (i) single imputation and (ii) multiple imputation, and (iii) multiple imputation ensemble without bootstrapping. We perform an extensive evaluation of the performance of the these ensemble strategies on 8 datasets by varying the missingness ratio. Our results show that bootstrapping followed by multiple imputation using expectation maximization is the most robust method even at high missingness ratio (up to 30%). For small missingness ratio (up to 10%) most of the ensemble methods perform quivalently but better than single imputation. Kappa-error plots suggest that accurate classifiers with reasonable diversity is the reason for this behaviour. A consistent observation in all the datasets suggests that for small missingness (up to 10%), bootstrapping on incomplete data without any imputation produces equivalent results to other ensemble methods.
Toward Marker-free 3D Pose Estimation in Lifting: A Deep Multi-view Solution
Authors: Rahil Mehrizi, Xi Peng, Zhiqiang Tang, Xu Xu, Dimitris Metaxas, Kang Li
Date: 2018-02-06 00:00:00
Tasks: 3D Pose Estimation, Pose Estimation

Lifting is a common manual material handling task performed in the
workplaces. It is considered as one of the main risk factors for Work-related
Musculoskeletal Disorders. To improve work place safety, it is necessary to
assess musculoskeletal and biomechanical risk exposures associated with these
tasks, which requires very accurate 3D pose. Existing approaches mainly utilize
marker-based sensors to collect 3D information. However, these methods are
usually expensive to setup, time-consuming in process, and sensitive to the
surrounding environment. In this study, we propose a multi-view based deep
perceptron approach to address aforementioned limitations. Our approach
consists of two modules: a "view-specific perceptron" network extracts rich
information independently from the image of view, which includes both 2D shape
and hierarchical texture information; while a "multi-view integration" network
synthesizes information from all available views to predict accurate 3D pose.
To fully evaluate our approach, we carried out comprehensive experiments to
compare different variants of our design. The results prove that our approach
achieves comparable performance with former marker-based methods, i.e. an
average error of $14.72 \pm 2.96$ mm on the lifting dataset. The results are
also compared with state-of-the-art methods on HumanEva-I dataset, which
demonstrates the superior performance of our approach.
Local Wealth Redistribution Promotes Cooperation in Multiagent Systems
Authors: Flávio L. Pinheiro, Fernando P. Santos
Date: 2018-02-05 00:00:00

Designing mechanisms that leverage cooperation between agents has been a
long-lasting goal in Multiagent Systems. The task is especially challenging
when agents are selfish, lack common goals and face social dilemmas, i.e.,
situations in which individual interest conflicts with social welfare. Past
works explored mechanisms that explain cooperation in biological and social
systems, providing important clues for the aim of designing cooperative
artificial societies. In particular, several works show that cooperation is
able to emerge when specific network structures underlie agents' interactions.
Notwithstanding, social dilemmas in which defection is highly tempting still
pose challenges concerning the effective sustainability of cooperation. Here we
propose a new redistribution mechanism that can be applied in structured
populations of agents. Importantly, we show that, when implemented locally
(i.e., agents share a fraction of their wealth surplus with their nearest
neighbors), redistribution excels in promoting cooperation under regimes where,
before, only defection prevailed.
Identity-preserving Face Recovery from Portraits
Authors: Fatemeh Shiri, Xin Yu, Fatih Porikli, Richard Hartley, Piotr Koniusz
Date: 2018-01-08 00:00:00

Recovering the latent photorealistic faces from their artistic portraits aids
human perception and facial analysis. However, a recovery process that can
preserve identity is challenging because the fine details of real faces can be
distorted or lost in stylized images. In this paper, we present a new
Identity-preserving Face Recovery from Portraits (IFRP) to recover latent
photorealistic faces from unaligned stylized portraits. Our IFRP method
consists of two components: Style Removal Network (SRN) and Discriminative
Network (DN). The SRN is designed to transfer feature maps of stylized images
to the feature maps of the corresponding photorealistic faces. By embedding
spatial transformer networks into the SRN, our method can compensate for
misalignments of stylized faces automatically and output aligned realistic face
images. The role of the DN is to enforce recovered faces to be similar to
authentic faces. To ensure the identity preservation, we promote the recovered
and ground-truth faces to share similar visual features via a distance measure
which compares features of recovered and ground-truth faces extracted from a
pre-trained VGG network. We evaluate our method on a large-scale synthesized
dataset of real and stylized face pairs and attain state of the art results. In
addition, our method can recover photorealistic faces from previously unseen
stylized portraits, original paintings and human-drawn sketches.
Compressive Light Field Reconstructions using Deep Learning
Authors: Mayank Gupta, Arjun Jauhari, Kuldeep Kulkarni, Suren Jayasuriya, Alyosha Molnar, Pavan Turaga
Date: 2018-02-05 00:00:00
Tasks: Compressive Sensing, Deep Learning

Light field imaging is limited in its computational processing demands of
high sampling for both spatial and angular dimensions. Single-shot light field
cameras sacrifice spatial resolution to sample angular viewpoints, typically by
multiplexing incoming rays onto a 2D sensor array. While this resolution can be
recovered using compressive sensing, these iterative solutions are slow in
processing a light field. We present a deep learning approach using a new, two
branch network architecture, consisting jointly of an autoencoder and a 4D CNN,
to recover a high resolution 4D light field from a single coded 2D image. This
network decreases reconstruction time significantly while achieving average
PSNR values of 26-32 dB on a variety of light fields. In particular,
reconstruction time is decreased from 35 minutes to 6.7 minutes as compared to
the dictionary method for equivalent visual quality. These reconstructions are
performed at small sampling/compression ratios as low as 8%, allowing for
cheaper coded light field cameras. We test our network reconstructions on
synthetic light fields, simulated coded measurements of real light fields
captured from a Lytro Illum camera, and real coded images from a custom CMOS
diffractive light field camera. The combination of compressive light field
capture with deep learning allows the potential for real-time light field video
acquisition systems in the future.
Equivalence of restricted Boltzmann machines and tensor network states
Authors: Jing Chen, Song Cheng, Haidong Xie, Lei Wang, Tao Xiang
Date: 2017-01-17 00:00:00
Tasks: Deep Learning, Recommendation Systems

The restricted Boltzmann machine (RBM) is one of the fundamental building
blocks of deep learning. RBM finds wide applications in dimensional reduction,
feature extraction, and recommender systems via modeling the probability
distributions of a variety of input data including natural images, speech
signals, and customer ratings, etc. We build a bridge between RBM and tensor
network states (TNS) widely used in quantum many-body physics research. We
devise efficient algorithms to translate an RBM into the commonly used TNS.
Conversely, we give sufficient and necessary conditions to determine whether a
TNS can be transformed into an RBM of given architectures. Revealing these
general and constructive connections can cross-fertilize both deep learning and
quantum many-body physics. Notably, by exploiting the entanglement entropy
bound of TNS, we can rigorously quantify the expressive power of RBM on complex
data sets. Insights into TNS and its entanglement capacity can guide the design
of more powerful deep learning architectures. On the other hand, RBM can
represent quantum many-body states with fewer parameters compared to TNS, which
may allow more efficient classical simulations.
Self-Supervised Intrinsic Image Decomposition
Authors: Michael Janner, Jiajun Wu, Tejas D. Kulkarni, Ilker Yildirim, Joshua B. Tenenbaum
Date: 2017-11-10 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Intrinsic Image Decomposition, Transfer Learning

Intrinsic decomposition from a single image is a highly challenging task, due
to its inherent ambiguity and the scarcity of training data. In contrast to
traditional fully supervised learning approaches, in this paper we propose
learning intrinsic image decomposition by explaining the input image. Our
model, the Rendered Intrinsics Network (RIN), joins together an image
decomposition pipeline, which predicts reflectance, shape, and lighting
conditions given a single image, with a recombination function, a learned
shading model used to recompose the original input based off of intrinsic image
predictions. Our network can then use unsupervised reconstruction error as an
additional signal to improve its intermediate representations. This allows
large-scale unlabeled data to be useful during training, and also enables
transferring learned knowledge to images of unseen object categories, lighting
conditions, and shapes. Extensive experiments demonstrate that our method
performs well on both intrinsic image decomposition and knowledge transfer.
Weakly-supervised Dictionary Learning
Authors: Zeyu You, Raviv Raich, Xiaoli Z. Fern, Jinsub Kim
Date: 2018-02-05 00:00:00
Tasks: Denoising, Dictionary Learning, General Classification, Time Series, Time Series Analysis

We present a probabilistic modeling and inference framework for
discriminative analysis dictionary learning under a weak supervision setting.
Dictionary learning approaches have been widely used for tasks such as
low-level signal denoising and restoration as well as high-level classification
tasks, which can be applied to audio and image analysis. Synthesis dictionary
learning aims at jointly learning a dictionary and corresponding sparse
coefficients to provide accurate data representation. This approach is useful
for denoising and signal restoration, but may lead to sub-optimal
classification performance. By contrast, analysis dictionary learning provides
a transform that maps data to a sparse discriminative representation suitable
for classification. We consider the problem of analysis dictionary learning for
time-series data under a weak supervision setting in which signals are assigned
with a global label instead of an instantaneous label signal. We propose a
discriminative probabilistic model that incorporates both label information and
sparsity constraints on the underlying latent instantaneous label signal using
cardinality control. We present the expectation maximization (EM) procedure for
maximum likelihood estimation (MLE) of the proposed model. To facilitate a
computationally efficient E-step, we propose both a chain and a novel tree
graph reformulation of the graphical model. The performance of the proposed
model is demonstrated on both synthetic and real-world data.
Artificial neural network based modelling approach for municipal solid waste gasification in a fluidized bed reactor
Authors: Daya Shankar Pandey, Saptarshi Das, Indranil Pan, James J. Leahy, Witold Kwapinski
Date: 2018-02-05 00:00:00
Tasks: Model Selection

In this paper, multi-layer feed forward neural networks are used to predict
the lower heating value of gas (LHV), lower heating value of gasification
products including tars and entrained char (LHVp) and syngas yield during
gasification of municipal solid waste (MSW) during gasification in a fluidized
bed reactor. These artificial neural networks (ANNs) with different
architectures are trained using the Levenberg-Marquardt (LM) back-propagation
algorithm and a cross validation is also performed to ensure that the results
generalise to other unseen datasets. A rigorous study is carried out on
optimally choosing the number of hidden layers, number of neurons in the hidden
layer and activation function in a network using multiple Monte Carlo runs.
Nine input and three output parameters are used to train and test various
neural network architectures in both multiple output and single output
prediction paradigms using the available experimental datasets. The model
selection procedure is carried out to ascertain the best network architecture
in terms of predictive accuracy. The simulation results show that the ANN based
methodology is a viable alternative which can be used to predict the
performance of a fluidized bed gasifier.
Deep Learning with a Rethinking Structure for Multi-label Classification
Authors: Yao-Yuan Yang, Yi-An Lin, Hong-Min Chu, Hsuan-Tien Lin
Date: 2018-02-05 00:00:00
Tasks: Classification, Deep Learning, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

Multi-label classification (MLC) is an important class of machine learning problems that come with a wide spectrum of applications, each demanding a possibly different evaluation criterion. When solving the MLC problems, we generally expect the learning algorithm to take the hidden correlation of the labels into account to improve the prediction performance. Extracting the hidden correlation is generally a challenging task. In this work, we propose a novel deep learning framework to better extract the hidden correlation with the help of the memory structure within recurrent neural networks. The memory stores the temporary guesses on the labels and effectively allows the framework to rethink about the goodness and correlation of the guesses before making the final prediction. Furthermore, the rethinking process makes it easy to adapt to different evaluation criteria to match real-world application needs. In particular, the framework can be trained in an end-to-end style with respect to any given MLC evaluation criteria. The end-to-end design can be seamlessly combined with other deep learning techniques to conquer challenging MLC problems like image tagging. Experimental results across many real-world data sets justify that the rethinking framework indeed improves MLC performance across different evaluation criteria and leads to superior performance over state-of-the-art MLC algorithms.
On the Feasibility of Generic Deep Disaggregation for Single-Load Extraction
Authors: Karim Said Barsim, Bin Yang
Date: 2018-02-05 00:00:00

Recently, and with the growing development of big energy datasets,
data-driven learning techniques began to represent a potential solution to the
energy disaggregation problem outperforming engineered and hand-crafted models.
However, most proposed deep disaggregation models are load-dependent in the
sense that either expert knowledge or a hyper-parameter optimization stage is
required prior to training and deployment (normally for each load category)
even upon acquisition and cleansing of aggregate and sub-metered data. In this
paper, we present a feasibility study on the development of a generic
disaggregation model based on data-driven learning. Specifically, we present a
generic deep disaggregation model capable of achieving state-of-art performance
in load monitoring for a variety of load categories. The developed model is
evaluated on the publicly available UK-DALE dataset with a moderately low
sampling frequency and various domestic loads.