pwc-archive/papers-with-abstracts huggingface dataset:
Sparse-View X-Ray CT Reconstruction Using $\ell_1$ Prior with Learned Transform
Authors: Xuehang Zheng, Il Yong Chun, Zhipeng Li, Yong Long, Jeffrey A. Fessler
Date: 2017-11-02 00:00:00
Tasks: Computed Tomography (CT), CT Reconstruction, Denoising, Image Reconstruction

A major challenge in X-ray computed tomography (CT) is reducing radiation dose while maintaining high quality of reconstructed images. To reduce the radiation dose, one can reduce the number of projection views (sparse-view CT); however, it becomes difficult to achieve high-quality image reconstruction as the number of projection views decreases. Researchers have applied the concept of learning sparse representations from (high-quality) CT image dataset to the sparse-view CT reconstruction. We propose a new statistical CT reconstruction model that combines penalized weighted-least squares (PWLS) and $\ell_1$ prior with learned sparsifying transform (PWLS-ST-$\ell_1$), and a corresponding efficient algorithm based on Alternating Direction Method of Multipliers (ADMM). To moderate the difficulty of tuning ADMM parameters, we propose a new ADMM parameter selection scheme based on approximated condition numbers. We interpret the proposed model by analyzing the minimum mean square error of its ($\ell_2$-norm relaxed) image update estimator. Our results with the extended cardiac-torso (XCAT) phantom data and clinical chest data show that, for sparse-view 2D fan-beam CT and 3D axial cone-beam CT, PWLS-ST-$\ell_1$ improves the quality of reconstructed images compared to the CT reconstruction methods using edge-preserving regularizer and $\ell_2$ prior with learned ST. These results also show that, for sparse-view 2D fan-beam CT, PWLS-ST-$\ell_1$ achieves comparable or better image quality and requires much shorter runtime than PWLS-DL using a learned overcomplete dictionary. Our results with clinical chest data show that, methods using the unsupervised learned prior generalize better than a state-of-the-art deep "denoising" neural network that does not use a physical imaging model.
Structured Generative Adversarial Networks
Authors: Zhijie Deng, Hao Zhang, Xiaodan Liang, Luona Yang, Shizhen Xu, Jun Zhu, Eric P. Xing
Date: 2017-11-02 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: image-classification, Image Classification, Semi-Supervised Image Classification, Style Transfer

We study the problem of conditional generative modeling based on designated
semantics or structures. Existing models that build conditional generators
either require massive labeled instances as supervision or are unable to
accurately control the semantics of generated samples. We propose structured
generative adversarial networks (SGANs) for semi-supervised conditional
generative modeling. SGAN assumes the data x is generated conditioned on two
independent latent variables: y that encodes the designated semantics, and z
that contains other factors of variation. To ensure disentangled semantics in y
and z, SGAN builds two collaborative games in the hidden space to minimize the
reconstruction error of y and z, respectively. Training SGAN also involves
solving two adversarial games that have their equilibrium concentrating at the
true joint data distributions p(x, z) and p(x, y), avoiding distributing the
probability mass diffusely over data space that MLE-based methods may suffer.
We assess SGAN by evaluating its trained networks, and its performance on
downstream tasks. We show that SGAN delivers a highly controllable generator,
and disentangled representations; it also establishes start-of-the-art results
across multiple datasets when applied for semi-supervised image classification
(1.27%, 5.73%, 17.26% error rates on MNIST, SVHN and CIFAR-10 using 50, 1000
and 4000 labels, respectively). Benefiting from the separate modeling of y and
z, SGAN can generate images with high visual quality and strictly following the
designated semantic, and can be extended to a wide spectrum of applications,
such as style transfer.
Shallow Updates for Deep Reinforcement Learning
Authors: Nir Levine, Tom Zahavy, Daniel J. Mankowitz, Aviv Tamar, Shie Mannor
Date: 2017-05-21 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Atari Games, Deep Reinforcement Learning, Feature Engineering, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep reinforcement learning (DRL) methods such as the Deep Q-Network (DQN)
have achieved state-of-the-art results in a variety of challenging,
high-dimensional domains. This success is mainly attributed to the power of
deep neural networks to learn rich domain representations for approximating the
value function or policy. Batch reinforcement learning methods with linear
representations, on the other hand, are more stable and require less hyper
parameter tuning. Yet, substantial feature engineering is necessary to achieve
good results. In this work we propose a hybrid approach -- the Least Squares
Deep Q-Network (LS-DQN), which combines rich feature representations learned by
a DRL algorithm with the stability of a linear least squares method. We do this
by periodically re-training the last hidden layer of a DRL network with a batch
least squares update. Key to our approach is a Bayesian regularization term for
the least squares update, which prevents over-fitting to the more recent data.
We tested LS-DQN on five Atari games and demonstrate significant improvement
over vanilla DQN and Double-DQN. We also investigated the reasons for the
superior performance of our method. Interestingly, we found that the
performance improvement can be attributed to the large batch size used by the
LS method when optimizing the last layer.
Set-to-Set Hashing with Applications in Visual Recognition
Authors: I-Hong Jhuo, Jun Wang
Date: 2017-11-02 00:00:00
Tasks: Retrieval

Visual data, such as an image or a sequence of video frames, is often naturally represented as a point set. In this paper, we consider the fundamental problem of finding a nearest set from a collection of sets, to a query set. This problem has obvious applications in large-scale visual retrieval and recognition, and also in applied fields beyond computer vision. One challenge stands out in solving the problem---set representation and measure of similarity. Particularly, the query set and the sets in dataset collection can have varying cardinalities. The training collection is large enough such that linear scan is impractical. We propose a simple representation scheme that encodes both statistical and structural information of the sets. The derived representations are integrated in a kernel framework for flexible similarity measurement. For the query set process, we adopt a learning-to-hash pipeline that turns the kernel representations into hash bits based on simple learners, using multiple kernel learning. Experiments on two visual retrieval datasets show unambiguously that our set-to-set hashing framework outperforms prior methods that do not take the set-to-set search setting.
Inhomogeneous Hypergraph Clustering with Applications
Authors: Pan Li, Olgica Milenkovic
Date: 2017-09-05 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Clustering, hypergraph partitioning

Hypergraph partitioning is an important problem in machine learning, computer
vision and network analytics. A widely used method for hypergraph partitioning
relies on minimizing a normalized sum of the costs of partitioning hyperedges
across clusters. Algorithmic solutions based on this approach assume that
different partitions of a hyperedge incur the same cost. However, this
assumption fails to leverage the fact that different subsets of vertices within
the same hyperedge may have different structural importance. We hence propose a
new hypergraph clustering technique, termed inhomogeneous hypergraph
partitioning, which assigns different costs to different hyperedge cuts. We
prove that inhomogeneous partitioning produces a quadratic approximation to the
optimal solution if the inhomogeneous costs satisfy submodularity constraints.
Moreover, we demonstrate that inhomogenous partitioning offers significant
performance improvements in applications such as structure learning of
rankings, subspace segmentation and motif clustering.
Correcting Nuisance Variation using Wasserstein Distance
Authors: Gil Tabak, Minjie Fan, Samuel J. Yang, Stephan Hoyer, Geoff Davis
Date: 2017-11-02 00:00:00
Proceeding: ICLR 2018 1

Profiling cellular phenotypes from microscopic imaging can provide meaningful biological information resulting from various factors affecting the cells. One motivating application is drug development: morphological cell features can be captured from images, from which similarities between different drug compounds applied at different doses can be quantified. The general approach is to find a function mapping the images to an embedding space of manageable dimensionality whose geometry captures relevant features of the input images. An important known issue for such methods is separating relevant biological signal from nuisance variation. For example, the embedding vectors tend to be more correlated for cells that were cultured and imaged during the same week than for those from different weeks, despite having identical drug compounds applied in both cases. In this case, the particular batch in which a set of experiments were conducted constitutes the domain of the data; an ideal set of image embeddings should contain only the relevant biological information (e.g. drug effects). We develop a general framework for adjusting the image embeddings in order to `forget' domain-specific information while preserving relevant biological information. To achieve this, we minimize a loss function based on distances between marginal distributions (such as the Wasserstein distance) of embeddings across domains for each replicated treatment. For the dataset we present results with, the only replicated treatment happens to be the negative control treatment, for which we do not expect any treatment-induced cell morphology changes. We find that for our transformed embeddings (i) the underlying geometric structure is not only preserved but the embeddings also carry improved biological signal; and (ii) less domain-specific information is present.
Associative content-addressable networks with exponentially many robust stable states
Authors: Rishidev Chaudhuri, Ila Fiete
Date: 2017-04-06 00:00:00

The brain must robustly store a large number of memories, corresponding to
the many events encountered over a lifetime. However, the number of memory
states in existing neural network models either grows weakly with network size
or recall fails catastrophically with vanishingly little noise. We construct an
associative content-addressable memory with exponentially many stable states
and robust error-correction. The network possesses expander graph connectivity
on a restricted Boltzmann machine architecture. The expansion property allows
simple neural network dynamics to perform at par with modern error-correcting
codes. Appropriate networks can be constructed with sparse random connections,
glomerular nodes, and associative learning using low dynamic-range weights.
Thus, sparse quasi-random structures---characteristic of important
error-correcting codes---may provide for high-performance computation in
artificial neural networks and the brain.
The (Un)reliability of saliency methods
Authors: Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T. Schütt, Sven Dähne, Dumitru Erhan, Been Kim
Date: 2017-11-02 00:00:00
Proceeding: ICLR 2018 1
Tasks: Attribute, Interpretable Machine Learning

Saliency methods aim to explain the predictions of deep neural networks. These methods lack reliability when the explanation is sensitive to factors that do not contribute to the model prediction. We use a simple and common pre-processing step ---adding a constant shift to the input data--- to show that a transformation with no effect on the model can cause numerous methods to incorrectly attribute. In order to guarantee reliability, we posit that methods should fulfill input invariance, the requirement that a saliency method mirror the sensitivity of the model with respect to transformations of the input. We show, through several examples, that saliency methods that do not satisfy input invariance result in misleading attribution.
Generalized Probabilistic Bisection for Stochastic Root-Finding
Authors: Sergio Rodriguez, Michael Ludkovski
Date: 2017-11-02 00:00:00
Tasks: Thompson Sampling

We consider numerical schemes for root finding of noisy responses through
generalizing the Probabilistic Bisection Algorithm (PBA) to the more practical
context where the sampling distribution is unknown and location-dependent. As
in standard PBA, we rely on a knowledge state for the approximate posterior of
the root location. To implement the corresponding Bayesian updating, we also
carry out inference of oracle accuracy, namely learning the probability of
correct response. To this end we utilize batched querying in combination with a
variety of frequentist and Bayesian estimators based on majority vote, as well
as the underlying functional responses, if available. For guiding sampling
selection we investigate both Information Directed sampling, as well as
Quantile sampling. Our numerical experiments show that these strategies perform
quite differently; in particular we demonstrate the efficiency of randomized
quantile sampling which is reminiscent of Thompson sampling. Our work is
motivated by the root-finding sub-routine in pricing of Bermudan financial
derivatives, illustrated in the last section of the paper.
Stochastic Gradient Descent in Continuous Time: A Central Limit Theorem
Authors: Justin Sirignano, Konstantinos Spiliopoulos
Date: 2017-10-11 00:00:00

Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance. The SGDCT algorithm follows a (noisy) descent direction along a continuous stream of data. The parameter updates occur in continuous time and satisfy a stochastic differential equation. This paper analyzes the asymptotic convergence rate of the SGDCT algorithm by proving a central limit theorem (CLT) for strongly convex objective functions and, under slightly stronger conditions, for non-convex objective functions as well. An $L^{p}$ convergence rate is also proven for the algorithm in the strongly convex case. The mathematical analysis lies at the intersection of stochastic analysis and statistical learning.
Channel masking for multivariate time series shapelets
Authors: Dripta S. Raychaudhuri, Josif Grabocka, Lars Schmidt-Thieme
Date: 2017-11-02 00:00:00
Tasks: General Classification, Time Series, Time Series Analysis, Time Series Classification

Time series shapelets are discriminative sub-sequences and their similarity
to time series can be used for time series classification. Initial shapelet
extraction algorithms searched shapelets by complete enumeration of all
possible data sub-sequences. Research on shapelets for univariate time series
proposed a mechanism called shapelet learning which parameterizes the shapelets
and learns them jointly with a prediction model in an optimization procedure.
Trivial extension of this method to multivariate time series does not yield
very good results due to the presence of noisy channels which lead to
overfitting. In this paper we propose a shapelet learning scheme for
multivariate time series in which we introduce channel masks to discount noisy
channels and serve as an implicit regularization.
Medical Image Segmentation Based on Multi-Modal Convolutional Neural Network: Study on Image Fusion Schemes
Authors: Zhe Guo, Xiang Li, Heng Huang, Ning Guo, Quanzheng Li
Date: 2017-10-31 00:00:00
Tasks: Computed Tomography (CT), Decision Making, Image Segmentation, Medical Image Segmentation, Semantic Segmentation

Image analysis using more than one modality (i.e. multi-modal) has been
increasingly applied in the field of biomedical imaging. One of the challenges
in performing the multimodal analysis is that there exist multiple schemes for
fusing the information from different modalities, where such schemes are
application-dependent and lack a unified framework to guide their designs. In
this work we firstly propose a conceptual architecture for the image fusion
schemes in supervised biomedical image analysis: fusing at the feature level,
fusing at the classifier level, and fusing at the decision-making level.
Further, motivated by the recent success in applying deep learning for natural
image analysis, we implement the three image fusion schemes above based on the
Convolutional Neural Network (CNN) with varied structures, and combined into a
single framework. The proposed image segmentation framework is capable of
analyzing the multi-modality images using different fusing schemes
simultaneously. The framework is applied to detect the presence of soft tissue
sarcoma from the combination of Magnetic Resonance Imaging (MRI), Computed
Tomography (CT) and Positron Emission Tomography (PET) images. It is found from
the results that while all the fusion schemes outperform the single-modality
schemes, fusing at the feature level can generally achieve the best performance
in terms of both accuracy and computational cost, but also suffers from the
decreased robustness in the presence of large errors in any image modalities.
Scalable Online Convolutional Sparse Coding
Authors: Yaqing Wang, Quanming Yao, James T. Kwok, Lionel M. Ni
Date: 2017-06-21 00:00:00

Convolutional sparse coding (CSC) improves sparse coding by learning a
shift-invariant dictionary from the data. However, existing CSC algorithms
operate in the batch mode and are expensive, in terms of both space and time,
on large datasets. In this paper, we alleviate these problems by using online
learning. The key is a reformulation of the CSC objective so that convolution
can be handled easily in the frequency domain and much smaller history matrices
are needed. We use the alternating direction method of multipliers (ADMM) to
solve the resulting optimization problem and the ADMM subproblems have
efficient closed-form solutions. Theoretical analysis shows that the learned
dictionary converges to a stationary point of the optimization problem.
Extensive experiments show that convergence of the proposed method is much
faster and its reconstruction performance is also better. Moreover, while
existing CSC algorithms can only run on a small number of images, the proposed
method can handle at least ten times more images.
A Comprehensive Low and High-level Feature Analysis for Early Rumor Detection on Twitter
Authors: Tu Nguyen
Date: 2017-11-02 00:00:00
Tasks: Misinformation

Recent work have done a good job in modeling rumors and detecting them over microblog streams. However, the performance of their automatic approaches are not relatively high when looking early in the diffusion. A first intuition is that, at early stage, most of the aggregated rumor features (e.g., propagation features) are not mature and distinctive enough. The objective of rumor debunking in microblogs, however, are to detect these misinformation as early as possible. In this work, we leverage neural models in learning the hidden representations of individual rumor-related tweets at the very beginning of a rumor. Our extensive experiments show that the resulting signal improves our classification performance over time, significantly within the first 10 hours. To deepen the understanding of these low and high-level features in contributing to the model performance over time, we conduct an extensive study on a wide range of high impact rumor features for the 48 hours range. The end model that engages these features are shown to be competitive, reaches over 90% accuracy and out-performs strong baselines in our carefully cured dataset.
Ocasta: Clustering Configuration Settings For Error Recovery
Authors: Zhen Huang, David Lie
Date: 2017-11-02 00:00:00
Tasks: Clustering

Effective machine-aided diagnosis and repair of configuration errors continues to elude computer systems designers. Most of the literature targets errors that can be attributed to a single erroneous configuration setting. However, a recent study found that a significant amount of configuration errors require fixing more than one setting together. To address this limitation, Ocasta statistically clusters dependent configuration settings based on the application's accesses to its configuration settings and utilizes the extracted clustering of configuration settings to fix configuration errors involving more than one configuration settings. Ocasta treats applications as black-boxes and only relies on the ability to observe application accesses to their configuration settings. We collected traces of real application usage from 24 Linux and 5 Windows desktops computers and found that Ocasta is able to correctly identify clusters with 88.6% accuracy. To demonstrate the effectiveness of Ocasta, we evaluated it on 16 real-world configuration errors of 11 Linux and Windows applications. Ocasta is able to successfully repair all evaluated configuration errors in 11 minutes on average and only requires the user to examine an average of 3 screenshots of the output of the application to confirm that the error is repaired. A user study we conducted shows that Ocasta is easy to use by both expert and non-expert users and is more efficient than manual configuration error troubleshooting.
Rotting Bandits
Authors: Nir Levine, Koby Crammer, Shie Mannor
Date: 2017-02-23 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Multi-Armed Bandits

The Multi-Armed Bandits (MAB) framework highlights the tension between
acquiring new knowledge (Exploration) and leveraging available knowledge
(Exploitation). In the classical MAB problem, a decision maker must choose an
arm at each time step, upon which she receives a reward. The decision maker's
objective is to maximize her cumulative expected reward over the time horizon.
The MAB problem has been studied extensively, specifically under the assumption
of the arms' rewards distributions being stationary, or quasi-stationary, over
time. We consider a variant of the MAB framework, which we termed Rotting
Bandits, where each arm's expected reward decays as a function of the number of
times it has been pulled. We are motivated by many real-world scenarios such as
online advertising, content recommendation, crowdsourcing, and more. We present
algorithms, accompanied by simulations, and derive theoretical guarantees.
Practical Bayesian Optimization for Model Fitting with Bayesian Adaptive Direct Search
Authors: Luigi Acerbi, Wei Ji Ma
Date: 2017-05-11 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Bayesian Optimization

Computational models in fields such as computational neuroscience are often
evaluated via stochastic simulation or numerical approximation. Fitting these
models implies a difficult optimization problem over complex, possibly noisy
parameter landscapes. Bayesian optimization (BO) has been successfully applied
to solving expensive black-box problems in engineering and machine learning.
Here we explore whether BO can be applied as a general tool for model fitting.
First, we present a novel hybrid BO algorithm, Bayesian adaptive direct search
(BADS), that achieves competitive performance with an affordable computational
overhead for the running time of typical models. We then perform an extensive
benchmark of BADS vs. many common and state-of-the-art nonconvex,
derivative-free optimizers, on a set of model-fitting problems with real data
and models from six studies in behavioral, cognitive, and computational
neuroscience. With default settings, BADS consistently finds comparable or
better solutions than other methods, including `vanilla' BO, showing great
promise for advanced BO techniques, and BADS in particular, as a general
model-fitting tool.