pwc-archive/papers-with-abstracts huggingface dataset:
Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules
Authors: Niklas W. A. Gebauer, Michael Gastegger, Kristof T. Schütt
Date: 2019-06-02 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: 3D geometry

Deep learning has proven to yield fast and accurate predictions of quantum-chemical properties to accelerate the discovery of novel molecules and materials. As an exhaustive exploration of the vast chemical space is still infeasible, we require generative models that guide our search towards systems with desired properties. While graph-based models have previously been proposed, they are restricted by a lack of spatial information such that they are unable to recognize spatial isomerism and non-bonded interactions. Here, we introduce a generative neural network for 3d point sets that respects the rotational invariance of the targeted structures. We apply it to the generation of molecules and demonstrate its ability to approximate the distribution of equilibrium structures using spatial metrics as well as established measures from chemoinformatics. As our model is able to capture the complex relationship between 3d geometry and electronic properties, we bias the distribution of the generator towards molecules with a small HOMO-LUMO gap - an important property for the design of organic solar cells.
Sample-Optimal Low-Rank Approximation of Distance Matrices
Authors: Piotr Indyk, Ali Vakilian, Tal Wagner, David Woodruff
Date: 2019-06-02 00:00:00
Tasks: Handwriting Recognition

A distance matrix $A \in \mathbb R^{n \times m}$ represents all pairwise distances, $A_{ij}=\mathrm{d}(x_i,y_j)$, between two point sets $x_1,...,x_n$ and $y_1,...,y_m$ in an arbitrary metric space $(\mathcal Z, \mathrm{d})$. Such matrices arise in various computational contexts such as learning image manifolds, handwriting recognition, and multi-dimensional unfolding. In this work we study algorithms for low-rank approximation of distance matrices. Recent work by Bakshi and Woodruff (NeurIPS 2018) showed it is possible to compute a rank-$k$ approximation of a distance matrix in time $O((n+m)^{1+\gamma}) \cdot \mathrm{poly}(k,1/\epsilon)$, where $\epsilon>0$ is an error parameter and $\gamma>0$ is an arbitrarily small constant. Notably, their bound is sublinear in the matrix size, which is unachievable for general matrices. We present an algorithm that is both simpler and more efficient. It reads only $O((n+m) k/\epsilon)$ entries of the input matrix, and has a running time of $O(n+m) \cdot \mathrm{poly}(k,1/\epsilon)$. We complement the sample complexity of our algorithm with a matching lower bound on the number of entries that must be read by any algorithm. We provide experimental results to validate the approximation quality and running time of our algorithm.
On the Efficiency of Entropic Regularized Algorithms for Optimal Transport
Authors: Tianyi Lin, Nhat Ho, Michael. I. Jordan
Date: 2019-06-01 00:00:00

We present several new complexity results for the entropic regularized algorithms that approximately solve the optimal transport (OT) problem between two discrete probability measures with at most $n$ atoms. First, we improve the complexity bound of a greedy variant of Sinkhorn, known as \textit{Greenkhorn}, from $\widetilde{O}(n^2\varepsilon^{-3})$ to $\widetilde{O}(n^2\varepsilon^{-2})$. Notably, our result can match the best known complexity bound of Sinkhorn and help clarify why Greenkhorn significantly outperforms Sinkhorn in practice in terms of row/column updates as observed by~\citet{Altschuler-2017-Near}. Second, we propose a new algorithm, which we refer to as \textit{APDAMD} and which generalizes an adaptive primal-dual accelerated gradient descent (APDAGD) algorithm~\citep{Dvurechensky-2018-Computational} with a prespecified mirror mapping $\phi$. We prove that APDAMD achieves the complexity bound of $\widetilde{O}(n^2\sqrt{\delta}\varepsilon^{-1})$ in which $\delta>0$ stands for the regularity of $\phi$. In addition, we show by a counterexample that the complexity bound of $\widetilde{O}(\min\{n^{9/4}\varepsilon^{-1}, n^2\varepsilon^{-2}\})$ proved for APDAGD before is invalid and give a refined complexity bound of $\widetilde{O}(n^{5/2}\varepsilon^{-1})$. Further, we develop a \textit{deterministic} accelerated variant of Sinkhorn via appeal to estimated sequence and prove the complexity bound of $\widetilde{O}(n^{7/3}\varepsilon^{-4/3})$. As such, we see that accelerated variant of Sinkhorn outperforms Sinkhorn and Greenkhorn in terms of $1/\varepsilon$ and APDAGD and accelerated alternating minimization (AAM)~\citep{Guminov-2021-Combination} in terms of $n$. Finally, we conduct the experiments on synthetic and real data and the numerical results show the efficiency of Greenkhorn, APDAMD and accelerated Sinkhorn in practice.
SHE: A Fast and Accurate Deep Neural Network for Encrypted Data
Authors: Qian Lou, Lei Jiang
Date: 2019-06-01 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Quantization

Homomorphic Encryption (HE) is one of the most promising security solutions to emerging Machine Learning as a Service (MLaaS). Leveled-HE (LHE)-enabled Convolutional Neural Networks (LHECNNs) are proposed to implement MLaaS to avoid large bootstrapping overhead. However, prior LHECNNs have to pay significant computing overhead but achieve only low inference accuracy, due to their polynomial approximation activations and poolings. Stacking many polynomial approximation activation layers in a network greatly reduces inference accuracy, since the polynomial approximation activation errors lead to a low distortion of the output distribution of the next batch normalization layer. So the polynomial approximation activations and poolings have become the obstacle to a fast and accurate LHECNN model. In this paper, we propose a Shift-accumulation-based LHE-enabled deep neural network (SHE) for fast and accurate inferences on encrypted data. We use the binary-operation-friendly Leveled Fast Homomorphic Encryption over Torus (LTFHE) encryption scheme to implement ReLU activations and max poolings. We also adopt the logarithmic quantization to accelerate inferences by replacing expensive LTFHE multiplications with cheap LTFHE shifts. We propose a mixed bitwidth accumulator to accelerate accumulations. Since the LTFHE ReLU activations, max poolings, shifts and accumulations have small multiplicative depth overhead, SHE can implement much deeper network architectures with more convolutional and activation layers. Our experimental results show SHE achieves the state-of-the-art inference accuracy and reduces the inference latency by 76.21% ~ 94.23% over prior LHECNNs on MNIST and CIFAR-10. The source code of SHE is available at https://github.com/qianlou/SHE.
Incorporating System-Level Objectives into Recommender Systems
Authors: Himan Abdollahpouri
Date: 2019-05-31 00:00:00
Tasks: Recommendation Systems

One of the most essential parts of any recommender system is personalization-- how acceptable the recommendations are from the user's perspective. However, in many real-world applications, there are other stakeholders whose needs and interests should be taken into account. In this work, we define the problem of multistakeholder recommendation and we focus on finding algorithms for a special case where the recommender system itself is also a stakeholder. In addition, we will explore the idea of incremental incorporation of system-level objectives into recommender systems over time to tackle the existing problems in the optimization techniques which only look for optimizing the individual users' lists.
Testing that a Local Optimum of the Likelihood is Globally Optimum using Reparameterized Embeddings
Authors: Joel W. LeBlanc, Brian J. Thelen, Alfred O. Hero
Date: 2019-05-31 00:00:00
Tasks: global-optimization

Many mathematical imaging problems are posed as non-convex optimization problems. When numerically tractable global optimization procedures are not available, one is often interested in testing ex post facto whether or not a locally convergent algorithm has found the globally optimal solution. When the problem is formulated in terms of maximizing the likelihood function under a statistical model for the measurements, one can construct a statistical test that a local maximum is in fact the global maximum. A one-sided test is proposed for the case that the statistical model is a member of the generalized location family of probability distributions, a condition often satisfied in imaging and other inverse problems. We propose a general method for improving the accuracy of the test by reparameterizing the likelihood function to embed its domain into a higher dimensional parameter space. We show that the proposed global maximum testing method results in improved accuracy and reduced computation for a physically-motivated joint-inverse problem arising in camera-blur estimation.
Knowledge-augmented Column Networks: Guiding Deep Learning with Advice
Authors: Mayukh Das, Devendra Singh Dhami, Yang Yu, Gautam Kunapuli, Sriraam Natarajan
Date: 2019-05-31 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning

Recently, deep models have had considerable success in several tasks, especially with low-level representations. However, effective learning from sparse noisy samples is a major challenge in most deep models, especially in domains with structured representations. Inspired by the proven success of human guided machine learning, we propose Knowledge-augmented Column Networks, a relational deep learning framework that leverages human advice/knowledge to learn better models in presence of sparsity and systematic noise.
Multimodal Ensemble Approach to Incorporate Various Types of Clinical Notes for Predicting Readmission
Authors: Bonggun Shin, Julien Hogan, Andrew B. Adams, Raymond J. Lynch, Rachel E. Patzer, Jinho D. Choi
Date: 2019-05-31 00:00:00

Electronic Health Records (EHRs) have been heavily used to predict various downstream clinical tasks such as readmission or mortality. One of the modalities in EHRs, clinical notes, has not been fully explored for these tasks due to its unstructured and inexplicable nature. Although recent advances in deep learning (DL) enables models to extract interpretable features from unstructured data, they often require a large amount of training data. However, many tasks in medical domains inherently consist of small sample data with lengthy documents; for a kidney transplant as an example, data from only a few thousand of patients are available and each patient's document consists of a couple of millions of words in major hospitals. Thus, complex DL methods cannot be applied to these kinds of domains. In this paper, we present a comprehensive ensemble model using vector space modeling and topic modeling. Our proposed model is evaluated on the readmission task of kidney transplant patients and improves 0.0211 in terms of c-statistics from the previous state-of-the-art approach using structured data, while typical DL methods fail to beat this approach. The proposed architecture provides the interpretable score for each feature from both modalities, structured and unstructured data, which is shown to be meaningful through a physician's evaluation.
Regularizing Black-box Models for Improved Interpretability (HILL 2019 Version)
Authors: Gregory Plumb, Maruan Al-Shedivat, Eric Xing, Ameet Talwalkar
Date: 2019-05-31 00:00:00
Tasks: BIG-bench Machine Learning, Interpretable Machine Learning

Most of the work on interpretable machine learning has focused on designing either inherently interpretable models, which typically trade-off accuracy for interpretability, or post-hoc explanation systems, which lack guarantees about their explanation quality. We propose an alternative to these approaches by directly regularizing a black-box model for interpretability at training time. Our approach explicitly connects three key aspects of interpretable machine learning: (i) the model's innate explainability, (ii) the explanation system used at test time, and (iii) the metrics that measure explanation quality. Our regularization results in substantial improvement in terms of the explanation fidelity and stability metrics across a range of datasets and black-box explanation systems while slightly improving accuracy. Further, if the resulting model is still not sufficiently interpretable, the weight of the regularization term can be adjusted to achieve the desired trade-off between accuracy and interpretability. Finally, we justify theoretically that the benefits of explanation-based regularization generalize to unseen points.
Representation Theoretic Patterns in Multi-Frequency Class Averaging for Three-Dimensional Cryo-Electron Microscopy
Authors: Yifeng Fan, Tingran Gao, Zhizhen Zhao
Date: 2019-05-31 00:00:00
Tasks: Clustering

We develop in this paper a novel intrinsic classification algorithm -- multi-frequency class averaging (MFCA) -- for classifying noisy projection images obtained from three-dimensional cryo-electron microscopy (cryo-EM) by the similarity among their viewing directions. This new algorithm leverages multiple irreducible representations of the unitary group to introduce additional redundancy into the representation of the optimal in-plane rotational alignment, extending and outperforming the existing class averaging algorithm that uses only a single representation. The formal algebraic model and representation theoretic patterns of the proposed MFCA algorithm extend the framework of Hadani and Singer to arbitrary irreducible representations of the unitary group. We conceptually establish the consistency and stability of MFCA by inspecting the spectral properties of a generalized local parallel transport operator through the lens of Wigner $D$-matrices. We demonstrate the efficacy of the proposed algorithm with numerical experiments.
Increasing Compactness Of Deep Learning Based Speech Enhancement Models With Parameter Pruning And Quantization Techniques
Authors: Jyun-Yi Wu, Cheng Yu, Szu-Wei Fu, Chih-Ting Liu, Shao-Yi Chien, Yu Tsao
Date: 2019-05-31 00:00:00
Tasks: Denoising, Quantization, Speech Enhancement

Most recent studies on deep learning based speech enhancement (SE) focused on improving denoising performance. However, successful SE applications require striking a desirable balance between denoising performance and computational cost in real scenarios. In this study, we propose a novel parameter pruning (PP) technique, which removes redundant channels in a neural network. In addition, a parameter quantization (PQ) technique was applied to reduce the size of a neural network by representing weights with fewer cluster centroids. Because the techniques are derived based on different concepts, the PP and PQ can be integrated to provide even more compact SE models. The experimental results show that the PP and PQ techniques produce a compacted SE model with a size of only 10.03% compared to that of the original model, resulting in minor performance losses of 1.43% (from 0.70 to 0.69) for STOI and 3.24% (from 1.85 to 1.79) for PESQ. The promising results suggest that the PP and PQ techniques can be used in a SE system in devices with limited storage and computation resources.
EnLLVM: Ensemble Based Nonlinear Bayesian Filtering Using Linear Latent Variable Models
Authors: Xiao Lin, Gabriel Terejanu
Date: 2017-08-08 00:00:00

Real-time nonlinear Bayesian filtering algorithms are overwhelmed by data volume, velocity and increasing complexity of computational models. In this paper, we propose a novel ensemble based nonlinear Bayesian filtering approach which only requires a small number of simulations and can be applied to high-dimensional systems in the presence of intractable likelihood functions. The proposed approach uses linear latent projections to estimate the joint probability distribution between states, parameters, and observables using a mixture of Gaussian components generated by the reconstruction error for each ensemble member. Since it leverages the computational machinery behind linear latent variable models, it can achieve fast implementations without the need to compute high-dimensional sample covariance matrices. The performance of the proposed approach is compared with the performance of ensemble Kalman filter on a high-dimensional Lorenz nonlinear dynamical system.
Neural Review Rating Prediction with Hierarchical Attentions and Latent Factors
Authors: Xianchen Wang, Hongtao Liu, Peiyi Wang, Fangzhao Wu, Hongyan Xu, Wenjun Wang, Xing Xie
Date: 2019-05-29 00:00:00
Tasks: Informativeness

Text reviews can provide rich useful semantic information for modeling users and items, which can benefit rating prediction in recommendation. Different words and reviews may have different informativeness for users or items. Besides, different users and items should be personalized. Most existing works regard all reviews equally or utilize a general attention mechanism. In this paper, we propose a hierarchical attention model fusing latent factor model for rating prediction with reviews, which can focus on important words and informative reviews. Specially, we use the factor vectors of Latent Factor Model to guide the attention network and combine the factor vectors with feature representation learned from reviews to predict the final ratings. Experiments on real-world datasets validate the effectiveness of our approach.
RL4health: Crowdsourcing Reinforcement Learning for Knee Replacement Pathway Optimization
Authors: Hao Lu, Mengdi Wang
Date: 2019-05-24 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Joint replacement is the most common inpatient surgical treatment in the US. We investigate the clinical pathway optimization for knee replacement, which is a sequential decision process from onset to recovery. Based on episodic claims from previous cases, we view the pathway optimization as an intelligence crowdsourcing problem and learn the optimal decision policy from data by imitating the best expert at every intermediate state. We develop a reinforcement learning-based pipeline that uses value iteration, state compression and aggregation learning, kernel representation and cross validation to predict the best treatment policy. It also provides forecast of the clinical pathway under the optimized policy. Empirical validation shows that the optimized policy reduces the overall cost by 7 percent and reduces the excessive cost premium by 33 percent.
Hypothesis-Driven Skill Discovery for Hierarchical Deep Reinforcement Learning
Authors: Caleb Chuck, Supawit Chockchowwat, Scott Niekum
Date: 2019-05-27 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep reinforcement learning (DRL) is capable of learning high-performing policies on a variety of complex high-dimensional tasks, ranging from video games to robotic manipulation. However, standard DRL methods often suffer from poor sample efficiency, partially because they aim to be entirely problem-agnostic. In this work, we introduce a novel approach to exploration and hierarchical skill learning that derives its sample efficiency from intuitive assumptions it makes about the behavior of objects both in the physical world and simulations which mimic physics. Specifically, we propose the Hypothesis Proposal and Evaluation (HyPE) algorithm, which discovers objects from raw pixel data, generates hypotheses about the controllability of observed changes in object state, and learns a hierarchy of skills to test these hypotheses. We demonstrate that HyPE can dramatically improve the sample efficiency of policy learning in two different domains: a simulated robotic block-pushing domain, and a popular benchmark task: Breakout. In these domains, HyPE learns high-scoring policies an order of magnitude faster than several state-of-the-art reinforcement learning methods.
Accelerating Physics-Based Simulations Using Neural Network Proxies: An Application in Oil Reservoir Modeling
Authors: Jiri Navratil, Alan King, Jesus Rios, Georgios Kollias, Ruben Torrado, Andres Codas
Date: 2019-05-23 00:00:00

We develop a proxy model based on deep learning methods to accelerate the simulations of oil reservoirs--by three orders of magnitude--compared to industry-strength physics-based PDE solvers. This paper describes a new architectural approach to this task, accompanied by a thorough experimental evaluation on a publicly available reservoir model. We demonstrate that in a practical setting a speedup of more than 2000X can be achieved with an average sequence error of about 10\% relative to the oil-field simulator. The proxy model is contrasted with a high-quality physics-based acceleration baseline and is shown to outperform it by several orders of magnitude. We believe the outcomes presented here are extremely promising and offer a valuable benchmark for continuing research in oil field development optimization. Due to its domain-agnostic architecture, the presented approach can be extended to many applications beyond the field of oil and gas exploration.
Approximate Computational Approaches for Bayesian Sensor Placement in High Dimensions
Authors: Xiao Lin, Asif Chowdhury, Xiaofan Wang, Gabriel Terejanu
Date: 2017-03-01 00:00:00
Tasks: Bayesian Optimization, Vocal Bursts Intensity Prediction

Since the cost of installing and maintaining sensors is usually high, sensor
locations are always strategically selected. For those aiming at inferring
certain quantities of interest (QoI), it is desirable to explore the dependency
between sensor measurements and QoI. One of the most popular metric for the
dependency is mutual information which naturally measures how much information
about one variable can be obtained given the other. However, computing mutual
information is always challenging, and the result is unreliable in high
dimension. In this paper, we propose an approach to find an approximate lower
bound of mutual information and compute it in a lower dimension. Then, sensors
are placed where highest mutual information (lower bound) is achieved and QoI
is inferred via Bayes rule given sensor measurements. In addition, Bayesian
optimization is introduced to provide a continuous mutual information surface
over the domain and thus reduce the number of evaluations. A chemical release
accident is simulated where multiple sensors are placed to locate the source of
the release. The result shows that the proposed approach is both effective and
efficient in inferring QoI.
Online Learning for Measuring Incentive Compatibility in Ad Auctions
Authors: Zhe Feng, Okke Schrijvers, Eric Sodomka
Date: 2019-01-21 00:00:00

In this paper we investigate the problem of measuring end-to-end Incentive
Compatibility (IC) regret given black-box access to an auction mechanism. Our
goal is to 1) compute an estimate for IC regret in an auction, 2) provide a
measure of certainty around the estimate of IC regret, and 3) minimize the time
it takes to arrive at an accurate estimate. We consider two main problems, with
different informational assumptions: In the \emph{advertiser problem} the goal
is to measure IC regret for some known valuation $v$, while in the more general
\emph{demand-side platform (DSP) problem} we wish to determine the worst-case
IC regret over all possible valuations. The problems are naturally phrased in
an online learning model and we design $Regret-UCB$ algorithms for both
problems. We give an online learning algorithm where for the advertiser problem
the error of determining IC shrinks as $O\Big(\frac{|B|}{T}\cdot\Big(\frac{\ln
T}{n} + \sqrt{\frac{\ln T}{n}}\Big)\Big)$ (where $B$ is the finite set of bids,
$T$ is the number of time steps, and $n$ is number of auctions per time step),
and for the DSP problem it shrinks as $O\Big(\frac{|B|}{T}\cdot\Big(
\frac{|B|\ln T}{n} + \sqrt{\frac{|B|\ln T}{n}}\Big)\Big)$. For the DSP problem,
we also consider stronger IC regret estimation and extend our $Regret-UCB$
algorithm to achieve better IC regret error. We validate the theoretical
results using simulations with Generalized Second Price (GSP) auctions, which
are known to not be incentive compatible and thus have strictly positive IC
regret.