pwc-archive/papers-with-abstracts huggingface dataset:
Global-Local Metamodel Assisted Two-Stage Optimization via Simulation
Authors: Wei Xie, Yuan Yi, Hua Zheng
Date: 2019-10-13 00:00:00
Tasks: Decision Making, Vocal Bursts Valence Prediction

To integrate strategic, tactical and operational decisions, the two-stage optimization has been widely used to guide dynamic decision making. In this paper, we study the two-stage stochastic programming for complex systems with unknown response estimated by simulation. We introduce the global-local metamodel assisted two-stage optimization via simulation that can efficiently employ the simulation resource to iteratively solve for the optimal first- and second-stage decisions. Specifically, at each visited first-stage decision, we develop a local metamodel to simultaneously solve a set of scenario-based second-stage optimization problems, which also allows us to estimate the optimality gap. Then, we construct a global metamodel accounting for the errors induced by: (1) using a finite number of scenarios to approximate the expected future cost occurring in the planning horizon, (2) second-stage optimality gap, and (3) finite visited first-stage decisions. Assisted by the global-local metamodel, we propose a new simulation optimization approach that can efficiently and iteratively search for the optimal first- and second-stage decisions. Our framework can guarantee the convergence of optimal solution for the discrete two-stage optimization with unknown objective, and the empirical study indicates that it achieves substantial efficiency and accuracy.
Constrained Non-Affine Alignment of Embeddings
Authors: Yuwei Wang, Yan Zheng, Yanqing Peng, Chin-Chia Michael Yeh, Zhongfang Zhuang, Das Mahashweta, Bendre Mangesh, Feifei Li, Wei zhang, Jeff M. Phillips
Date: 2019-10-13 00:00:00

Embeddings are one of the fundamental building blocks for data analysis tasks. Embeddings are already essential tools for large language models and image analysis, and their use is being extended to many other research domains. The generation of these distributed representations is often a data- and computation-expensive process; yet the holistic analysis and adjustment of them after they have been created is still a developing area. In this paper, we first propose a very general quantitatively measure for the presence of features in the embedding data based on if it can be learned. We then devise a method to remove or alleviate undesired features in the embedding while retaining the essential structure of the data. We use a Domain Adversarial Network (DAN) to generate a non-affine transformation, but we add constraints to ensure the essential structure of the embedding is preserved. Our empirical results demonstrate that the proposed algorithm significantly outperforms the state-of-art unsupervised algorithm on several data sets, including novel applications from the industry.
Machine Learning for Prediction with Missing Dynamics
Authors: John Harlim, Shixiao W. Jiang, Senwei Liang, Haizhao Yang
Date: 2019-10-13 00:00:00
Tasks: BIG-bench Machine Learning, Prediction

This article presents a general framework for recovering missing dynamical systems using available data and machine learning techniques. The proposed framework reformulates the prediction problem as a supervised learning problem to approximate a map that takes the memories of the resolved and identifiable unresolved variables to the missing components in the resolved dynamics. We demonstrate the effectiveness of the proposed framework with a theoretical guarantee of a path-wise convergence of the resolved variables up to finite time and numerical tests on prototypical models in various scientific domains. These include the 57-mode barotropic stress models with multiscale interactions that mimic the blocked and unblocked patterns observed in the atmosphere, the nonlinear Schr\"{o}dinger equation which found many applications in physics such as optics and Bose-Einstein-Condense, the Kuramoto-Sivashinsky equation which spatiotemporal chaotic pattern formation models trapped ion mode in plasma and phase dynamics in reaction-diffusion systems. While many machine learning techniques can be used to validate the proposed framework, we found that recurrent neural networks outperform kernel regression methods in terms of recovering the trajectory of the resolved components and the equilibrium one-point and two-point statistics. This superb performance suggests that recurrent neural networks are an effective tool for recovering the missing dynamics that involves approximation of high-dimensional functions.
Accelerated Structured Alternating Projections for Robust Spectrally Sparse Signal Recovery
Authors: HanQin Cai, Jian-Feng Cai, Tianming Wang, Guojian Yin
Date: 2019-10-13 00:00:00
Tasks: Computational Efficiency

Consider a spectrally sparse signal $\boldsymbol{x}$ that consists of $r$ complex sinusoids with or without damping. We study the robust recovery problem for the spectrally sparse signal under the fully observed setting, which is about recovering $\boldsymbol{x}$ and a sparse corruption vector $\boldsymbol{s}$ from their sum $\boldsymbol{z}=\boldsymbol{x}+\boldsymbol{s}$. In this paper, we exploit the low-rank property of the Hankel matrix formed by $\boldsymbol{x}$, and formulate the problem as the robust recovery of a corrupted low-rank Hankel matrix. We develop a highly efficient non-convex algorithm, coined Accelerated Structured Alternating Projections (ASAP). The high computational efficiency and low space complexity of ASAP are achieved by fast computations involving structured matrices, and a subspace projection method for accelerated low-rank approximation. Theoretical recovery guarantee with a linear convergence rate has been established for ASAP, under some mild assumptions on $\boldsymbol{x}$ and $\boldsymbol{s}$. Empirical performance comparisons on both synthetic and real-world data confirm the advantages of ASAP, in terms of computational efficiency and robustness aspects.
Deep Kernels with Probabilistic Embeddings for Small-Data Learning
Authors: Ankur Mallick, Chaitanya Dwivedi, Bhavya Kailkhura, Gauri Joshi, T. Yong-Jin Han
Date: 2019-10-13 00:00:00
Tasks: Gaussian Processes, Representation Learning, Small Data Image Classification

Gaussian Processes (GPs) are known to provide accurate predictions and uncertainty estimates even with small amounts of labeled data by capturing similarity between data points through their kernel function. However traditional GP kernels are not very effective at capturing similarity between high dimensional data points. Neural networks can be used to learn good representations that encode intricate structures in high dimensional data, and can be used as inputs to the GP kernel. However the huge data requirement of neural networks makes this approach ineffective in small data settings. To solves the conflicting problems of representation learning and data efficiency, we propose to learn deep kernels on probabilistic embeddings by using a probabilistic neural network. Our approach maps high-dimensional data to a probability distribution in a low dimensional subspace and then computes a kernel between these distributions to capture similarity. To enable end-to-end learning, we derive a functional gradient descent procedure for training the model. Experiments on a variety of datasets show that our approach outperforms the state-of-the-art in GP kernel learning in both supervised and semi-supervised settings. We also extend our approach to other small-data paradigms such as few-shot classification where it outperforms previous approaches on mini-Imagenet and CUB datasets.
Improving the Sample and Communication Complexity for Decentralized Non-Convex Optimization: A Joint Gradient Estimation and Tracking Approach
Authors: Haoran Sun, Songtao Lu, Mingyi Hong
Date: 2019-10-13 00:00:00
Tasks: Stochastic Optimization

Many modern large-scale machine learning problems benefit from decentralized and stochastic optimization. Recent works have shown that utilizing both decentralized computing and local stochastic gradient estimates can outperform state-of-the-art centralized algorithms, in applications involving highly non-convex problems, such as training deep neural networks. In this work, we propose a decentralized stochastic algorithm to deal with certain smooth non-convex problems where there are $m$ nodes in the system, and each node has a large number of samples (denoted as $n$). Differently from the majority of the existing decentralized learning algorithms for either stochastic or finite-sum problems, our focus is given to both reducing the total communication rounds among the nodes, while accessing the minimum number of local data samples. In particular, we propose an algorithm named D-GET (decentralized gradient estimation and tracking), which jointly performs decentralized gradient estimation (which estimates the local gradient using a subset of local samples) and gradient tracking (which tracks the global full gradient using local estimates). We show that, to achieve certain $\epsilon$ stationary solution of the deterministic finite sum problem, the proposed algorithm achieves an $\mathcal{O}(mn^{1/2}\epsilon^{-1})$ sample complexity and an $\mathcal{O}(\epsilon^{-1})$ communication complexity. These bounds significantly improve upon the best existing bounds of $\mathcal{O}(mn\epsilon^{-1})$ and $\mathcal{O}(\epsilon^{-1})$, respectively. Similarly, for online problems, the proposed method achieves an $\mathcal{O}(m \epsilon^{-3/2})$ sample complexity and an $\mathcal{O}(\epsilon^{-1})$ communication complexity, while the best existing bounds are $\mathcal{O}(m\epsilon^{-2})$ and $\mathcal{O}(\epsilon^{-2})$, respectively.
Implicit competitive regularization in GANs
Authors: Florian Schäfer, Hongkai Zheng, Anima Anandkumar
Date: 2019-10-13 00:00:00
Proceeding: ICML 2020 1
Tasks: Image Generation

To improve the stability of GAN training we need to understand why they can produce realistic samples. Presently, this is attributed to properties of the divergence obtained under an optimal discriminator. This argument has a fundamental flaw: If we do not impose regularity of the discriminator, it can exploit visually imperceptible errors of the generator to always achieve the maximal generator loss. In practice, gradient penalties are used to regularize the discriminator. However, this needs a metric on the space of images that captures visual similarity. Such a metric is not known, which explains the limited success of gradient penalties in stabilizing GANs. We argue that the performance of GANs is instead due to the implicit competitive regularization (ICR) arising from the simultaneous optimization of generator and discriminator. ICR promotes solutions that look real to the discriminator and thus leverages its inductive biases to generate realistic images. We show that opponent-aware modelling of generator and discriminator, as present in competitive gradient descent (CGD), can significantly strengthen ICR and thus stabilize GAN training without explicit regularization. In our experiments, we use an existing implementation of WGAN-GP and show that by training it with CGD we can improve the inception score (IS) on CIFAR10 for a wide range of scenarios, without any hyperparameter tuning. The highest IS is obtained by combining CGD with the WGAN-loss, without any explicit regularization.
Nonstationary Multivariate Gaussian Processes for Electronic Health Records
Authors: Rui Meng, Braden Soper, Herbert Lee, Vincent X. Liu, John D. Greene, Priyadip Ray
Date: 2019-10-13 00:00:00
Tasks: Gaussian Processes

We propose multivariate nonstationary Gaussian processes for jointly modeling multiple clinical variables, where the key parameters, length-scales, standard deviations and the correlations between the observed output, are all time dependent. We perform posterior inference via Hamiltonian Monte Carlo (HMC). We also provide methods for obtaining computationally efficient gradient-based maximum a posteriori (MAP) estimates. We validate our model on synthetic data as well as on electronic health records (EHR) data from Kaiser Permanente (KP). We show that the proposed model provides better predictive performance over a stationary model as well as uncovers interesting latent correlation processes across vitals which are potentially predictive of patient risk.
Hierarchical Hidden Markov Jump Processes for Cancer Screening Modeling
Authors: Rui Meng, Soper Braden, Jan Nygard, Mari Nygrad, Herbert Lee
Date: 2019-10-13 00:00:00

Hidden Markov jump processes are an attractive approach for modeling clinical disease progression data because they are explainable and capable of handling both irregularly sampled and noisy data. Most applications in this context consider time-homogeneous models due to their relative computational simplicity. However, the time homogeneous assumption is too strong to accurately model the natural history of many diseases. Moreover, the population at risk is not homogeneous either, since disease exposure and susceptibility can vary considerably. In this paper, we propose a piece-wise stationary transition matrix to explain the heterogeneity in time. We propose a hierarchical structure for the heterogeneity in population, where prior information is considered to deal with unbalanced data. Moreover, an efficient, scalable EM algorithm is proposed for inference. We demonstrate the feasibility and superiority of our model on a cervical cancer screening dataset from the Cancer Registry of Norway. Experiments show that our model outperforms state-of-the-art recurrent neural network models in terms of prediction accuracy and significantly outperforms a standard hidden Markov jump process in generating Kaplan-Meier estimators.
Regularized Sparse Gaussian Processes
Authors: Rui Meng, Herbert Lee, Soper Braden, Priyadip Ray
Date: 2019-10-13 00:00:00
Tasks: Facial Expression Recognition (FER), Gaussian Processes, Image Reconstruction, Pose Estimation, Variational Inference

Gaussian processes are a flexible Bayesian nonparametric modelling approach that has been widely applied but poses computational challenges. To address the poor scaling of exact inference methods, approximation methods based on sparse Gaussian processes (SGP) are attractive. An issue faced by SGP, especially in latent variable models, is the inefficient learning of the inducing inputs, which leads to poor model prediction. We propose a regularization approach by balancing the reconstruction performance of data and the approximation performance of the model itself. This regularization improves both inference and prediction performance. We extend this regularization approach into latent variable models with SGPs and show that performing variational inference (VI) on those models is equivalent to performing VI on a related empirical Bayes model.
RGB-Infrared Cross-Modality Person Re-Identification via Joint Pixel and Feature Alignment
Authors: Guan'an Wang, Tianzhu Zhang, Jian Cheng, Si Liu, Yang Yang, Zeng-Guang Hou
Date: 2019-10-13 00:00:00
Proceeding: ICCV 2019 10
Tasks: Cross-Modality Person Re-identification, Generative Adversarial Network, Person Re-Identification, Representation Learning

RGB-Infrared (IR) person re-identification is an important and challenging task due to large cross-modality variations between RGB and IR images. Most conventional approaches aim to bridge the cross-modality gap with feature alignment by feature representation learning. Different from existing methods, in this paper, we propose a novel and end-to-end Alignment Generative Adversarial Network (AlignGAN) for the RGB-IR RE-ID task. The proposed model enjoys several merits. First, it can exploit pixel alignment and feature alignment jointly. To the best of our knowledge, this is the first work to model the two alignment strategies jointly for the RGB-IR RE-ID problem. Second, the proposed model consists of a pixel generator, a feature generator, and a joint discriminator. By playing a min-max game among the three components, our model is able to not only alleviate the cross-modality and intra-modality variations but also learn identity-consistent features. Extensive experimental results on two standard benchmarks demonstrate that the proposed model performs favorably against state-of-the-art methods. Especially, on SYSU-MM01 dataset, our model can achieve an absolute gain of 15.4% and 12.9% in terms of Rank-1 and mAP.
Modelling Resistive and Phase Change Memory with Passive Selector Arrays -- A Matlab Tool
Authors: Yasir J Noori, C H, de Groot
Date: 2019-10-13 00:00:00
Tasks: CPU

Memristor devices are crucial for developing neuromorphic computers and next-generation memory technologies. In this work, we provide a comprehensive modelling tool for simulating static DC reading operations of memristor crossbar arrays that use passive selectors with matrix algebra in MATLAB. The software tool was parallel coded and optimized to run with personal computers and distributed computer clusters with minimized CPU and memory consumption. Using the tool, we demonstrate the effect of changing the line resistance, array size, voltage selection scheme, selector diode's ideality factor, reverse saturation current, temperature and sense resistance on the electrical behavior and expected sense margin of one-diode-one-resistor crossbar arrays. We then investigate the effect of single and dual side array biasing and grounding on the dissipated current throughout the array cells. The tool we offer to the memristor community and the studies we present enables the design of larger and more practical memristor arrays for application in data storage and neuromorphic computing.
A multi-label, dual-output deep neural network for automated bug triaging
Authors: Christopher A. Choquette-Choo, David Sheldon, Jonny Proppe, John Alphonso-Gibbs, Harsha Gupta
Date: 2019-10-13 00:00:00
Tasks: Incremental Learning, Multi-class Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

Bug tracking enables the monitoring and resolution of issues and bugs within organizations. Bug triaging, or assigning bugs to the owner(s) who will resolve them, is a critical component of this process because there are many incorrect assignments that waste developer time and reduce bug resolution throughput. In this work, we explore the use of a novel two-output deep neural network architecture (Dual DNN) for triaging a bug to both an individual team and developer, simultaneously. Dual DNN leverages this simultaneous prediction by exploiting its own guess of the team classes to aid in developer assignment. A multi-label classification approach is used for each of the two outputs to learn from all interim owners, not just the last one who closed the bug. We make use of a heuristic combination of the interim owners (owner-importance-weighted labeling) which is converted into a probability mass function (pmf). We employ a two-stage learning scheme, whereby the team portion of the model is trained first and then held static to train the team--developer and bug--developer relationships. The scheme employed to encode the team--developer relationships is based on an organizational chart (org chart), which renders the model robust to organizational changes as it can adapt to role changes within an organization. There is an observed average lift (with respect to both team and developer assignment) of 13%-points in 11-fold incremental-learning cross-validation (IL-CV) accuracy for Dual DNN utilizing owner-weighted labels compared with the traditional multi-class classification approach. Furthermore, Dual DNN with owner-weighted labels achieves average 11-fold IL-CV accuracies of 76% (team assignment) and 55% (developer assignment), outperforming reference models by 14%- and 25%-points, respectively, on a proprietary dataset with 236,865 entries.
Generative Image Translation for Data Augmentation in Colorectal Histopathology Images
Authors: Jerry Wei, Arief Suriawinata, Louis Vaickus, Bing Ren, Xiaoying Liu, Jason Wei, Saeed Hassanpour
Date: 2019-10-13 00:00:00
Tasks: Data Augmentation, image-classification, Image Classification, Translation

We present an image translation approach to generate augmented data for mitigating data imbalances in a dataset of histopathology images of colorectal polyps, adenomatous tumors that can lead to colorectal cancer if left untreated. By applying cycle-consistent generative adversarial networks (CycleGANs) to a source domain of normal colonic mucosa images, we generate synthetic colorectal polyp images that belong to diagnostically less common polyp classes. Generated images maintain the general structure of their source image but exhibit adenomatous features that can be enhanced with our proposed filtration module, called Path-Rank-Filter. We evaluate the quality of generated images through Turing tests with four gastrointestinal pathologists, finding that at least two of the four pathologists could not identify generated images at a statistically significant level. Finally, we demonstrate that using CycleGAN-generated images to augment training data improves the AUC of a convolutional neural network for detecting sessile serrated adenomas by over 10%, suggesting that our approach might warrant further research for other histopathology image classification tasks.
Policy Poisoning in Batch Reinforcement Learning and Control
Authors: Yuzhe Ma, Xuezhou Zhang, Wen Sun, Xiaojin Zhu
Date: 2019-10-13 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We study a security threat to batch reinforcement learning and control where the attacker aims to poison the learned policy. The victim is a reinforcement learner / controller which first estimates the dynamics and the rewards from a batch data set, and then solves for the optimal policy with respect to the estimates. The attacker can modify the data set slightly before learning happens, and wants to force the learner into learning a target policy chosen by the attacker. We present a unified framework for solving batch policy poisoning attacks, and instantiate the attack on two standard victims: tabular certainty equivalence learner in reinforcement learning and linear quadratic regulator in control. We show that both instantiation result in a convex optimization problem on which global optimality is guaranteed, and provide analysis on attack feasibility and attack cost. Experiments show the effectiveness of policy poisoning attacks.
Discovering a sparse set of pairwise discriminating features in high dimensional data
Authors: Samuel Melton, Sharad Ramanathan
Date: 2019-10-13 00:00:00
Tasks: Clustering, General Classification

Extracting an understanding of the underlying system from high dimensional data is a growing problem in science. Discovering informative and meaningful features is crucial for clustering, classification, and low dimensional data embedding. Here we propose to construct features based on their ability to discriminate between clusters of the data points. We define a class of problems in which linear separability of clusters is hidden in a low dimensional space. We propose an unsupervised method to identify the subset of features that define a low dimensional subspace in which clustering can be conducted. This is achieved by averaging over discriminators trained on an ensemble of proposed cluster configurations. We then apply our method to single cell RNA-seq data from mouse gastrulation, and identify 27 key transcription factors (out of 409 total), 18 of which are known to define cell states through their expression levels. In this inferred subspace, we find clear signatures of known cell types that eluded classification prior to discovery of the correct low dimensional subspace.