pwc-archive/papers-with-abstracts huggingface dataset:
Policing Chronic and Temporary Hot Spots of Violent Crime: A Controlled Field Experiment
Authors: Dylan J. Fitzpatrick, Wilpen L. Gorr, Daniel B. Neill
Date: 2020-11-11 00:00:00

Hot-spot-based policing programs aim to deter crime through increased proactive patrols at high-crime locations. While most hot spot programs target easily identified chronic hot spots, we introduce models for predicting temporary hot spots to address effectiveness and equity objectives for crime prevention, and present findings from a crossover experiment evaluating application of hot spot predictions to prevent serious violent crime in Pittsburgh, PA. Over a 12-month experimental period, the Pittsburgh Bureau of Police assigned uniformed patrol officers to weekly predicted chronic and temporary hot spots of serious violent crimes comprising 0.5 percent of the city's area. We find statistically and practically significant reductions in serious violent crime counts within treatment hot spots as compared to control hot spots, with an overall reduction of 25.3 percent in the FBI-classified Part 1 Violent (P1V) crimes of homicide, rape, robbery, and aggravated assault, and a 39.7 percent reduction of African-American and other non-white victims of P1V crimes. We find that temporary hot spots increase spatial dispersion of patrols and have a greater percentage reduction in P1V crimes than chronic hot spots but fewer total number of crimes prevented. Only foot patrols, not car patrols, had statistically significant crime reductions in hot spots. We find no evidence of crime displacement; instead, we find weakly statistically significant spillover of crime prevention benefits to adjacent areas. In addition, we find no evidence that the community-oriented hot spot patrols produced over-policing arrests of minority or other populations.
GANMEX: One-vs-One Attributions Guided by GAN-based Counterfactual Explanation Baselines
Authors: Sheng-Min Shih, Pin-Ju Tien, Zohar Karnin
Date: 2020-11-11 00:00:00
Tasks: counterfactual, Counterfactual Explanation

Attribution methods have been shown as promising approaches for identifying key features that led to learned model predictions. While most existing attribution methods rely on a baseline input for performing feature perturbations, limited research has been conducted to address the baseline selection issues. Poor choices of baselines limit the ability of one-vs-one (1-vs-1) explanations for multi-class classifiers, which means the attribution methods were not able to explain why an input belongs to its original class but not the other specified target class. 1-vs-1 explanation is crucial when certain classes are more similar than others, e.g. two bird types among multiple animals, by focusing on key differentiating features rather than shared features across classes. In this paper, we present GAN-based Model EXplainability (GANMEX), a novel approach applying Generative Adversarial Networks (GAN) by incorporating the to-be-explained classifier as part of the adversarial networks. Our approach effectively selects the counterfactual baseline as the closest realistic sample belong to the target class, which allows attribution methods to provide true 1-vs-1 explanations. We showed that GANMEX baselines improved the saliency maps and led to stronger performance on perturbation-based evaluation metrics over the existing baselines. Existing attribution results are known for being insensitive to model randomization, and we demonstrated that GANMEX baselines led to better outcome under the cascading randomization of the model.
Towards NNGP-guided Neural Architecture Search
Authors: Daniel S. Park, Jaehoon Lee, Daiyi Peng, Yuan Cao, Jascha Sohl-Dickstein
Date: 2020-11-11 00:00:00
Tasks: Neural Architecture Search

The predictions of wide Bayesian neural networks are described by a Gaussian process, known as the Neural Network Gaussian Process (NNGP). Analytic forms for NNGP kernels are known for many models, but computing the exact kernel for convolutional architectures is prohibitively expensive. One can obtain effective approximations of these kernels through Monte-Carlo estimation using finite networks at initialization. Monte-Carlo NNGP inference is orders-of-magnitude cheaper in FLOPs compared to gradient descent training when the dataset size is small. Since NNGP inference provides a cheap measure of performance of a network architecture, we investigate its potential as a signal for neural architecture search (NAS). We compute the NNGP performance of approximately 423k networks in the NAS-bench 101 dataset on CIFAR-10 and compare its utility against conventional performance measures obtained by shortened gradient-based training. We carry out a similar analysis on 10k randomly sampled networks in the mobile neural architecture search (MNAS) space for ImageNet. We discover comparative advantages of NNGP-based metrics, and discuss potential applications. In particular, we propose that NNGP performance is an inexpensive signal independent of metrics obtained from training that can either be used for reducing big search spaces, or improving training-based performance measures.
Solving high-dimensional parameter inference: marginal posterior densities & Moment Networks
Authors: Niall Jeffrey, Benjamin D. Wandelt
Date: 2020-11-11 00:00:00
Tasks: Density Estimation, Time Series, Time Series Analysis, Vocal Bursts Intensity Prediction

High-dimensional probability density estimation for inference suffers from the "curse of dimensionality". For many physical inference problems, the full posterior distribution is unwieldy and seldom used in practice. Instead, we propose direct estimation of lower-dimensional marginal distributions, bypassing high-dimensional density estimation or high-dimensional Markov chain Monte Carlo (MCMC) sampling. By evaluating the two-dimensional marginal posteriors we can unveil the full-dimensional parameter covariance structure. We additionally propose constructing a simple hierarchy of fast neural regression models, called Moment Networks, that compute increasing moments of any desired lower-dimensional marginal posterior density; these reproduce exact results from analytic posteriors and those obtained from Masked Autoregressive Flows. We demonstrate marginal posterior density estimation using high-dimensional LIGO-like gravitational wave time series and describe applications for problems of fundamental cosmology.
Linear Dilation-Erosion Perceptron for Binary Classification
Authors: Angelica Lourenço Oliveira, Marcos Eduardo Valle
Date: 2020-11-11 00:00:00
Tasks: Binary Classification, Classification, General Classification

In this work, we briefly revise the reduced dilation-erosion perceptron (r-DEP) models for binary classification tasks. Then, we present the so-called linear dilation-erosion perceptron (l-DEP), in which a linear transformation is applied before the application of the morphological operators. Furthermore, we propose to train the l-DEP classifier by minimizing a regularized hinge-loss function subject to concave-convex restrictions. A simple example is given for illustrative purposes.
Maximum sampled conditional likelihood for informative subsampling
Authors: Haiying Wang, Jae Kwang Kim
Date: 2020-11-11 00:00:00

Subsampling is a computationally effective approach to extract information from massive data sets when computing resources are limited. After a subsample is taken from the full data, most available methods use an inverse probability weighted (IPW) objective function to estimate the model parameters. The IPW estimator does not fully utilize the information in the selected subsample. In this paper, we propose to use the maximum sampled conditional likelihood estimator (MSCLE) based on the sampled data. We established the asymptotic normality of the MSCLE and prove that its asymptotic variance covariance matrix is the smallest among a class of asymptotically unbiased estimators, including the IPW estimator. We further discuss the asymptotic results with the L-optimal subsampling probabilities and illustrate the estimation procedure with generalized linear models. Numerical experiments are provided to evaluate the practical performance of the proposed method.
Physics-constrained Deep Learning of Multi-zone Building Thermal Dynamics
Authors: Jan Drgona, Aaron R. Tuor, Vikas Chandan, Draguna L. Vrabie
Date: 2020-11-11 00:00:00
Tasks: Deep Learning

We present a physics-constrained control-oriented deep learning method for modeling building thermal dynamics. The proposed method is based on the systematic encoding of physics-based prior knowledge into a structured recurrent neural architecture. Specifically, our method incorporates structural priors from traditional physics-based building modeling into the neural network thermal dynamics model structure. Further, we leverage penalty methods to provide inequality constraints, thereby bounding predictions within physically realistic and safe operating ranges. Observing that stable eigenvalues accurately characterize the dissipativeness of the system, we additionally use a constrained matrix parameterization based on the Perron-Frobenius theorem to bound the dominant eigenvalues of the building thermal model parameter matrices. We demonstrate the proposed data-driven modeling approach's effectiveness and physical interpretability on a dataset obtained from a real-world office building with 20 thermal zones. Using only 10 days' measurements for training, we demonstrate generalization over 20 consecutive days, significantly improving the accuracy compared to prior state-of-the-art results reported in the literature.
Towards Semi-Supervised Semantics Understanding from Speech
Authors: Cheng-I Lai, Jin Cao, Sravan Bodapati, Shang-Wen Li
Date: 2020-11-11 00:00:00
Tasks: speech-recognition, Speech Recognition, Spoken Language Understanding

Much recent work on Spoken Language Understanding (SLU) falls short in at least one of three ways: models were trained on oracle text input and neglected the Automatics Speech Recognition (ASR) outputs, models were trained to predict only intents without the slot values, or models were trained on a large amount of in-house data. We proposed a clean and general framework to learn semantics directly from speech with semi-supervision from transcribed speech to address these. Our framework is built upon pretrained end-to-end (E2E) ASR and self-supervised language models, such as BERT, and fine-tuned on a limited amount of target SLU corpus. In parallel, we identified two inadequate settings under which SLU models have been tested: noise-robustness and E2E semantics evaluation. We tested the proposed framework under realistic environmental noises and with a new metric, the slots edit F1 score, on two public SLU corpora. Experiments show that our SLU framework with speech as input can perform on par with those with oracle text as input in semantics understanding, while environmental noises are present, and a limited amount of labeled semantics data is available.
Deep machine learning-assisted multiphoton microscopy to reduce light exposure and expedite imaging
Authors: Stephen Mcaleer, Alex Fast, Yuntian Xue, Magdalene Seiler, William Tang, Mihaela Balu, Pierre Baldi, Andrew W. Browne
Date: 2020-11-10 00:00:00
Tasks: BIG-bench Machine Learning, SSIM, Super-Resolution

Two-photon excitation fluorescence (2PEF) allows imaging of tissue up to about one millimeter in thickness. Typically, reducing fluorescence excitation exposure reduces the quality of the image. However, using deep learning super resolution techniques, these low-resolution images can be converted to high-resolution images. This work explores improving human tissue imaging by applying deep learning to maximize image quality while reducing fluorescence excitation exposure. We analyze two methods: a method based on U-Net, and a patch-based regression method. Both methods are evaluated on a skin dataset and an eye dataset. The eye dataset includes 1200 paired high power and low power images of retinal organoids. The skin dataset contains multiple frames of each sample of human skin. High-resolution images were formed by averaging 70 frames for each sample and low-resolution images were formed by averaging the first 7 and 15 frames for each sample. The skin dataset includes 550 images for each of the resolution levels. We track two measures of performance for the two methods: mean squared error (MSE) and structural similarity index measure (SSIM). For the eye dataset, the patches method achieves an average MSE of 27,611 compared to 146,855 for the U-Net method, and an average SSIM of 0.636 compared to 0.607 for the U-Net method. For the skin dataset, the patches method achieves an average MSE of 3.768 compared to 4.032 for the U-Net method, and an average SSIM of 0.824 compared to 0.783 for the U-Net method. Despite better performance on image quality, the patches method is worse than the U-Net method when comparing the speed of prediction, taking 303 seconds to predict one image compared to less than one second for the U-Net method.
Dirichlet Pruning for Neural Network Compression
Authors: Kamil Adamczewski, Mijung Park
Date: 2020-11-10 00:00:00
Tasks: Neural Network Compression, Variational Inference

We introduce Dirichlet pruning, a novel post-processing technique to transform a large neural network model into a compressed one. Dirichlet pruning is a form of structured pruning that assigns the Dirichlet distribution over each layer's channels in convolutional layers (or neurons in fully-connected layers) and estimates the parameters of the distribution over these units using variational inference. The learned distribution allows us to remove unimportant units, resulting in a compact architecture containing only crucial features for a task at hand. The number of newly introduced Dirichlet parameters is only linear in the number of channels, which allows for rapid training, requiring as little as one epoch to converge. We perform extensive experiments, in particular on larger architectures such as VGG and ResNet (45% and 58% compression rate, respectively) where our method achieves the state-of-the-art compression performance and provides interpretable features as a by-product.
End-to-end optimized image compression for machines, a study
Authors: Lahiru D. Chamain, Fabien Racapé, Jean Bégaint, Akshay Pushparaja, Simon Feltman
Date: 2020-11-10 00:00:00
Tasks: Decoder, Image Compression

An increasing share of image and video content is analyzed by machines rather than viewed by humans, and therefore it becomes relevant to optimize codecs for such applications where the analysis is performed remotely. Unfortunately, conventional coding tools are challenging to specialize for machine tasks as they were originally designed for human perception. However, neural network based codecs can be jointly trained end-to-end with any convolutional neural network (CNN)-based task model. In this paper, we propose to study an end-to-end framework enabling efficient image compression for remote machine task analysis, using a chain composed of a compression module and a task algorithm that can be optimized end-to-end. We show that it is possible to significantly improve the task accuracy when fine-tuning jointly the codec and the task networks, especially at low bit-rates. Depending on training or deployment constraints, selective fine-tuning can be applied only on the encoder, decoder or task network and still achieve rate-accuracy improvements over an off-the-shelf codec and task network. Our results also demonstrate the flexibility of end-to-end pipelines for practical applications.
Statistical learning for change point and anomaly detection in graphs
Authors: Anna Malinovskaya, Philipp Otto, Torben Peters
Date: 2020-11-10 00:00:00
Tasks: Anomaly Detection

Complex systems which can be represented in the form of static and dynamic graphs arise in different fields, e.g. communication, engineering and industry. One of the interesting problems in analysing dynamic network structures is to monitor changes in their development. Statistical learning, which encompasses both methods based on artificial intelligence and traditional statistics, can be used to progress in this research area. However, the majority of approaches apply only one or the other framework. In this paper, we discuss the possibility of bringing together both disciplines in order to create enhanced network monitoring procedures focussing on the example of combining statistical process control and deep learning algorithms. Together with the presentation of change point and anomaly detection in network data, we propose to monitor the response times of ambulance services, applying jointly the control chart for quantile function values and a graph convolutional network.
Forecasting Emergency Department Capacity Constraints for COVID Isolation Beds
Authors: Erik Drysdale, Devin Singh, Anna Goldenberg
Date: 2020-11-09 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Forecasting

Predicting patient volumes in a hospital setting is a well-studied application of time series forecasting. Existing tools usually make forecasts at the daily or weekly level to assist in planning for staffing requirements. Prompted by new COVID-related capacity constraints placed on our pediatric hospital's emergency department, we developed an hourly forecasting tool to make predictions over a 24 hour window. These forecasts would give our hospital sufficient time to be able to martial resources towards expanding capacity and augmenting staff (e.g. transforming wards or bringing in physicians on call). Using Gaussian Process Regressions (GPRs), we obtain strong performance for both point predictions (average R-squared: 82%) as well as classification accuracy when predicting the ordinal tiers of our hospital's capacity (average precision/recall: 82%/74%). Compared to traditional regression approaches, GPRs not only obtain consistently higher performance, but are also robust to the dataset shifts that have occurred throughout 2020. Hospital stakeholders are encouraged by the strength of our results, and we are currently working on moving our tool to a real-time setting with the goal of augmenting the capabilities of our healthcare workers.
FusedMM: A Unified SDDMM-SpMM Kernel for Graph Embedding and Graph Neural Networks
Authors: Md. Khaledur Rahman, Majedul Haque Sujon, Ariful Azad
Date: 2020-11-07 00:00:00
Tasks: Graph Embedding

We develop a fused matrix multiplication kernel that unifies sampled dense-dense matrix multiplication and sparse-dense matrix multiplication under a single operation called FusedMM. By using user-defined functions, FusedMM can capture almost all computational patterns needed by popular graph embedding and GNN approaches. FusedMM is an order of magnitude faster than its equivalent kernels in Deep Graph Library. The superior performance of FusedMM comes from the low-level vectorized kernels, a suitable load balancing scheme and an efficient utilization of the memory bandwidth. FusedMM can tune its performance using a code generator and perform equally well on Intel, AMD and ARM processors. FusedMM speeds up an end-to-end graph embedding algorithm by up to 28x on different processors.
R-TOD: Real-Time Object Detector with Minimized End-to-End Delay for Autonomous Driving
Authors: Wonseok Jang, Hansaem Jeong, Kyungtae Kang, Nikil Dutt, Jong-Chan Kim
Date: 2020-10-23 00:00:00
Tasks: Autonomous Driving, object-detection, Object Detection, Real-Time Object Detection

For realizing safe autonomous driving, the end-to-end delays of real-time object detection systems should be thoroughly analyzed and minimized. However, despite recent development of neural networks with minimized inference delays, surprisingly little attention has been paid to their end-to-end delays from an object's appearance until its detection is reported. With this motivation, this paper aims to provide more comprehensive understanding of the end-to-end delay, through which precise best- and worst-case delay predictions are formulated, and three optimization methods are implemented: (i) on-demand capture, (ii) zero-slack pipeline, and (iii) contention-free pipeline. Our experimental results show a 76% reduction in the end-to-end delay of Darknet YOLO (You Only Look Once) v3 (from 1070 ms to 261 ms), thereby demonstrating the great potential of exploiting the end-to-end delay analysis for autonomous driving. Furthermore, as we only modify the system architecture and do not change the neural network architecture itself, our approach incurs no penalty on the detection accuracy.
On the Performance of Convolutional Neural Networks under High and Low Frequency Information
Authors: Roshan Reddy Yedla, Shiv Ram Dubey
Date: 2020-10-30 00:00:00
Tasks: Data Augmentation, Face Recognition, Medical Image Analysis, Object Recognition

Convolutional neural networks (CNNs) have shown very promising performance in recent years for different problems, including object recognition, face recognition, medical image analysis, etc. However, generally the trained CNN models are tested over the test set which is very similar to the trained set. The generalizability and robustness of the CNN models are very important aspects to make it to work for the unseen data. In this letter, we study the performance of CNN models over the high and low frequency information of the images. We observe that the trained CNN fails to generalize over the high and low frequency images. In order to make the CNN robust against high and low frequency images, we propose the stochastic filtering based data augmentation during training. A satisfactory performance improvement has been observed in terms of the high and low frequency generalization and robustness with the proposed stochastic filtering based data augmentation approach. The experimentations are performed using ResNet50 model over the CIFAR-10 dataset and ResNet101 model over Tiny-ImageNet dataset.
Turn-level Dialog Evaluation with Dialog-level Weak Signals for Bot-Human Hybrid Customer Service Systems
Authors: Ruofeng Wen
Date: 2020-10-25 00:00:00
Tasks: Goal-Oriented Dialog

We developed a machine learning approach that quantifies multiple aspects of the success or values in Customer Service contacts, at anytime during the interaction. Specifically, the value/reward function regarding to the turn-level behaviors across human agents, chatbots and other hybrid dialog systems is characterized by the incremental information and confidence gain between sentences, based on the token-level predictions from a multi-task neural network trained with only weak signals in dialog-level attributes/states. The resulting model, named Value Profiler, serves as a goal-oriented dialog manager that enhances conversations by regulating automated decisions with its reward and state predictions. It supports both real-time monitoring and scalable offline customer experience evaluation, for both bot- and human-handled contacts. We show how it improves Amazon customer service quality in several applications.