pwc-archive/papers-with-abstracts huggingface dataset:
Limit Theorems for Network Dependent Random Variables
Authors: Denis Kojevnikov, Vadim Marmer, Kyungchul Song
Date: 2019-03-04 00:00:00

This paper is concerned with cross-sectional dependence arising because observations are interconnected through an observed network. Following Doukhan and Louhichi (1999), we measure the strength of dependence by covariances of nonlinearly transformed variables. We provide a law of large numbers and central limit theorem for network dependent variables. We also provide a method of calculating standard errors robust to general forms of network dependence. For that purpose, we rely on a network heteroskedasticity and autocorrelation consistent (HAC) variance estimator, and show its consistency. The results rely on conditions characterized by tradeoffs between the rate of decay of dependence across a network and network's denseness. Our approach can accommodate data generated by network formation models, random fields on graphs, conditional dependency graphs, and large functional-causal systems of equations.
Beyond Convolutions: A Novel Deep Learning Approach for Raw Seismic Data Ingestion
Authors: Zhaozhuo Xu, Aditya Desai, Menal Gupta, Anu Chandran, Antoine Vial-Aussavy, Anshumali Shrivastava
Date: 2021-02-26 00:00:00
Tasks: Irregular Time Series, SSIM, Time Series Analysis

Traditional seismic processing workflows (SPW) are expensive, requiring over a year of human and computational effort. Deep learning (DL) based data-driven seismic workflows (DSPW) hold the potential to reduce these timelines to a few minutes. Raw seismic data (terabytes) and required subsurface prediction (gigabytes) are enormous. This large-scale, spatially irregular time-series data poses seismic data ingestion (SDI) as an unconventional yet fundamental problem in DSPW. Current DL research is limited to small-scale simplified synthetic datasets as they treat seismic data like images and process them with convolution networks. Real seismic data, however, is at least 5D. Applying 5D convolutions to this scale is computationally prohibitive. Moreover, raw seismic data is highly unstructured and hence inherently non-image like. We propose a fundamental shift to move away from convolutions and introduce SESDI: Set Embedding based SDI approach. SESDI first breaks down the mammoth task of large-scale prediction into an efficient compact auxiliary task. SESDI gracefully incorporates irregularities in data with its novel model architecture. We believe SESDI is the first successful demonstration of end-to-end learning on real seismic data. SESDI achieves SSIM of over 0.8 on velocity inversion task on real proprietary data from the Gulf of Mexico and outperforms the state-of-the-art U-Net model on synthetic datasets.
Learning Prediction Intervals for Regression: Generalization and Calibration
Authors: Haoxian Chen, Ziyi Huang, Henry Lam, Huajie Qian, Haofeng Zhang
Date: 2021-02-26 00:00:00
Tasks: Learning Theory, Prediction, Prediction Intervals, regression, Uncertainty Quantification

We study the generation of prediction intervals in regression for uncertainty quantification. This task can be formalized as an empirical constrained optimization problem that minimizes the average interval width while maintaining the coverage accuracy across data. We strengthen the existing literature by studying two aspects of this empirical optimization. First is a general learning theory to characterize the optimality-feasibility tradeoff that encompasses Lipschitz continuity and VC-subgraph classes, which are exemplified in regression trees and neural networks. Second is a calibration machinery and the corresponding statistical theory to optimally select the regularization parameter that manages this tradeoff, which bypasses the overfitting issues in previous approaches in coverage attainment. We empirically demonstrate the strengths of our interval generation and calibration algorithms in terms of testing performances compared to existing benchmarks.
What Doesn't Kill You Makes You Robust(er): How to Adversarially Train against Data Poisoning
Authors: Jonas Geiping, Liam Fowl, Gowthami Somepalli, Micah Goldblum, Michael Moeller, Tom Goldstein
Date: 2021-02-26 00:00:00
Tasks: Data Poisoning

Data poisoning is a threat model in which a malicious actor tampers with training data to manipulate outcomes at inference time. A variety of defenses against this threat model have been proposed, but each suffers from at least one of the following flaws: they are easily overcome by adaptive attacks, they severely reduce testing performance, or they cannot generalize to diverse data poisoning threat models. Adversarial training, and its variants, are currently considered the only empirically strong defense against (inference-time) adversarial attacks. In this work, we extend the adversarial training framework to defend against (training-time) data poisoning, including targeted and backdoor attacks. Our method desensitizes networks to the effects of such attacks by creating poisons during training and injecting them into training batches. We show that this defense withstands adaptive attacks, generalizes to diverse threat models, and incurs a better performance trade-off than previous defenses such as DP-SGD or (evasion) adversarial training.
A Meta-embedding-based Ensemble Approach for ICD Coding Prediction
Authors: Pavithra Rajendran, Alexandros Zenonos, Josh Spear, Rebecca Pope
Date: 2021-02-26 00:00:00
Tasks: Articles, Diagnostic, General Classification, Retrieval

International Classification of Diseases (ICD) are the de facto codes used globally for clinical coding. These codes enable healthcare providers to claim reimbursement and facilitate efficient storage and retrieval of diagnostic information. The problem of automatically assigning ICD codes has been approached in literature as a multilabel classification, using neural models on unstructured data. Our proposed approach enhances the performance of neural models by effectively training word vectors using routine medical data as well as external knowledge from scientific articles. Furthermore, we exploit the geometric properties of the two sets of word vectors and combine them into a common dimensional space, using meta-embedding techniques. We demonstrate the efficacy of this approach for a multimodal setting, using unstructured and structured information. We empirically show that our approach improves the current state-of-the-art deep learning architectures and benefits ensemble models.
Towards Robust and Reliable Algorithmic Recourse
Authors: Sohini Upadhyay, Shalmali Joshi, Himabindu Lakkaraju
Date: 2021-02-26 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Decision Making

As predictive models are increasingly being deployed in high-stakes decision making (e.g., loan approvals), there has been growing interest in post hoc techniques which provide recourse to affected individuals. These techniques generate recourses under the assumption that the underlying predictive model does not change. However, in practice, models are often regularly updated for a variety of reasons (e.g., dataset shifts), thereby rendering previously prescribed recourses ineffective. To address this problem, we propose a novel framework, RObust Algorithmic Recourse (ROAR), that leverages adversarial training for finding recourses that are robust to model shifts. To the best of our knowledge, this work proposes the first solution to this critical problem. We also carry out detailed theoretical analysis which underscores the importance of constructing recourses that are robust to model shifts: 1) we derive a lower bound on the probability of invalidation of recourses generated by existing approaches which are not robust to model shifts. 2) we prove that the additional cost incurred due to the robust recourses output by our framework is bounded. Experimental evaluation on multiple synthetic and real-world datasets demonstrates the efficacy of the proposed framework and supports our theoretical findings.
ECO: Enabling Energy-Neutral IoT Devices through Runtime Allocation of Harvested Energy
Authors: Yigit Tuncel, Ganapati Bhat, JaeHyun Park, Umit Ogras
Date: 2021-02-26 00:00:00

Energy harvesting offers an attractive and promising mechanism to power low-energy devices. However, it alone is insufficient to enable an energy-neutral operation, which can eliminate tedious battery charging and replacement requirements. Achieving an energy-neutral operation is challenging since the uncertainties in harvested energy undermine the quality of service requirements. To address this challenge, we present a runtime energy-allocation framework that optimizes the utility of the target device under energy constraints using a rollout algorithm, which is a sequential approach to solve dynamic optimization problems. The proposed framework uses an efficient iterative algorithm to compute initial energy allocations at the beginning of a day. The initial allocations are then corrected at every interval to compensate for the deviations from the expected energy harvesting pattern. We evaluate this framework using solar and motion energy harvesting modalities and American Time Use Survey data from 4772 different users. Compared to prior techniques, the proposed framework achieves up to 35% higher utility even under energy-limited scenarios. Moreover, measurements on a wearable device prototype show that the proposed framework has 1000x smaller energy overhead than iterative approaches with a negligible loss in utility.
Federated Edge Learning with Misaligned Over-The-Air Computation
Authors: Yulin Shao, Deniz Gunduz, Soung Chang Liew
Date: 2021-02-26 00:00:00

Over-the-air computation (OAC) is a promising technique to realize fast model aggregation in the uplink of federated edge learning. OAC, however, hinges on accurate channel-gain precoding and strict synchronization among the edge devices, which are challenging in practice. As such, how to design the maximum likelihood (ML) estimator in the presence of residual channel-gain mismatch and asynchronies is an open problem. To fill this gap, this paper formulates the problem of misaligned OAC for federated edge learning and puts forth a whitened matched filtering and sampling scheme to obtain oversampled, but independent, samples from the misaligned and overlapped signals. Given the whitened samples, a sum-product ML estimator and an aligned-sample estimator are devised to estimate the arithmetic sum of the transmitted symbols. In particular, the computational complexity of our sum-product ML estimator is linear in the packet length and hence is significantly lower than the conventional ML estimator. Extensive simulations on the test accuracy versus the average received energy per symbol to noise power spectral density ratio (EsN0) yield two main results: 1) In the low EsN0 regime, the aligned-sample estimator can achieve superior test accuracy provided that the phase misalignment is non-severe. In contrast, the ML estimator does not work well due to the error propagation and noise enhancement in the estimation process. 2) In the high EsN0 regime, the ML estimator attains the optimal learning performance regardless of the severity of phase misalignment. On the other hand, the aligned-sample estimator suffers from a test-accuracy loss caused by phase misalignment.
Distribution-Aware Testing of Neural Networks Using Generative Models
Authors: Swaroopa Dola, Matthew B. Dwyer, Mary Lou Soffa
Date: 2021-02-26 00:00:00
Tasks: DNN Testing, valid

The reliability of software that has a Deep Neural Network (DNN) as a component is urgently important today given the increasing number of critical applications being deployed with DNNs. The need for reliability raises a need for rigorous testing of the safety and trustworthiness of these systems. In the last few years, there have been a number of research efforts focused on testing DNNs. However the test generation techniques proposed so far lack a check to determine whether the test inputs they are generating are valid, and thus invalid inputs are produced. To illustrate this situation, we explored three recent DNN testing techniques. Using deep generative model based input validation, we show that all the three techniques generate significant number of invalid test inputs. We further analyzed the test coverage achieved by the test inputs generated by the DNN testing techniques and showed how invalid test inputs can falsely inflate test coverage metrics. To overcome the inclusion of invalid inputs in testing, we propose a technique to incorporate the valid input space of the DNN model under test in the test generation process. Our technique uses a deep generative model-based algorithm to generate only valid inputs. Results of our empirical studies show that our technique is effective in eliminating invalid tests and boosting the number of valid test inputs generated.
Potential benefits of delaying the second mRNA COVID-19 vaccine dose
Authors: Benjamin F. Maier, Angelique Burdinski, Annika H. Rose, Frank Schlosser, David Hinrichs, Cornelia Betsch, Lars Korn, Philipp Sprengholz, Michael Meyer-Hermann, Tanmay Mitra, Karl Lauterbach, Dirk Brockmann
Date: 2021-02-26 00:00:00

Vaccination against COVID-19 with the recently approved mRNA vaccines BNT162b2 (BioNTech/Pfizer) and mRNA-1273 (Moderna) is currently underway in a large number of countries. However, high incidence rates and rapidly spreading SARS-CoV-2 variants are concerning. In combination with acute supply deficits in Europe in early 2021, the question arises of whether stretching the vaccine, for instance by delaying the second dose, can make a significant contribution to preventing deaths, despite associated risks such as lower vaccine efficacy, the potential emergence of escape mutants, enhancement, waning immunity, reduced social acceptance of off-label vaccination, and liability shifts. A quantitative epidemiological assessment of risks and benefits of non-standard vaccination protocols remains elusive. To clarify the situation and to provide a quantitative epidemiological foundation we develop a stochastic epidemiological model that integrates specific vaccine rollout protocols into a risk-group structured infectious disease dynamical model. Using the situation and conditions in Germany as a reference system, we show that delaying the second vaccine dose is expected to prevent deaths in the four to five digit range, should the incidence resurge. We show that this considerable public health benefit relies on the fact that both mRNA vaccines provide substantial protection against severe COVID-19 and death beginning 12 to 14 days after the first dose. The benefits of protocol change are attenuated should vaccine compliance decrease substantially. To quantify the impact of protocol change on vaccination adherence we performed a large-scale online survey. We find that, in Germany, changing vaccination protocols may lead to small reductions in vaccination intention. In sum, we therefore expect the benefits of a strategy change to remain substantial and stable.
Estimation and Distributed Eradication of SIR Epidemics on Networks
Authors: Ciyuan Zhang, Humphrey Leung, Brooks Butler, Philip. E. Paré
Date: 2021-02-24 00:00:00

This work examines the discrete-time networked SIR (susceptible-infected-recovered) epidemic model, where the infection and recovery parameters may be time-varying. We provide a sufficient condition for the SIR model to converge to the set of healthy states exponentially. We propose a stochastic framework to estimate the system states from observed testing data and provide an analytic expression for the error of the estimation algorithm. Employing the estimated and the true system states, we provide two novel eradication strategies that guarantee at least exponential convergence to the set of healthy states. We illustrate the results via simulations over northern Indiana, USA.
Robust Pollen Imagery Classification with Generative Modeling and Mixup Training
Authors: Jaideep Murkute
Date: 2021-02-25 00:00:00
Tasks: Classification, Data Augmentation, Deep Learning, General Classification, image-classification, Image Classification

Deep learning approaches have shown great success in image classification tasks and can aid greatly towards the fast and reliable classification of pollen grain aerial imagery. However, often-times deep learning methods in the setting of natural images can suffer generalization problems and yield poor performance on unseen test distribution. In this work, we present and a robust deep learning framework that can generalize well for pollen grain aerobiological imagery classification. We develop a convolutional neural network-based pollen grain classification approach and combine some of the best practices in deep learning for better generalization. In addition to commonplace approaches like data-augmentation and weight regularization, we utilize implicit regularization methods like manifold mixup to allow learning of smoother decision boundaries. We also make use of proven state-of-the-art architectural choices like EfficientNet convolutional neural networks. Inspired by the success of generative modeling with variational autoencoders, we train models with a richer learning objective which can allow the model to focus on the relevant parts of the image. Finally, we create an ensemble of neural networks, for the robustness of the test set predictions. Based on our experiments, we show improved generalization performance as measured with a weighted F1-score with the aforementioned approaches. The proposed approach earned a fourth-place in the final rankings in the ICPR-2020 Pollen Grain Classification Challenge; with a 0.972578 weighted F1 score,0.950828 macro average F1 scores, and 0.972877 recognition accuracy.
Multi-Domain Learning by Meta-Learning: Taking Optimal Steps in Multi-Domain Loss Landscapes by Inner-Loop Learning
Authors: Anthony Sicilia, Xingchen Zhao, Davneet Minhas, Erin O'Connor, Howard Aizenstein, William Klunk, Dana Tudorascu, Seong Jae Hwang
Date: 2021-02-25 00:00:00
Tasks: Meta-Learning, Semantic Segmentation

We consider a model-agnostic solution to the problem of Multi-Domain Learning (MDL) for multi-modal applications. Many existing MDL techniques are model-dependent solutions which explicitly require nontrivial architectural changes to construct domain-specific modules. Thus, properly applying these MDL techniques for new problems with well-established models, e.g. U-Net for semantic segmentation, may demand various low-level implementation efforts. In this paper, given emerging multi-modal data (e.g., various structural neuroimaging modalities), we aim to enable MDL purely algorithmically so that widely used neural networks can trivially achieve MDL in a model-independent manner. To this end, we consider a weighted loss function and extend it to an effective procedure by employing techniques from the recently active area of learning-to-learn (meta-learning). Specifically, we take inner-loop gradient steps to dynamically estimate posterior distributions over the hyperparameters of our loss function. Thus, our method is model-agnostic, requiring no additional model parameters and no network architecture changes; instead, only a few efficient algorithmic modifications are needed to improve performance in MDL. We demonstrate our solution to a fitting problem in medical imaging, specifically, in the automatic segmentation of white matter hyperintensity (WMH). We look at two neuroimaging modalities (T1-MR and FLAIR) with complementary information fitting for our problem.
Boundary-induced and scene-aggregated network for monocular depth prediction
Authors: Feng Xue, Junfeng Cao, Yu Zhou, Fei Sheng, Yankai Wang, Anlong Ming
Date: 2021-02-26 00:00:00
Tasks: Depth Estimation, Depth Prediction, Scene Understanding

Monocular depth prediction is an important task in scene understanding. It aims to predict the dense depth of a single RGB image. With the development of deep learning, the performance of this task has made great improvements. However, two issues remain unresolved: (1) The deep feature encodes the wrong farthest region in a scene, which leads to a distorted 3D structure of the predicted depth; (2) The low-level features are insufficient utilized, which makes it even harder to estimate the depth near the edge with sudden depth change. To tackle these two issues, we propose the Boundary-induced and Scene-aggregated network (BS-Net). In this network, the Depth Correlation Encoder (DCE) is first designed to obtain the contextual correlations between the regions in an image, and perceive the farthest region by considering the correlations. Meanwhile, the Bottom-Up Boundary Fusion (BUBF) module is designed to extract accurate boundary that indicates depth change. Finally, the Stripe Refinement module (SRM) is designed to refine the dense depth induced by the boundary cue, which improves the boundary accuracy of the predicted depth. Several experimental results on the NYUD v2 dataset and \xff{the iBims-1 dataset} illustrate the state-of-the-art performance of the proposed approach. And the SUN-RGBD dataset is employed to evaluate the generalization of our method. Code is available at https://github.com/XuefengBUPT/BS-Net.
Improving Robustness of Learning-based Autonomous Steering Using Adversarial Images
Authors: Yu Shen, Laura Zheng, Manli Shu, Weizi Li, Tom Goldstein, Ming C. Lin
Date: 2021-02-26 00:00:00
Tasks: Autonomous Driving, Data Augmentation, Decision Making

For safety of autonomous driving, vehicles need to be able to drive under various lighting, weather, and visibility conditions in different environments. These external and environmental factors, along with internal factors associated with sensors, can pose significant challenges to perceptual data processing, hence affecting the decision-making and control of the vehicle. In this work, we address this critical issue by introducing a framework for analyzing robustness of the learning algorithm w.r.t varying quality in the image input for autonomous driving. Using the results of sensitivity analysis, we further propose an algorithm to improve the overall performance of the task of "learning to steer". The results show that our approach is able to enhance the learning outcomes up to 48%. A comparative study drawn between our approach and other related techniques, such as data augmentation and adversarial training, confirms the effectiveness of our algorithm as a way to improve the robustness and generalization of neural network training for autonomous driving.
Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification
Authors: Yi Zhou, Lei Huang, Tianfei Zhou, Ling Shao
Date: 2021-02-26 00:00:00

Chest X-rays are an important and accessible clinical imaging tool for the detection of many thoracic diseases. Over the past decade, deep learning, with a focus on the convolutional neural network (CNN), has become the most powerful computer-aided diagnosis technology for improving disease identification performance. However, training an effective and robust deep CNN usually requires a large amount of data with high annotation quality. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. Thus, existing public chest X-ray datasets usually adopt language pattern based methods to automatically mine labels from reports. However, this results in label uncertainty and inconsistency. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods from two perspectives to improve a single model's disease identification performance, rather than focusing on an ensemble of models. MODL integrates multiple models to obtain a soft label distribution for optimizing the single target model, which can reduce the effects of original label uncertainty. Moreover, KNNS aims to enhance the robustness of the target model to provide consistent predictions on images with similar medical findings. Extensive experiments on the public NIH Chest X-ray and CheXpert datasets show that our model achieves consistent improvements over the state-of-the-art methods.