pwc-archive/papers-with-abstracts huggingface dataset:
A Framework for Causal Discovery in non-intervenable systems
Authors: Peter Jan van Leeuwen, Michael DeCaria, Nachiketa Chakaborty, Manuel Pulido
Date: 2020-10-05 00:00:00
Tasks: Blocking, Causal Discovery, Disentanglement

Many frameworks exist to infer cause and effect relations in complex nonlinear systems but a complete theory is lacking. A new framework is presented that is fully nonlinear, provides a complete information theoretic disentanglement of causal processes, allows for nonlinear interactions between causes, identifies the causal strength of missing or unknown processes, and can analyze systems that cannot be represented on Directed Acyclic Graphs. The basic building blocks are information theoretic measures such as (conditional) mutual information and a new concept called certainty that monotonically increases with the information available about the target process. The framework is presented in detail and compared with other existing frameworks, and the treatment of confounders is discussed. While there are systems with structures that the framework cannot disentangle, it is argued that any causal framework that is based on integrated quantities will miss out potentially important information of the underlying probability density functions. The framework is tested on several highly simplified stochastic processes to demonstrate how blocking and gateways are handled, and on the chaotic Lorentz 1963 system. We show that the framework provides information on the local dynamics, but also reveals information on the larger scale structure of the underlying attractor. Furthermore, by applying it to real observations related to the El-Nino-Southern-Oscillation system we demonstrate its power and advantage over other methodologies.
MedFilter: Improving Extraction of Task-relevant Utterances from Doctor-Patient Conversations through Integration of Discourse Structure and Ontological Knowledge
Authors: Sopan Khosla, Shikhar Vashishth, Jill Fain Lehman, Carolyn Rose
Date: 2020-10-05 00:00:00
Proceeding: EMNLP 2020 11

Information extraction from conversational data is particularly challenging because the task-centric nature of conversation allows for effective communication of implicit information by humans, but is challenging for machines. The challenges may differ between utterances depending on the role of the speaker within the conversation, especially when relevant expertise is distributed asymmetrically across roles. Further, the challenges may also increase over the conversation as more shared context is built up through information communicated implicitly earlier in the dialogue. In this paper, we propose the novel modeling approach MedFilter, which addresses these insights in order to increase performance at identifying and categorizing task-relevant utterances, and in so doing, positively impacts performance at a downstream information extraction task. We evaluate this approach on a corpus of nearly 7,000 doctor-patient conversations where MedFilter is used to identify medically relevant contributions to the discussion (achieving a 10% improvement over SOTA baselines in terms of area under the PR curve). Identifying task-relevant utterances benefits downstream medical processing, achieving improvements of 15%, 105%, and 23% respectively for the extraction of symptoms, medications, and complaints.
Conditional Generative Adversarial Networks to Model Urban Outdoor Air Pollution
Authors: Jamal Toutouh
Date: 2020-10-05 00:00:00
Tasks: Time Series, Time Series Analysis

This is a relevant problem because the design of most cities prioritizes the use of motorized vehicles, which has degraded air quality in recent years, having a negative effect on urban health. Modeling, predicting, and forecasting ambient air pollution is an important way to deal with this issue because it would be helpful for decision-makers and urban city planners to understand the phenomena and to take solutions. In general, data-driven methods for modeling, predicting, and forecasting outdoor pollution requires an important amount of data, which may limit their accuracy. In order to deal with such a lack of data, we propose to train models able to generate synthetic nitrogen dioxide daily time series according to a given classification that will allow an unlimited generation of realistic data. The main experimental results indicate that the proposed approach is able to generate accurate and diverse pollution daily time series, while requiring reduced computational time.
Acrostic Poem Generation
Authors: Rajat Agarwal, Katharina Kann
Date: 2020-10-05 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Language Modelling

We propose a new task in the area of computational creativity: acrostic poem generation in English. Acrostic poems are poems that contain a hidden message; typically, the first letter of each line spells out a word or short phrase. We define the task as a generation task with multiple constraints: given an input word, 1) the initial letters of each line should spell out the provided word, 2) the poem's semantics should also relate to it, and 3) the poem should conform to a rhyming scheme. We further provide a baseline model for the task, which consists of a conditional neural language model in combination with a neural rhyming model. Since no dedicated datasets for acrostic poem generation exist, we create training data for our task by first training a separate topic prediction model on a small set of topic-annotated poems and then predicting topics for additional poems. Our experiments show that the acrostic poems generated by our baseline are received well by humans and do not lose much quality due to the additional constraints. Last, we confirm that poems generated by our model are indeed closely related to the provided prompts, and that pretraining on Wikipedia can boost performance.
Learning to Generalize for Sequential Decision Making
Authors: Xusen Yin, Ralph Weischedel, Jonathan May
Date: 2020-10-05 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Decision Making, Imitation Learning, Natural Language Understanding, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

We consider problems of making sequences of decisions to accomplish tasks, interacting via the medium of language. These problems are often tackled with reinforcement learning approaches. We find that these models do not generalize well when applied to novel task domains. However, the large amount of computation necessary to adequately train and explore the search space of sequential decision making, under a reinforcement learning paradigm, precludes the inclusion of large contextualized language models, which might otherwise enable the desired generalization ability. We introduce a teacher-student imitation learning methodology and a means of converting a reinforcement learning model into a natural language understanding model. Together, these methodologies enable the introduction of contextualized language models into the sequential decision making problem space. We show that models can learn faster and generalize more, leveraging both the imitation learning and the reformulation. Our models exceed teacher performance on various held-out decision problems, by up to 7% on in-domain problems and 24% on out-of-domain problems.
CO2: Consistent Contrast for Unsupervised Visual Representation Learning
Authors: Chen Wei, Huiyu Wang, Wei Shen, Alan Yuille
Date: 2020-10-05 00:00:00
Proceeding: ICLR 2021 1
Tasks: Contrastive Learning, image-classification, Image Classification, object-detection, Object Detection, Pseudo Label, Representation Learning, Semantic Segmentation

Contrastive learning has been adopted as a core method for unsupervised visual representation learning. Without human annotation, the common practice is to perform an instance discrimination task: Given a query image crop, this task labels crops from the same image as positives, and crops from other randomly sampled images as negatives. An important limitation of this label assignment strategy is that it can not reflect the heterogeneous similarity between the query crop and each crop from other images, taking them as equally negative, while some of them may even belong to the same semantic class as the query. To address this issue, inspired by consistency regularization in semi-supervised learning on unlabeled data, we propose Consistent Contrast (CO2), which introduces a consistency regularization term into the current contrastive learning framework. Regarding the similarity of the query crop to each crop from other images as "unlabeled", the consistency term takes the corresponding similarity of a positive crop as a pseudo label, and encourages consistency between these two similarities. Empirically, CO2 improves Momentum Contrast (MoCo) by 2.9% top-1 accuracy on ImageNet linear protocol, 3.8% and 1.1% top-5 accuracy on 1% and 10% labeled semi-supervised settings. It also transfers to image classification, object detection, and semantic segmentation on PASCAL VOC. This shows that CO2 learns better visual representations for these downstream tasks.
Using Bayesian deep learning approaches for uncertainty-aware building energy surrogate models
Authors: Paul Westermann, Ralph Evins
Date: 2020-10-05 00:00:00

Fast machine learning-based surrogate models are trained to emulate slow, high-fidelity engineering simulation models to accelerate engineering design tasks. This introduces uncertainty as the surrogate is only an approximation of the original model. Bayesian methods can quantify that uncertainty, and deep learning models exist that follow the Bayesian paradigm. These models, namely Bayesian neural networks and Gaussian process models, enable us to give predictions together with an estimate of the model's uncertainty. As a result we can derive uncertainty-aware surrogate models that can automatically suspect unseen design samples that cause large emulation errors. For these samples, the high-fidelity model can be queried instead. This outlines how the Bayesian paradigm allows us to hybridize fast, but approximate, and slow, but accurate models. In this paper, we train two types of Bayesian models, dropout neural networks and stochastic variational Gaussian Process models, to emulate a complex high dimensional building energy performance simulation problem. The surrogate model processes 35 building design parameters (inputs) to estimate 12 different performance metrics (outputs). We benchmark both approaches, prove their accuracy to be competitive, and show that errors can be reduced by up to 30% when the 10% of samples with the highest uncertainty are transferred to the high-fidelity model.
Improved Analysis of Clipping Algorithms for Non-convex Optimization
Authors: Bohang Zhang, Jikai Jin, Cong Fang, LiWei Wang
Date: 2020-10-05 00:00:00
Proceeding: NeurIPS 2020 12

Gradient clipping is commonly used in training deep neural networks partly due to its practicability in relieving the exploding gradient problem. Recently, \citet{zhang2019gradient} show that clipped (stochastic) Gradient Descent (GD) converges faster than vanilla GD/SGD via introducing a new assumption called $(L_0, L_1)$-smoothness, which characterizes the violent fluctuation of gradients typically encountered in deep neural networks. However, their iteration complexities on the problem-dependent parameters are rather pessimistic, and theoretical justification of clipping combined with other crucial techniques, e.g. momentum acceleration, are still lacking. In this paper, we bridge the gap by presenting a general framework to study the clipping algorithms, which also takes momentum methods into consideration. We provide convergence analysis of the framework in both deterministic and stochastic setting, and demonstrate the tightness of our results by comparing them with existing lower bounds. Our results imply that the efficiency of clipping methods will not degenerate even in highly non-smooth regions of the landscape. Experiments confirm the superiority of clipping-based methods in deep learning tasks.
Adversarial Boot Camp: label free certified robustness in one epoch
Authors: Ryan Campbell, Chris Finlay, Adam M Oberman
Date: 2020-10-05 00:00:00

Machine learning models are vulnerable to adversarial attacks. One approach to addressing this vulnerability is certification, which focuses on models that are guaranteed to be robust for a given perturbation size. A drawback of recent certified models is that they are stochastic: they require multiple computationally expensive model evaluations with random noise added to a given input. In our work, we present a deterministic certification approach which results in a certifiably robust model. This approach is based on an equivalence between training with a particular regularized loss, and the expected values of Gaussian averages. We achieve certified models on ImageNet-1k by retraining a model with this loss for one epoch without the use of label information.
Wound and episode level readmission risk or weeks to readmit: Why do patients get readmitted? How long does it take for a patient to get readmitted?
Authors: Subba Reddy Oota, Nafisur Rahman, Shahid Saleem Mohammed, Jeffrey Galitz, Ming Liu
Date: 2020-10-05 00:00:00
Tasks: Management

The Affordable care Act of 2010 had introduced Readmission reduction program in 2012 to reduce avoidable re-admissions to control rising healthcare costs. Wound care impacts 15 of medicare beneficiaries making it one of the major contributors of medicare health care cost. Health plans have been exploring proactive health care services that can focus on preventing wound recurrences and re-admissions to control the wound care costs. With rising costs of Wound care industry, it has become of paramount importance to reduce wound recurrences & patient re-admissions. What factors are responsible for a Wound to recur which ultimately lead to hospitalization or re-admission? Is there a way to identify the patients at risk of re-admission before the occurrence using data driven analysis? Patient re-admission risk management has become critical for patients suffering from chronic wounds such as diabetic ulcers, pressure ulcers, and vascular ulcers. Understanding the risk & the factors that cause patient readmission can help care providers and patients avoid wound recurrences. Our work focuses on identifying patients who are at high risk of re-admission & determining the time period with in which a patient might get re-admitted. Frequent re-admissions add financial stress to the patient & Health plan and deteriorate the quality of life of the patient. Having this information can allow a provider to set up preventive measures that can delay, if not prevent, patients' re-admission. On a combined wound & episode-level data set of patient's wound care information, our extended autoprognosis achieves a recall of 92 and a precision of 92 for the predicting a patient's re-admission risk. For new patient class, precision and recall are as high as 91 and 98, respectively. We are also able to predict the patient's discharge event for a re-admission event to occur through our model with a MAE of 2.3 weeks.
Improving Reconstructive Surgery Design using Gaussian Process Surrogates to Capture Material Behavior Uncertainty
Authors: Casey Stowers, Taeksang Lee, Ilias Bilionis, Arun Gosain, Adrian Buganza Tepole
Date: 2020-10-05 00:00:00

Excessive loads near wounds produce pathological scarring and other complications. Presently, stress cannot easily be measured by surgeons in the operating room. Instead, surgeons rely on intuition and experience. Predictive computational tools are ideal candidates for surgery planning. Finite element (FE) simulations have shown promise in predicting stress fields on large skin patches and complex cases, helping to identify potential regions of complication. Unfortunately, these simulations are computationally expensive and deterministic. However, running a few, well-selected FE simulations allows us to create Gaussian process (GP) surrogate models of local cutaneous flaps that are computationally efficient and able to predict stress and strain for arbitrary material parameters. Here, we create GP surrogates for the advancement, rotation, and transposition flaps. We then use the predictive capability of these surrogates to perform a global sensitivity analysis, ultimately showing that fiber direction has the most significant impact on strain field variations. We then perform an optimization to determine the optimal fiber direction for each flap for three different objectives driven by clinical guidelines. While material properties are not controlled by the surgeon and are actually a source of uncertainty, the surgeon can in fact control the orientation of the flap. Therefore, fiber direction is the only material parameter that can be optimized clinically. The optimization task relies on the efficiency of the GP surrogates to calculate the expected cost of different strategies when the uncertainty of other material parameters is included. We propose optimal flap orientations for the three cost functions and that can help in reducing stress resulting from the surgery and ultimately reduce complications associated with excessive mechanical loading near wounds.
Image Translation for Medical Image Generation -- Ischemic Stroke Lesions
Authors: Moritz Platscher, Jonathan Zopes, Christian Federau
Date: 2020-10-05 00:00:00
Tasks: Data Augmentation, Generative Adversarial Network, Image Generation, Image-to-Image Translation, Lesion Segmentation, Medical Image Generation, Semantic Segmentation, Translation

Deep learning based disease detection and segmentation algorithms promise to improve many clinical processes. However, such algorithms require vast amounts of annotated training data, which are typically not available in the medical context due to data privacy, legal obstructions, and non-uniform data acquisition protocols. Synthetic databases with annotated pathologies could provide the required amounts of training data. We demonstrate with the example of ischemic stroke that an improvement in lesion segmentation is feasible using deep learning based augmentation. To this end, we train different image-to-image translation models to synthesize magnetic resonance images of brain volumes with and without stroke lesions from semantic segmentation maps. In addition, we train a generative adversarial network to generate synthetic lesion masks. Subsequently, we combine these two components to build a large database of synthetic stroke images. The performance of the various models is evaluated using a U-Net which is trained to segment stroke lesions on a clinical test set. We report a Dice score of $\mathbf{72.8}$% [$\mathbf{70.8\pm1.0}$%] for the model with the best performance, which outperforms the model trained on the clinical images alone $\mathbf{67.3}$% [$\mathbf{63.2\pm1.9}$%], and is close to the human inter-reader Dice score of $\mathbf{76.9}$%. Moreover, we show that for a small database of only 10 or 50 clinical cases, synthetic data augmentation yields significant improvement compared to a setting where no synthetic data is used. To the best of our knowledge, this presents the first comparative analysis of synthetic data augmentation based on image-to-image translation, and first application to ischemic stroke.
Characterization of surface motion patterns in highly deformable soft tissue organs from dynamic MRI: An application to assess 4D bladder motion
Authors: Karim Makki, Amine Bohi, Augustin . C Ogier, Marc Emmanuel Bellemare
Date: 2020-10-05 00:00:00

Dynamic MRI may capture temporal anatomical changes in soft tissue organs with high contrast but the obtained sequences usually suffer from limited volume coverage which makes the high resolution reconstruction of organ shape trajectories a major challenge in temporal studies. Because of the variability of abdominal organ shapes across time and subjects, the objective of this study is to go towards 3D dense velocity measurements to fully cover the entire surface and to extract meaningful features characterizing the observed organ deformations and enabling clinical action or decision. We present a pipeline for characterization of bladder surface dynamics during deep respiratory movements. For a compact shape representation, the reconstructed temporal volumes were first used to establish subject-specific dynamical 4D mesh sequences using the LDDMM framework. Then, we performed a statistical characterization of organ dynamics from mechanical parameters such as mesh elongations and distortions. Since we refer to organs as non flat surfaces, we have also used the mean curvature changes as metric to quantify surface evolution. However, the numerical computation of curvature is strongly dependant on the surface parameterization. To cope with this dependency, we employed a new method for surface deformation analysis. Independent of parameterization and minimizing the length of the geodesic curves, it stretches smoothly the surface curves towards a sphere by minimizing a Dirichlet energy. An Eulerian PDE approach is used to derive a shape descriptor from the curve-shortening flow. Intercorrelations between individual motion patterns are computed using the Laplace Beltrami operator eigenfunctions for spherical mapping. Application to extracting characterization correlation curves for locally controlled simulated shape trajectories demonstrates the stability of the proposed shape descriptor.
Neural Generation of Blocks for Video Coding
Authors: Jonah Probell
Date: 2020-10-05 00:00:00

Well-trained generative neural networks (GNN) are very efficient at compressing visual information for static images in their learned parameters but not as efficient as inter- and intra-prediction for most video content. However, for content entering a frame, such as during panning or zooming out, and content with curves, irregular shapes, or fine detail, generation by a GNN can give better compression efficiency (lower rate-distortion). This paper proposes encoding content-specific learned parameters of a GNN within a video bitstream at specific times and using the GNN to generate content for specific ranges of blocks and frames. The blocks to generate are just the ones for which generation gives more efficient compression than inter- or intra- prediction. This approach maximizes the usefulness of the information contained in the learned parameters.
Mapping of Sparse 3D Data using Alternating Projection
Authors: Siddhant Ranade, Xin Yu, Shantnu Kakkar, Pedro Miraldo, Srikumar Ramalingam
Date: 2020-10-04 00:00:00

We propose a novel technique to register sparse 3D scans in the absence of texture. While existing methods such as KinectFusion or Iterative Closest Points (ICP) heavily rely on dense point clouds, this task is particularly challenging under sparse conditions without RGB data. Sparse texture-less data does not come with high-quality boundary signal, and this prohibits the use of correspondences from corners, junctions, or boundary lines. Moreover, in the case of sparse data, it is incorrect to assume that the same point will be captured in two consecutive scans. We take a different approach and first re-parameterize the point-cloud using a large number of line segments. In this re-parameterized data, there exists a large number of line intersection (and not correspondence) constraints that allow us to solve the registration task. We propose the use of a two-step alternating projection algorithm by formulating the registration as the simultaneous satisfaction of intersection and rigidity constraints. The proposed approach outperforms other top-scoring algorithms on both Kinect and LiDAR datasets. In Kinect, we can use 100X downsampled sparse data and still outperform competing methods operating on full-resolution data.