pwc-archive/papers-with-abstracts huggingface dataset:
Graph Embedding with Hierarchical Attentive Membership
Authors: Lu Lin, Ethan Blaser, Hongning Wang
Date: 2021-10-31 00:00:00
Tasks: Graph Embedding, Link Prediction, Node Classification

The exploitation of graph structures is the key to effectively learning representations of nodes that preserve useful information in graphs. A remarkable property of graph is that a latent hierarchical grouping of nodes exists in a global perspective, where each node manifests its membership to a specific group based on the context composed by its neighboring nodes. Most prior works ignore such latent groups and nodes' membership to different groups, not to mention the hierarchy, when modeling the neighborhood structure. Thus, they fall short of delivering a comprehensive understanding of the nodes under different contexts in a graph. In this paper, we propose a novel hierarchical attentive membership model for graph embedding, where the latent memberships for each node are dynamically discovered based on its neighboring context. Both group-level and individual-level attentions are performed when aggregating neighboring states to generate node embeddings. We introduce structural constraints to explicitly regularize the inferred memberships of each node, such that a well-defined hierarchical grouping structure is captured. The proposed model outperformed a set of state-of-the-art graph embedding solutions on node classification and link prediction tasks in a variety of graphs including citation networks and social networks. Qualitative evaluations visualize the learned node embeddings along with the inferred memberships, which proved the concept of membership hierarchy and enables explainable embedding learning in graphs.
TorchXRayVision: A library of chest X-ray datasets and models
Authors: Joseph Paul Cohen, Joseph D. Viviano, Paul Bertin, Paul Morrison, Parsa Torabian, Matteo Guarrera, Matthew P Lungren, Akshay Chaudhari, Rupert Brooks, Mohammad Hashir, Hadrien Bertrand
Date: 2021-10-31 00:00:00
Tasks: Image Classification, Medical Image Retrieval, Medical Image Segmentation, Medical X-Ray Image Segmentation, Multi-class Classification, Representation Learning, Small Data Image Classification

TorchXRayVision is an open source software library for working with chest X-ray datasets and deep learning models. It provides a common interface and common pre-processing chain for a wide set of publicly available chest X-ray datasets. In addition, a number of classification and representation learning models with different architectures, trained on different data combinations, are available through the library to serve as baselines or feature extractors.
Unsupervised Learning to Subphenotype Delirium Patients from Electronic Health Records
Authors: Yiqing Zhao, Yuan Luo
Date: 2021-10-31 00:00:00
Tasks: Feature Importance

Delirium is a common acute onset brain dysfunction in the emergency setting and is associated with higher mortality. It is difficult to detect and monitor since its presentations and risk factors can be different depending on the underlying medical condition of patients. In our study, we aimed to identify subtypes within the delirium population and build subgroup-specific predictive models to detect delirium using Medical Information Mart for Intensive Care IV (MIMIC-IV) data. We showed that clusters exist within the delirium population. Differences in feature importance were also observed for subgroup-specific predictive models. Our work could recalibrate existing delirium prediction models for each delirium subgroup and improve the precision of delirium detection and monitoring for ICU or emergency department patients who had highly heterogeneous medical conditions.
What Went Wrong? Explaining Overall Dialogue Quality through Utterance-Level Impacts
Authors: James D. Finch, Sarah E. Finch, Jinho D. Choi
Date: 2021-10-31 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11
Tasks: Chatbot, Weakly-supervised Learning

Improving user experience of a dialogue system often requires intensive developer effort to read conversation logs, run statistical analyses, and intuit the relative importance of system shortcomings. This paper presents a novel approach to automated analysis of conversation logs that learns the relationship between user-system interactions and overall dialogue quality. Unlike prior work on utterance-level quality prediction, our approach learns the impact of each interaction from the overall user rating without utterance-level annotation, allowing resultant model conclusions to be derived on the basis of empirical evidence and at low cost. Our model identifies interactions that have a strong correlation with the overall dialogue quality in a chatbot setting. Experiments show that the automated analysis from our model agrees with expert judgments, making this work the first to show that such weakly-supervised learning of utterance-level quality prediction is highly achievable.
An Approach to Inference-Driven Dialogue Management within a Social Chatbot
Authors: Sarah E. Finch, James D. Finch, Daniil Huryn, William Hutsell, Xiaoyuan Huang, Han He, Jinho D. Choi
Date: 2021-10-31 00:00:00
Tasks: Chatbot, Collaborative Inference, Dialogue Management, Graph Matching, Management, Response Generation

We present a chatbot implementing a novel dialogue management approach based on logical inference. Instead of framing conversation a sequence of response generation tasks, we model conversation as a collaborative inference process in which speakers share information to synthesize new knowledge in real time. Our chatbot pipeline accomplishes this modelling in three broad stages. The first stage translates user utterances into a symbolic predicate representation. The second stage then uses this structured representation in conjunction with a larger knowledge base to synthesize new predicates using efficient graph matching. In the third and final stage, our bot selects a small subset of predicates and translates them into an English response. This approach lends itself to understanding latent semantics of user inputs, flexible initiative taking, and responses that are novel and coherent with the dialogue context.
Policy Optimization for Constrained MDPs with Provable Fast Global Convergence
Authors: Tao Liu, Ruida Zhou, Dileep Kalathil, P. R. Kumar, Chao Tian
Date: 2021-10-31 00:00:00

We address the problem of finding the optimal policy of a constrained Markov decision process (CMDP) using a gradient descent-based algorithm. Previous results have shown that a primal-dual approach can achieve an $\mathcal{O}(1/\sqrt{T})$ global convergence rate for both the optimality gap and the constraint violation. We propose a new algorithm called policy mirror descent-primal dual (PMD-PD) algorithm that can provably achieve a faster $\mathcal{O}(\log(T)/T)$ convergence rate for both the optimality gap and the constraint violation. For the primal (policy) update, the PMD-PD algorithm utilizes a modified value function and performs natural policy gradient steps, which is equivalent to a mirror descent step with appropriate regularization. For the dual update, the PMD-PD algorithm uses modified Lagrange multipliers to ensure a faster convergence rate. We also present two extensions of this approach to the settings with zero constraint violation and sample-based estimation. Experimental results demonstrate the faster convergence rate and the better performance of the PMD-PD algorithm compared with existing policy gradient-based algorithms.
Template Filling for Controllable Commonsense Reasoning
Authors: Dheeraj Rajagopal, Vivek Khetan, Bogdan Sacaleanu, Anatole Gershman, Andrew Fano, Eduard Hovy
Date: 2021-10-31 00:00:00
Tasks: Multiple-choice

Large-scale sequence-to-sequence models have shown to be adept at both multiple-choice and open-domain commonsense reasoning tasks. However, the current systems do not provide the ability to control the various attributes of the reasoning chain. To enable better controllability, we propose to study the commonsense reasoning as a template filling task (TemplateCSR) -- where the language models fills reasoning templates with the given constraints as control factors. As an approach to TemplateCSR, we (i) propose a dataset of commonsense reasoning template-expansion pairs and (ii) introduce POTTER, a pretrained sequence-to-sequence model using prompts to perform commonsense reasoning across concepts. Our experiments show that our approach outperforms baselines both in generation metrics and factuality metrics. We also present a detailed error analysis on our approach's ability to reliably perform commonsense reasoning.
Focal Attention Networks: optimising attention for biomedical image segmentation
Authors: Michael Yeung, Leonardo Rundo, Evis Sala, Carola-Bibiane Schönlieb, Guang Yang
Date: 2021-10-31 00:00:00
Tasks: Image Segmentation, Semantic Segmentation

In recent years, there has been increasing interest to incorporate attention into deep learning architectures for biomedical image segmentation. The modular design of attention mechanisms enables flexible integration into convolutional neural network architectures, such as the U-Net. Whether attention is appropriate to use, what type of attention to use, and where in the network to incorporate attention modules, are all important considerations that are currently overlooked. In this paper, we investigate the role of the Focal parameter in modulating attention, revealing a link between attention in loss functions and networks. By incorporating a Focal distance penalty term, we extend the Unified Focal loss framework to include boundary-based losses. Furthermore, we develop a simple and interpretable, dataset and model-specific heuristic to integrate the Focal parameter into the Squeeze-and-Excitation block and Attention Gate, achieving optimal performance with fewer number of attention modules on three well-validated biomedical imaging datasets, suggesting judicious use of attention modules results in better performance and efficiency.
Incorporating Boundary Uncertainty into loss functions for biomedical image segmentation
Authors: Michael Yeung, Guang Yang, Evis Sala, Carola-Bibiane Schönlieb, Leonardo Rundo
Date: 2021-10-31 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Manual segmentation is used as the gold-standard for evaluating neural networks on automated image segmentation tasks. Due to considerable heterogeneity in shapes, colours and textures, demarcating object boundaries is particularly difficult in biomedical images, resulting in significant inter and intra-rater variability. Approaches, such as soft labelling and distance penalty term, apply a global transformation to the ground truth, redefining the loss function with respect to uncertainty. However, global operations are computationally expensive, and neither approach accurately reflects the uncertainty underlying manual annotation. In this paper, we propose the Boundary Uncertainty, which uses morphological operations to restrict soft labelling to object boundaries, providing an appropriate representation of uncertainty in ground truth labels, and may be adapted to enable robust model training where systematic manual segmentation errors are present. We incorporate Boundary Uncertainty with the Dice loss, achieving consistently improved performance across three well-validated biomedical imaging datasets compared to soft labelling and distance-weighted penalty. Boundary Uncertainty not only more accurately reflects the segmentation process, but it is also efficient, robust to segmentation errors and exhibits better generalisation.
Calibrating the Dice loss to handle neural network overconfidence for biomedical image segmentation
Authors: Michael Yeung, Leonardo Rundo, Yang Nan, Evis Sala, Carola-Bibiane Schönlieb, Guang Yang
Date: 2021-10-31 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

The Dice similarity coefficient (DSC) is both a widely used metric and loss function for biomedical image segmentation due to its robustness to class imbalance. However, it is well known that the DSC loss is poorly calibrated, resulting in overconfident predictions that cannot be usefully interpreted in biomedical and clinical practice. Performance is often the only metric used to evaluate segmentations produced by deep neural networks, and calibration is often neglected. However, calibration is important for translation into biomedical and clinical practice, providing crucial contextual information to model predictions for interpretation by scientists and clinicians. In this study, we provide a simple yet effective extension of the DSC loss, named the DSC++ loss, that selectively modulates the penalty associated with overconfident, incorrect predictions. As a standalone loss function, the DSC++ loss achieves significantly improved calibration over the conventional DSC loss across six well-validated open-source biomedical imaging datasets, including both 2D binary and 3D multi-class segmentation tasks. Similarly, we observe significantly improved calibration when integrating the DSC++ loss into four DSC-based loss functions. Finally, we use softmax thresholding to illustrate that well calibrated outputs enable tailoring of recall-precision bias, which is an important post-processing technique to adapt the model predictions to suit the biomedical or clinical task. The DSC++ loss overcomes the major limitation of the DSC loss, providing a suitable loss function for training deep learning segmentation models for use in biomedical and clinical practice. Source code is available at: https://github.com/mlyg/DicePlusPlus.
PnPOOD : Out-Of-Distribution Detection for Text Classification via Plug andPlay Data Augmentation
Authors: Mrinal Rawat, Ramya Hebbalaguppe, Lovekesh Vig
Date: 2021-10-31 00:00:00
Tasks: Data Augmentation, Language Modeling, Language Modelling, Out-of-Distribution Detection, Out of Distribution (OOD) Detection, text-classification, Text Classification

While Out-of-distribution (OOD) detection has been well explored in computer vision, there have been relatively few prior attempts in OOD detection for NLP classification. In this paper we argue that these prior attempts do not fully address the OOD problem and may suffer from data leakage and poor calibration of the resulting models. We present PnPOOD, a data augmentation technique to perform OOD detection via out-of-domain sample generation using the recently proposed Plug and Play Language Model (Dathathri et al., 2020). Our method generates high quality discriminative samples close to the class boundaries, resulting in accurate OOD detection at test time. We demonstrate that our model outperforms prior models on OOD sample detection, and exhibits lower calibration error on the 20 newsgroup text and Stanford Sentiment Treebank dataset (Lang, 1995; Socheret al., 2013). We further highlight an important data leakage issue with datasets used in prior attempts at OOD detection, and share results on a new dataset for OOD detection that does not suffer from the same problem.
DSC-IITISM at FinCausal 2021: Combining POS tagging with Attention-based Contextual Representations for Identifying Causal Relationships in Financial Documents
Authors: Gunjan Haldar, Aman Mittal, Pradyumna Gupta
Date: 2021-10-31 00:00:00
Proceeding: FNP 2021 9
Tasks: Financial Analysis, Information Retrieval, POS, POS Tagging, Question Answering, Retrieval

Causality detection draws plenty of attention in the field of Natural Language Processing and linguistics research. It has essential applications in information retrieval, event prediction, question answering, financial analysis, and market research. In this study, we explore several methods to identify and extract cause-effect pairs in financial documents using transformers. For this purpose, we propose an approach that combines POS tagging with the BIO scheme, which can be integrated with modern transformer models to address this challenge of identifying causality in a given text. Our best methodology achieves an F1-Score of 0.9551, and an Exact Match Score of 0.8777 on the blind test in the FinCausal-2021 Shared Task at the FinCausal 2021 Workshop.
All-In-One: Artificial Association Neural Networks
Authors: Seokjun Kim, Jaeeun Jang, Hyeoncheol Kim
Date: 2021-10-31 00:00:00
Tasks: All

Most deep learning models are limited to specific datasets or tasks because of network structures using fixed layers. In this paper, we discuss the differences between existing neural networks and real human neurons, propose association networks to connect existing models, and describe multiple types of deep learning exercises performed using a single structure. Further, we propose a new neural data structure that can express all basic models of existing neural networks in a tree structure. We also propose an approach in which information propagates from leaf to a root node using the proposed recursive convolution approach (i.e., depth-first convolution) and feed-forward propagation is performed. Thus, we design a ``data-based,'' as opposed to a ``model-based,'' neural network. In experiments conducted, we compared the learning performances of the models specializing in specific domains with those of models simultaneously learning various domains using an association network. The model learned well without significant performance degradation compared to that for models performing individual learning. In addition, the performance results were similar to those of the special case models; the output of the tree contained all information from the tree. Finally, we developed a theory for using arbitrary input data and learning all data simultaneously.
Interpreting Deep Knowledge Tracing Model on EdNet Dataset
Authors: DeLiang Wang, Yu Lu, Qinggang Meng, Penghe Chen
Date: 2021-10-31 00:00:00
Tasks: Knowledge Tracing, model

With more deep learning techniques being introduced into the knowledge tracing domain, the interpretability issue of the knowledge tracing models has aroused researchers' attention. Our previous study(Lu et al. 2020) on building and interpreting the KT model mainly adopts the ASSISTment dataset(Feng, Heffernan, and Koedinger 2009),, whose size is relatively small. In this work, we perform the similar tasks but on a large and newly available dataset, called EdNet(Choi et al. 2020). The preliminary experiment results show the effectiveness of the interpreting techniques, while more questions and tasks are worthy to be further explored and accomplished.
A Simple Approach to Image Tilt Correction with Self-Attention MobileNet for Smartphones
Authors: Siddhant Garg, Debi Prasanna Mohanty, Siva Prasad Thota, Sukumar Moharana
Date: 2021-10-31 00:00:00
Tasks: Multi-Label Learning

The main contributions of our work are two-fold. First, we present a Self-Attention MobileNet, called SA-MobileNet Network that can model long-range dependencies between the image features instead of processing the local region as done by standard convolutional kernels. SA-MobileNet contains self-attention modules integrated with the inverted bottleneck blocks of the MobileNetV3 model which results in modeling of both channel-wise attention and spatial attention of the image features and at the same time introduce a novel self-attention architecture for low-resource devices. Secondly, we propose a novel training pipeline for the task of image tilt detection. We treat this problem in a multi-label scenario where we predict multiple angles for a tilted input image in a narrow interval of range 1-2 degrees, depending on the dataset used. This process induces an implicit correlation between labels without any computational overhead of the second or higher-order methods in multi-label learning. With the combination of our novel approach and the architecture, we present state-of-the-art results on detecting the image tilt angle on mobile devices as compared to the MobileNetV3 model. Finally, we establish that SA-MobileNet is more accurate than MobileNetV3 on SUN397, NYU-V1, and ADE20K datasets by 6.42%, 10.51%, and 9.09% points respectively, and faster by at least 4 milliseconds on Snapdragon 750 Octa-core.
A robust single-pixel particle image velocimetry based on fully convolutional networks with cross-correlation embedded
Authors: Qi Gao, Hongtao Lin, Han Tu, Haoran Zhu, Runjie Wei, Guoping Zhang, Xueming Shao
Date: 2021-10-31 00:00:00
Tasks: Super-Resolution

Particle image velocimetry (PIV) is essential in experimental fluid dynamics. In the current work, we propose a new velocity field estimation paradigm, which achieves a synergetic combination of the deep learning method and the traditional cross-correlation method. Specifically, the deep learning method is used to optimize and correct a coarse velocity guess to achieve a super-resolution calculation. And the cross-correlation method provides the initial velocity field based on a coarse correlation with a large interrogation window. As a reference, the coarse velocity guess helps with improving the robustness of the proposed algorithm. This fully convolutional network with embedded cross-correlation is named as CC-FCN. CC-FCN has two types of input layers, one is for the particle images, and the other is for the initial velocity field calculated using cross-correlation with a coarse resolution. Firstly, two pyramidal modules extract features of particle images and initial velocity field respectively. Then the fusion module appropriately fuses these features. Finally, CC-FCN achieves the super-resolution calculation through a series of deconvolution layers to obtain the single-pixel velocity field. As the supervised learning strategy is considered, synthetic data sets including ground-truth fluid motions are generated to train the network parameters. Synthetic and real experimental PIV data sets are used to test the trained neural network in terms of accuracy, precision, spatial resolution and robustness. The test results show that these attributes of CC-FCN are further improved compared with those of other tested PIV algorithms. The proposed model could therefore provide competitive and robust estimations for PIV experiments.
Conical Classification For Computationally Efficient One-Class Topic Determination
Authors: Sameer Khanna
Date: 2021-10-31 00:00:00
Tasks: Classification, One-Class Classification

As the Internet grows in size, so does the amount of text based information that exists. For many application spaces it is paramount to isolate and identify texts that relate to a particular topic. While one-class classification would be ideal for such analysis, there is a relative lack of research regarding efficient approaches with high predictive power. By noting that the range of documents we wish to identify can be represented as positive linear combinations of the Vector Space Model representing our text, we propose Conical classification, an approach that allows us to identify if a document is of a particular topic in a computationally efficient manner. We also propose Normal Exclusion, a modified version of Bi-Normal Separation that makes it more suitable within the one-class classification context. We show in our analysis that our approach not only has higher predictive power on our datasets, but is also faster to compute.