pwc-archive/papers-with-abstracts huggingface dataset:
SSMF: Shifting Seasonal Matrix Factorization
Authors: Koki Kawabata, Siddharth Bhatia, Rui Liu, Mohit Wadhwa, Bryan Hooi
Date: 2021-10-25 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Data Compression

Given taxi-ride counts information between departure and destination locations, how can we forecast their future demands? In general, given a data stream of events with seasonal patterns that innovate over time, how can we effectively and efficiently forecast future events? In this paper, we propose Shifting Seasonal Matrix Factorization approach, namely SSMF, that can adaptively learn multiple seasonal patterns (called regimes), as well as switching between them. Our proposed method has the following properties: (a) it accurately forecasts future events by detecting regime shifts in seasonal patterns as the data stream evolves; (b) it works in an online setting, i.e., processes each observation in constant time and memory; (c) it effectively realizes regime shifts without human intervention by using a lossless data compression scheme. We demonstrate that our algorithm outperforms state-of-the-art baseline methods by accurately forecasting upcoming events on three real-world data streams.
Planning for Risk-Aversion and Expected Value in MDPs
Authors: Marc Rigter, Paul Duckworth, Bruno Lacerda, Nick Hawes
Date: 2021-10-25 00:00:00

Planning in Markov decision processes (MDPs) typically optimises the expected cost. However, optimising the expectation does not consider the risk that for any given run of the MDP, the total cost received may be unacceptably high. An alternative approach is to find a policy which optimises a risk-averse objective such as conditional value at risk (CVaR). However, optimising the CVaR alone may result in poor performance in expectation. In this work, we begin by showing that there can be multiple policies which obtain the optimal CVaR. This motivates us to propose a lexicographic approach which minimises the expected cost subject to the constraint that the CVaR of the total cost is optimal. We present an algorithm for this problem and evaluate our approach on four domains. Our results demonstrate that our lexicographic approach improves the expected cost compared to the state of the art algorithm, while achieving the optimal CVaR.
Fine-tuning of Pre-trained Transformers for Hate, Offensive, and Profane Content Detection in English and Marathi
Authors: Anna Glazkova, Michael Kadantsev, Maksim Glazkov
Date: 2021-10-25 00:00:00
Tasks: Hate Speech Detection, Sentence, Sentence Embedding, Sentence-Embedding

This paper describes neural models developed for the Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages Shared Task 2021. Our team called neuro-utmn-thales participated in two tasks on binary and fine-grained classification of English tweets that contain hate, offensive, and profane content (English Subtasks A & B) and one task on identification of problematic content in Marathi (Marathi Subtask A). For English subtasks, we investigate the impact of additional corpora for hate speech detection to fine-tune transformer models. We also apply a one-vs-rest approach based on Twitter-RoBERTa to discrimination between hate, profane and offensive posts. Our models ranked third in English Subtask A with the F1-score of 81.99% and ranked second in English Subtask B with the F1-score of 65.77%. For the Marathi tasks, we propose a system based on the Language-Agnostic BERT Sentence Embedding (LaBSE). This model achieved the second result in Marathi Subtask A obtaining an F1 of 88.08%.
TODSum: Task-Oriented Dialogue Summarization with State Tracking
Authors: Lulu Zhao, Fujia Zheng, Keqing He, Weihao Zeng, Yuejie Lei, Huixing Jiang, Wei Wu, Weiran Xu, Jun Guo, Fanyu Meng
Date: 2021-10-25 00:00:00

Previous dialogue summarization datasets mainly focus on open-domain chitchat dialogues, while summarization datasets for the broadly used task-oriented dialogue haven't been explored yet. Automatically summarizing such task-oriented dialogues can help a business collect and review needs to improve the service. Besides, previous datasets pay more attention to generate good summaries with higher ROUGE scores, but they hardly understand the structured information of dialogues and ignore the factuality of summaries. In this paper, we introduce a large-scale public Task-Oriented Dialogue Summarization dataset, TODSum, which aims to summarize the key points of the agent completing certain tasks with the user. Compared to existing work, TODSum suffers from severe scattered information issues and requires strict factual consistency, which makes it hard to directly apply recent dialogue summarization models. Therefore, we introduce additional dialogue state knowledge for TODSum to enhance the faithfulness of generated summaries. We hope a better understanding of conversational content helps summarization models generate concise and coherent summaries. Meanwhile, we establish a comprehensive benchmark for TODSum and propose a state-aware structured dialogue summarization model to integrate dialogue state information and dialogue history. Exhaustive experiments and qualitative analysis prove the effectiveness of dialogue structure guidance. Finally, we discuss the current issues of TODSum and potential development directions for future work.
Nearly Tight Convergence Bounds for Semi-discrete Entropic Optimal Transport
Authors: Alex Delalande
Date: 2021-10-25 00:00:00

We derive nearly tight and non-asymptotic convergence bounds for solutions of entropic semi-discrete optimal transport. These bounds quantify the stability of the dual solutions of the regularized problem (sometimes called Sinkhorn potentials) w.r.t. the regularization parameter, for which we ensure a better than Lipschitz dependence. Such facts may be a first step towards a mathematical justification of annealing or $\varepsilon$-scaling heuristics for the numerical resolution of regularized semi-discrete optimal transport. Our results also entail a non-asymptotic and tight expansion of the difference between the entropic and the unregularized costs.
QuantifyML: How Good is my Machine Learning Model?
Authors: Muhammad Usman, Divya Gopinath, Corina S. Păsăreanu
Date: 2021-10-25 00:00:00
Tasks: BIG-bench Machine Learning

The efficacy of machine learning models is typically determined by computing their accuracy on test data sets. However, this may often be misleading, since the test data may not be representative of the problem that is being studied. With QuantifyML we aim to precisely quantify the extent to which machine learning models have learned and generalized from the given data. Given a trained model, QuantifyML translates it into a C program and feeds it to the CBMC model checker to produce a formula in Conjunctive Normal Form (CNF). The formula is analyzed with off-the-shelf model counters to obtain precise counts with respect to different model behavior. QuantifyML enables i) evaluating learnability by comparing the counts for the outputs to ground truth, expressed as logical predicates, ii) comparing the performance of models built with different machine learning algorithms (decision-trees vs. neural networks), and iii) quantifying the safety and robustness of models.
Complete Agent-driven Model-based System Testing for Autonomous Systems
Authors: Kerstin I. Eder, Wen-ling Huang, Jan Peleska
Date: 2021-10-25 00:00:00

In this position paper, a novel approach to testing complex autonomous transportation systems (ATS) in the automotive, avionic, and railway domains is described. It is intended to mitigate some of the most critical problems regarding verification and validation (V&V) effort for ATS. V&V is known to become infeasible for complex ATS, when using conventional methods only. The approach advocated here uses complete testing methods on the module level, because these establish formal proofs for the logical correctness of the software. Having established logical correctness, system-level tests are performed in simulated cloud environments and on the target system. To give evidence that 'sufficiently many' system tests have been performed with the target system, a formally justified coverage criterion is introduced. To optimise the execution of very large system test suites, we advocate an online testing approach where multiple tests are executed in parallel, and test steps are identified on-the-fly. The coordination and optimisation of these executions is achieved by an agent-based approach. Each aspect of the testing approach advocated here is shown to either be consistent with existing standards for development and V&V of safety-critical transportation systems, or it is justified why it should become acceptable in future revisions of the applicable standards.
Towards Partial Monitoring: It is Always too Soon to Give Up
Authors: Angelo Ferrando, Rafael C. Cardoso
Date: 2021-10-25 00:00:00

Runtime Verification is a lightweight formal verification technique. It is used to verify at runtime whether the system under analysis behaves as expected. The expected behaviour is usually formally specified by means of properties, which are used to automatically synthesise monitors. A monitor is a device that, given a sequence of events representing a system execution, returns a verdict symbolising the satisfaction or violation of the formal property. Properties that can (resp. cannot) be verified at runtime by a monitor are called monitorable and non-monitorable, respectively. In this paper, we revise the notion of monitorability from a practical perspective, where we show how non-monitorable properties can still be used to generate partial monitors, which can partially check the properties. Finally, we present the implications both from a theoretical and practical perspectives.
Partially Intervenable Causal Models
Authors: AmirEmad Ghassami, Ilya Shpitser
Date: 2021-10-24 00:00:00
Tasks: Causal Inference

Graphical causal models led to the development of complete non-parametric identification theory in arbitrary structured systems, and general approaches to efficient inference. Nevertheless, graphical approaches to causal inference have not been embraced by the statistics and public health communities. In those communities causal assumptions are instead expressed in terms of potential outcomes, or responses to hypothetical interventions. Such interventions are generally conceptualized only on a limited set of variables, where the corresponding experiment could, in principle, be performed. By contrast, graphical approaches to causal inference generally assume interventions on all variables are well defined - an overly restrictive and unrealistic assumption that may have limited the adoption of these approaches in applied work in statistics and public health. In this paper, we build on a unification of graphical and potential outcomes approaches to causality exemplified by Single World Intervention Graphs (SWIGs) to define graphical models with a restricted set of allowed interventions. We give a complete identification theory for such models, and develop a complete calculus of interventions based on a generalization of the do-calculus, and axioms that govern probabilistic operations on Markov kernels. A corollary of our results is a complete identification theory for causal effects in another graphical framework with a restricted set of interventions, the decision theoretic graphical formulation of causality.
Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels
Authors: Jochen Görtler, Fred Hohman, Dominik Moritz, Kanit Wongsuphasawat, Donghao Ren, Rahul Nair, Marc Kirchner, Kayur Patel
Date: 2021-10-24 00:00:00
Tasks: BIG-bench Machine Learning

The confusion matrix, a ubiquitous visualization for helping people evaluate machine learning models, is a tabular layout that compares predicted class labels against actual class labels over all data instances. We conduct formative research with machine learning practitioners at Apple and find that conventional confusion matrices do not support more complex data-structures found in modern-day applications, such as hierarchical and multi-output labels. To express such variations of confusion matrices, we design an algebra that models confusion matrices as probability distributions. Based on this algebra, we develop Neo, a visual analytics system that enables practitioners to flexibly author and interact with hierarchical and multi-output confusion matrices, visualize derived metrics, renormalize confusions, and share matrix specifications. Finally, we demonstrate Neo's utility with three model evaluation scenarios that help people better understand model performance and reveal hidden confusions.
Reachability Embeddings: Scalable Self-Supervised Representation Learning from Mobility Trajectories for Multimodal Geospatial Computer Vision
Authors: Swetava Ganguli, C. V. Krishnakumar Iyer, Vipul Pandey
Date: 2021-10-24 00:00:00
Tasks: Representation Learning, Semantic Segmentation

Self-supervised representation learning techniques utilize large datasets without semantic annotations to learn meaningful, universal features that can be conveniently transferred to solve a wide variety of downstream supervised tasks. In this paper, we propose a self-supervised method for learning representations of geographic locations from unlabeled GPS trajectories to solve downstream geospatial computer vision tasks. Tiles resulting from a raster representation of the earth's surface are modeled as nodes on a graph or pixels of an image. GPS trajectories are modeled as allowed Markovian paths on these nodes. A scalable and distributed algorithm is presented to compute image-like tensors, called reachability summaries, of the spatial connectivity patterns between tiles and their neighbors implied by the observed Markovian paths. A convolutional, contractive autoencoder is trained to learn compressed representations, called reachability embeddings, of reachability summaries for every tile. Reachability embeddings serve as task-agnostic, feature representations of geographic locations. Using reachability embeddings as pixel representations for five different downstream geospatial tasks, cast as supervised semantic segmentation problems, we quantitatively demonstrate that reachability embeddings are semantically meaningful representations and result in 4-23% gain in performance, while using upto 67% less trajectory data, as measured using area under the precision-recall curve (AUPRC) metric, when compared to baseline models that use pixel representations that do not account for the spatial connectivity between tiles. Reachability embeddings transform sequential, spatiotemporal mobility data into semantically meaningful image-like tensor representations that can be combined with other sources of imagery and are designed to facilitate multimodal learning in geospatial computer vision.
Estimating Total Lung Volume from Pixel-level Thickness Maps of Chest Radiographs Using Deep Learning
Authors: Tina Dorosti, Manuel Schultheiss, Philipp Schmette, Jule Heuchert, Johannes Thalhammer, Florian T. Gassert, Thorsten Sellerer, Rafael Schick, Kirsten Taphorn, Korbinian Mechlem, Lorenz Birnbacher, Florian Schaff, Franz Pfeiffer, Daniela Pfeiffer
Date: 2021-10-24 00:00:00
Tasks: Capacity Estimation, Pulmonary Embolism Detection

Purpose: To estimate the total lung volume (TLV) from real and synthetic frontal chest radiographs (CXR) on a pixel level using lung thickness maps generated by a U-Net deep learning model. Methods: This retrospective study included 5,959 chest CT scans from two public datasets: the lung nodule analysis 2016 (n=656) and the Radiological Society of North America (RSNA) pulmonary embolism detection challenge 2020 (n=5,303). Additionally, 72 participants were selected from the Klinikum Rechts der Isar dataset (October 2018 to December 2019), each with a corresponding chest radiograph taken within seven days. Synthetic radiographs and lung thickness maps were generated using forward projection of CT scans and their lung segmentations. A U-Net model was trained on synthetic radiographs to predict lung thickness maps and estimate TLV. Model performance was assessed using mean squared error (MSE), Pearson correlation coefficient (r), and two-sided Student's t-distribution. Results: The study included 72 participants (45 male, 27 female, 33 healthy: mean age 62 years [range 34-80]; 39 with chronic obstructive pulmonary disease: mean age 69 years [range 47-91]). TLV predictions showed low error rates ($MSE_{Public-Synthetic}$=0.16 $L^2$, $MSE_{KRI-Synthetic}$=0.20 $L^2$, $MSE_{KRI-Real}$=0.35 $L^2$) and strong correlations with CT-derived reference standard TLV ($n_{Public-Synthetic}$=1,191, r=0.99, P<0.001; $n_{KRI-Synthetic}$=72, r=0.97, P<0.001; $n_{KRI-Real}$=72, r=0.91, P<0.001). The Luna16 test data demonstrated the highest performance, with the lowest mean squared error (MSE = 0.09 $L^2$) and strongest correlation (r = 0.99, P <0.001) for TLV estimation. Conclusion: The U-Net-generated pixel-level lung thickness maps successfully estimated TLV for both synthetic and real radiographs.
Detecting model drift using polynomial relations
Authors: Eliran Roffe, Samuel Ackerman, Orna Raz, Eitan Farchi
Date: 2021-10-24 00:00:00
Tasks: model, Relation

Machine learning models serve critical functions, such as classifying loan applicants as good or bad risks. Each model is trained under the assumption that the data used in training and in the field come from the same underlying unknown distribution. Often, this assumption is broken in practice. It is desirable to identify when this occurs, to minimize the impact on model performance. We suggest a new approach to detecting change in the data distribution by identifying polynomial relations between the data features. We measure the strength of each identified relation using its R-square value. A strong polynomial relation captures a significant trait of the data which should remain stable if the data distribution does not change. We thus use a set of learned strong polynomial relations to identify drift. For a set of polynomial relations that are stronger than a given threshold, we calculate the amount of drift observed for that relation. The amount of drift is measured by calculating the Bayes Factor for the polynomial relation likelihood of the baseline data versus field data. We empirically validate the approach by simulating a range of changes, and identify drift using the Bayes Factor of the polynomial relation likelihood change.
Scaling Neural Program Synthesis with Distribution-based Search
Authors: Nathanaël Fijalkow, Guillaume Lagarde, Théo Matricon, Kevin Ellis, Pierre Ohlmann, Akarsh Potta
Date: 2021-10-24 00:00:00
Tasks: Program Synthesis

We consider the problem of automatically constructing computer programs from input-output examples. We investigate how to augment probabilistic and neural program synthesis methods with new search algorithms, proposing a framework called distribution-based search. Within this framework, we introduce two new search algorithms: Heap Search, an enumerative method, and SQRT Sampling, a probabilistic method. We prove certain optimality guarantees for both methods, show how they integrate with probabilistic and neural techniques, and demonstrate how they can operate at scale across parallel compute environments. Collectively these findings offer theoretical and applied studies of search algorithms for program synthesis that integrate with recent developments in machine-learned program synthesizers.
False Correlation Reduction for Offline Reinforcement Learning
Authors: Zhihong Deng, Zuyue Fu, Lingxiao Wang, Zhuoran Yang, Chenjia Bai, Tianyi Zhou, Zhaoran Wang, Jing Jiang
Date: 2021-10-24 00:00:00
Tasks: D4RL, Decision Making, Offline RL, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Offline reinforcement learning (RL) harnesses the power of massive datasets for resolving sequential decision problems. Most existing papers only discuss defending against out-of-distribution (OOD) actions while we investigate a broader issue, the false correlations between epistemic uncertainty and decision-making, an essential factor that causes suboptimality. In this paper, we propose falSe COrrelation REduction (SCORE) for offline RL, a practically effective and theoretically provable algorithm. We empirically show that SCORE achieves the SoTA performance with 3.1x acceleration on various tasks in a standard benchmark (D4RL). The proposed algorithm introduces an annealing behavior cloning regularizer to help produce a high-quality estimation of uncertainty which is critical for eliminating false correlations from suboptimality. Theoretically, we justify the rationality of the proposed method and prove its convergence to the optimal policy with a sublinear rate under mild assumptions.
De Novo Molecular Generation with Stacked Adversarial Model
Authors: Yuansan Liu, James Bailey
Date: 2021-10-24 00:00:00
Tasks: Drug Design, valid

Generating novel drug molecules with desired biological properties is a time consuming and complex task. Conditional generative adversarial models have recently been proposed as promising approaches for de novo drug design. In this paper, we propose a new generative model which extends an existing adversarial autoencoder (AAE) based model by stacking two models together. Our stacked approach generates more valid molecules, as well as molecules that are more similar to known drugs. We break down this challenging task into two sub-problems. A first stage model to learn primitive features from the molecules and gene expression data. A second stage model then takes these features to learn properties of the molecules and refine more valid molecules. Experiments and comparison to baseline methods on the LINCS L1000 dataset demonstrate that our proposed model has promising performance for molecular generation.
Bangla Image Caption Generation through CNN-Transformer based Encoder-Decoder Network
Authors: Md Aminul Haque Palash, MD Abdullah Al Nasim, Sourav Saha, Faria Afrin, Raisa Mallik, Sathishkumar Samiappan
Date: 2021-10-24 00:00:00
Tasks: Caption Generation, Decoder, Image Captioning

Automatic Image Captioning is the never-ending effort of creating syntactically and validating the accuracy of textual descriptions of an image in natural language with context. The encoder-decoder structure used throughout existing Bengali Image Captioning (BIC) research utilized abstract image feature vectors as the encoder's input. We propose a novel transformer-based architecture with an attention mechanism with a pre-trained ResNet-101 model image encoder for feature extraction from images. Experiments demonstrate that the language decoder in our technique captures fine-grained information in the caption and, then paired with image features, produces accurate and diverse captions on the BanglaLekhaImageCaptions dataset. Our approach outperforms all existing Bengali Image Captioning work and sets a new benchmark by scoring 0.694 on BLEU-1, 0.630 on BLEU-2, 0.582 on BLEU-3, and 0.337 on METEOR.