pwc-archive/papers-with-abstracts huggingface dataset:
Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulness
Authors: Mathieu Seurin, Florian Strub, Philippe Preux, Olivier Pietquin
Date: 2021-05-20 00:00:00

Sparse rewards are double-edged training signals in reinforcement learning: easy to design but hard to optimize. Intrinsic motivation guidances have thus been developed toward alleviating the resulting exploration problem. They usually incentivize agents to look for new states through novelty signals. Yet, such methods encourage exhaustive exploration of the state space rather than focusing on the environment's salient interaction opportunities. We propose a new exploration method, called Don't Do What Doesn't Matter (DoWhaM), shifting the emphasis from state novelty to state with relevant actions. While most actions consistently change the state when used, \textit{e.g.} moving the agent, some actions are only effective in specific states, \textit{e.g.}, \emph{opening} a door, \emph{grabbing} an object. DoWhaM detects and rewards actions that seldom affect the environment. We evaluate DoWhaM on the procedurally-generated environment MiniGrid, against state-of-the-art methods and show that DoWhaM greatly reduces sample complexity.
Multi-group Agnostic PAC Learnability
Authors: Guy N Rothblum, Gal Yona
Date: 2021-05-20 00:00:00
Tasks: Fairness, PAC learning

An agnostic PAC learning algorithm finds a predictor that is competitive with the best predictor in a benchmark hypothesis class, where competitiveness is measured with respect to a given loss function. However, its predictions might be quite sub-optimal for structured subgroups of individuals, such as protected demographic groups. Motivated by such fairness concerns, we study "multi-group agnostic PAC learnability": fixing a measure of loss, a benchmark class $\H$ and a (potentially) rich collection of subgroups $\G$, the objective is to learn a single predictor such that the loss experienced by every group $g \in \G$ is not much larger than the best possible loss for this group within $\H$. Under natural conditions, we provide a characterization of the loss functions for which such a predictor is guaranteed to exist. For any such loss function we construct a learning algorithm whose sample complexity is logarithmic in the size of the collection $\G$. Our results unify and extend previous positive and negative results from the multi-group fairness literature, which applied for specific loss functions.
XGBoost energy consumption prediction based on multi-system data HVAC
Authors: YunLong Li, Yiming Peng, Dengzheng Zhang, Yingan Mai, Zhengrong Ruan
Date: 2021-05-20 00:00:00
Tasks: Prediction

The energy consumption of the HVAC system accounts for a significant portion of the energy consumption of the public building system, and using an efficient energy consumption prediction model can assist it in carrying out effective energy-saving transformation. Unlike the traditional energy consumption prediction model, this paper extracts features from large data sets using XGBoost, trains them separately to obtain multiple models, then fuses them with LightGBM's independent prediction results using MAE, infers energy consumption related variables, and successfully applies this model to the self-developed Internet of Things platform.
Temporal convolutional networks predict dynamic oxygen uptake response from wearable sensors across exercise intensities
Authors: Robert Amelard, Eric T Hedge, Richard L Hughson
Date: 2021-05-20 00:00:00

Oxygen consumption (VO$_2$) provides established clinical and physiological indicators of cardiorespiratory function and exercise capacity. However, VO$_2$ monitoring is largely limited to specialized laboratory settings, making its widespread monitoring elusive. Here, we investigate temporal prediction of VO$_2$ from wearable sensors during cycle ergometer exercise using a temporal convolutional network (TCN). Cardiorespiratory signals were acquired from a smart shirt with integrated textile sensors alongside ground-truth VO$_2$ from a metabolic system on twenty-two young healthy adults. Participants performed one ramp-incremental and three pseudorandom binary sequence exercise protocols to assess a range of VO$_2$ dynamics. A TCN model was developed using causal convolutions across an effective history length to model the time-dependent nature of VO$_2$. Optimal history length was determined through minimum validation loss across hyperparameter values. The best performing model encoded 218 s history length (TCN-VO$_2$ A), with 187 s, 97 s, and 76 s yielding less than 3% deviation from the optimal validation loss. TCN-VO$_2$ A showed strong prediction accuracy (mean, 95% CI) across all exercise intensities (-22 ml.min$^{-1}$, [-262, 218]), spanning transitions from low-moderate (-23 ml.min$^{-1}$, [-250, 204]), low-high (14 ml.min$^{-1}$, [-252, 280]), ventilatory threshold-high (-49 ml.min$^{-1}$, [-274, 176]), and maximal (-32 ml.min$^{-1}$, [-261, 197]) exercise. Second-by-second classification of physical activity across 16090 s of predicted VO$_2$ was able to discern between vigorous, moderate, and light activity with high accuracy (94.1%). This system enables quantitative aerobic activity monitoring in non-laboratory settings across a range of exercise intensities using wearable sensors for monitoring exercise prescription adherence and personal fitness.
Measuring Model Fairness under Noisy Covariates: A Theoretical Perspective
Authors: Flavien Prost, Pranjal Awasthi, Nick Blumm, Aditee Kumthekar, Trevor Potter, Li Wei, Xuezhi Wang, Ed H. Chi, Jilin Chen, Alex Beutel
Date: 2021-05-20 00:00:00
Tasks: Fairness

In this work we study the problem of measuring the fairness of a machine learning model under noisy information. Focusing on group fairness metrics, we investigate the particular but common situation when the evaluation requires controlling for the confounding effect of covariate variables. In a practical setting, we might not be able to jointly observe the covariate and group information, and a standard workaround is to then use proxies for one or more of these variables. Prior works have demonstrated the challenges with using a proxy for sensitive attributes, and strong independence assumptions are needed to provide guarantees on the accuracy of the noisy estimates. In contrast, in this work we study using a proxy for the covariate variable and present a theoretical analysis that aims to characterize weaker conditions under which accurate fairness evaluation is possible. Furthermore, our theory identifies potential sources of errors and decouples them into two interpretable parts $\gamma$ and $\epsilon$. The first part $\gamma$ depends solely on the performance of the proxy such as precision and recall, whereas the second part $\epsilon$ captures correlations between all the variables of interest. We show that in many scenarios the error in the estimates is dominated by $\gamma$ via a linear dependence, whereas the dependence on the correlations $\epsilon$ only constitutes a lower order term. As a result we expand the understanding of scenarios where measuring model fairness via proxies can be an effective approach. Finally, we compare, via simulations, the theoretical upper-bounds to the distribution of simulated estimation errors and show that assuming some structure on the data, even weak, is key to significantly improve both theoretical guarantees and empirical results.
Data-driven discovery of interpretable causal relations for deep learning material laws with uncertainty propagation
Authors: Xiao Sun, Bahador Bahmani, Nikolaos N. Vlassis, WaiChing Sun, Yanxun Xu
Date: 2021-05-20 00:00:00
Tasks: Causal Discovery, Uncertainty Quantification

This paper presents a computational framework that generates ensemble predictive mechanics models with uncertainty quantification (UQ). We first develop a causal discovery algorithm to infer causal relations among time-history data measured during each representative volume element (RVE) simulation through a directed acyclic graph (DAG). With multiple plausible sets of causal relationships estimated from multiple RVE simulations, the predictions are propagated in the derived causal graph while using a deep neural network equipped with dropout layers as a Bayesian approximation for uncertainty quantification. We select two representative numerical examples (traction-separation laws for frictional interfaces, elastoplasticity models for granular assembles) to examine the accuracy and robustness of the proposed causal discovery method for the common material law predictions in civil engineering applications.
Wide & Deep neural network model for patch aggregation in CNN-based prostate cancer detection systems
Authors: Lourdes Duran-Lopez, Juan P. Dominguez-Morales, Daniel Gutierrez-Galan, Antonio Rios-Navarro, Angel Jimenez-Fernandez, Saturnino Vicente-Diaz, Alejandro Linares-Barranco
Date: 2021-05-20 00:00:00
Tasks: Medical Image Analysis, whole slide images

Prostate cancer (PCa) is one of the most commonly diagnosed cancer and one of the leading causes of death among men, with almost 1.41 million new cases and around 375,000 deaths in 2020. Artificial Intelligence algorithms have had a huge impact in medical image analysis, including digital histopathology, where Convolutional Neural Networks (CNNs) are used to provide a fast and accurate diagnosis, supporting experts in this task. To perform an automatic diagnosis, prostate tissue samples are first digitized into gigapixel-resolution whole-slide images. Due to the size of these images, neural networks cannot use them as input and, therefore, small subimages called patches are extracted and predicted, obtaining a patch-level classification. In this work, a novel patch aggregation method based on a custom Wide & Deep neural network model is presented, which performs a slide-level classification using the patch-level classes obtained from a CNN. The malignant tissue ratio, a 10-bin malignant probability histogram, the least squares regression line of the histogram, and the number of malignant connected components are used by the proposed model to perform the classification. An accuracy of 94.24% and a sensitivity of 98.87% were achieved, proving that the proposed system could aid pathologists by speeding up the screening process and, thus, contribute to the fight against PCa.
On planetary systems as ordered sequences
Authors: Emily Sandford, David Kipping, Michael Collins
Date: 2021-05-20 00:00:00
Tasks: Part-Of-Speech Tagging, Selection bias, Unsupervised Part-Of-Speech Tagging

A planetary system consists of a host star and one or more planets, arranged into a particular configuration. Here, we consider what information belongs to the configuration, or ordering, of 4286 Kepler planets in their 3277 planetary systems. First, we train a neural network model to predict the radius and period of a planet based on the properties of its host star and the radii and period of its neighbors. The mean absolute error of the predictions of the trained model is a factor of 2.1 better than the MAE of the predictions of a naive model which draws randomly from dynamically allowable periods and radii. Second, we adapt a model used for unsupervised part-of-speech tagging in computational linguistics to investigate whether planets or planetary systems fall into natural categories with physically interpretable "grammatical rules." The model identifies two robust groups of planetary systems: (1) compact multi-planet systems and (2) systems around giant stars ($\log{g} \lesssim 4.0$), although the latter group is strongly sculpted by the selection bias of the transit method. These results reinforce the idea that planetary systems are not random sequences -- instead, as a population, they contain predictable patterns that can provide insight into the formation and evolution of planetary systems.
Definite Non-Ancestral Relations and Structure Learning
Authors: Wenyu Chen, Mathias Drton, Ali Shojaie
Date: 2021-05-20 00:00:00

In causal graphical models based on directed acyclic graphs (DAGs), directed paths represent causal pathways between the corresponding variables. The variable at the beginning of such a path is referred to as an ancestor of the variable at the end of the path. Ancestral relations between variables play an important role in causal modeling. In existing literature on structure learning, these relations are usually deduced from learned structures and used for orienting edges or formulating constraints of the space of possible DAGs. However, they are usually not posed as immediate target of inference. In this work we investigate the graphical characterization of ancestral relations via CPDAGs and d-separation relations. We propose a framework that can learn definite non-ancestral relations without first learning the skeleton. This frame-work yields structural information that can be used in both score- and constraint-based algorithms to learn causal DAGs more efficiently.
Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Mining
Authors: Ivana Kvapilikova, Mikel Artetxe, Gorka Labaka, Eneko Agirre, Ondřej Bojar
Date: 2021-05-21 00:00:00
Proceeding: ACL 2020 6
Tasks: Language Modeling, Language Modelling, Machine Translation, Parallel Corpus Mining, Sentence, Sentence Embeddings, Translation, Unsupervised Machine Translation

Existing models of multilingual sentence embeddings require large parallel data resources which are not available for low-resource languages. We propose a novel unsupervised method to derive multilingual sentence embeddings relying only on monolingual data. We first produce a synthetic parallel corpus using unsupervised machine translation, and use it to fine-tune a pretrained cross-lingual masked language model (XLM) to derive the multilingual sentence representations. The quality of the representations is evaluated on two parallel corpus mining tasks with improvements of up to 22 F1 points over vanilla XLM. In addition, we observe that a single synthetic bilingual corpus is able to improve results for other language pairs.
Functionals in the Clouds: An abstract architecture of serverless Cloud-Native Apps
Authors: Stanislaw Ambroszkiewicz, Waldemar Bartyna, Stanislaw Bylka
Date: 2021-05-21 00:00:00

Cloud Native Application CNApp (as a distributed system) is a collection of independent components (micro-services) interacting via communication protocols. This gives rise to present an abstract architecture of CNApp as dynamically re-configurable acyclic directed multi graph where vertices are microservices, and edges are the protocols. Generic mechanisms for such reconfigurations evidently correspond to higher-level functions (functionals). This implies also internal abstract architecture of microservice as a collection of event-triggered serverless functions (including functions implementing the protocols) that are dynamically composed into event-dependent data-flow graphs. Again, generic mechanisms for such compositions correspond to calculus of functionals and relations.
Language Understanding for Field and Service Robots in a Priori Unknown Environments
Authors: Matthew R. Walter, Siddharth Patki, Andrea F. Daniele, Ethan Fahnestock, Felix Duvallet, Sachithra Hemachandra, Jean Oh, Anthony Stentz, Nicholas Roy, Thomas M. Howard
Date: 2021-05-21 00:00:00
Tasks: Imitation Learning, Natural Language Understanding

Contemporary approaches to perception, planning, estimation, and control have allowed robots to operate robustly as our remote surrogates in uncertain, unstructured environments. This progress now creates an opportunity for robots to operate not only in isolation, but also with and alongside humans in our complex environments. Realizing this opportunity requires an efficient and flexible medium through which humans can communicate with collaborative robots. Natural language provides one such medium, and through significant progress in statistical methods for natural-language understanding, robots are now able to interpret a diverse array of free-form commands. However, most contemporary approaches require a detailed, prior spatial-semantic map of the robot's environment that models the space of possible referents of an utterance. Consequently, these methods fail when robots are deployed in new, previously unknown, or partially-observed environments, particularly when mental models of the environment differ between the human operator and the robot. This paper provides a comprehensive description of a novel learning framework that allows field and service robots to interpret and correctly execute natural-language instructions in a priori unknown, unstructured environments. Integral to our approach is its use of language as a "sensor" -- inferring spatial, topological, and semantic information implicit in the utterance and then exploiting this information to learn a distribution over a latent environment model. We incorporate this distribution in a probabilistic, language grounding model and infer a distribution over a symbolic representation of the robot's action space. We use imitation learning to identify a belief-space policy that reasons over the environment and behavior distributions. We evaluate our framework through a variety navigation and mobile-manipulation experiments.
Learning from My Friends: Few-Shot Personalized Conversation Systems via Social Networks
Authors: Zhiliang Tian, Wei Bi, Zihan Zhang, Dongkyu Lee, Yiping Song, Nevin L. Zhang
Date: 2021-05-21 00:00:00
Tasks: Diversity, Meta-Learning

Personalized conversation models (PCMs) generate responses according to speaker preferences. Existing personalized conversation tasks typically require models to extract speaker preferences from user descriptions or their conversation histories, which are scarce for newcomers and inactive users. In this paper, we propose a few-shot personalized conversation task with an auxiliary social network. The task requires models to generate personalized responses for a speaker given a few conversations from the speaker and a social network. Existing methods are mainly designed to incorporate descriptions or conversation histories. Those methods can hardly model speakers with so few conversations or connections between speakers. To better cater for newcomers with few resources, we propose a personalized conversation model (PCM) that learns to adapt to new speakers as well as enabling new speakers to learn from resource-rich speakers. Particularly, based on a meta-learning based PCM, we propose a task aggregator (TA) to collect other speakers' information from the social network. The TA provides prior knowledge of the new speaker in its meta-learning. Experimental results show our methods outperform all baselines in appropriateness, diversity, and consistency with speakers.
Measuring the impact of spammers on e-mail and Twitter networks
Authors: A. Fronzetti Colladon, P. A. Gloor
Date: 2021-05-21 00:00:00
Tasks: Decision Making

This paper investigates the research question if senders of large amounts of irrelevant or unsolicited information - commonly called "spammers" - distort the network structure of social networks. Two large social networks are analyzed, the first extracted from the Twitter discourse about a big telecommunication company, and the second obtained from three years of email communication of 200 managers working for a large multinational company. This work compares network robustness and the stability of centrality and interaction metrics, as well as the use of language, after removing spammers and the most and least connected nodes. The results show that spammers do not significantly alter the structure of the information-carrying network, for most of the social indicators. The authors additionally investigate the correlation between e-mail subject line and content by tracking language sentiment, emotionality, and complexity, addressing the cases where collecting email bodies is not permitted for privacy reasons. The findings extend the research about robustness and stability of social networks metrics, after the application of graph simplification strategies. The results have practical implication for network analysts and for those company managers who rely on network analytics (applied to company emails and social media data) to support their decision-making processes.
Semantic Representation for Dialogue Modeling
Authors: Xuefeng Bai, Yulong Chen, Linfeng Song, Yue Zhang
Date: 2021-05-21 00:00:00
Proceeding: ACL 2021 5
Tasks: Abstract Meaning Representation, Dialog Relation Extraction, Dialogue Understanding, Response Generation, Sentence

Although neural models have achieved competitive results in dialogue systems, they have shown limited ability in representing core semantics, such as ignoring important entities. To this end, we exploit Abstract Meaning Representation (AMR) to help dialogue modeling. Compared with the textual input, AMR explicitly provides core semantic knowledge and reduces data sparsity. We develop an algorithm to construct dialogue-level AMR graphs from sentence-level AMRs and explore two ways to incorporate AMRs into dialogue systems. Experimental results on both dialogue understanding and response generation tasks show the superiority of our model. To our knowledge, we are the first to leverage a formal semantic representation into neural dialogue modeling.
Should We Trust This Summary? Bayesian Abstractive Summarization to The Rescue
Authors: Alexios Gidiotis, Grigorios Tsoumakas
Date: 2021-05-21 00:00:00
Proceeding: Findings (ACL) 2022 5
Tasks: Abstractive Text Summarization, Active Learning, Bayesian Inference

We explore the notion of uncertainty in the context of modern abstractive summarization models, using the tools of Bayesian Deep Learning. Our approach approximates Bayesian inference by first extending state-of-the-art summarization models with Monte Carlo dropout and then using them to perform multiple stochastic forward passes. Based on Bayesian inference we are able to effectively quantify uncertainty at prediction time. Having a reliable uncertainty measure, we can improve the experience of the end user by filtering out generated summaries of high uncertainty. Furthermore, uncertainty estimation could be used as a criterion for selecting samples for annotation, and can be paired nicely with active learning and human-in-the-loop approaches. Finally, Bayesian inference enables us to find a Bayesian summary which performs better than a deterministic one and is more robust to uncertainty. In practice, we show that our Variational Bayesian equivalents of BART and PEGASUS can outperform their deterministic counterparts on multiple benchmark datasets.
Training Bi-Encoders for Word Sense Disambiguation
Authors: Harsh Kohli
Date: 2021-05-21 00:00:00
Tasks: Word Sense Disambiguation

Modern transformer-based neural architectures yield impressive results in nearly every NLP task and Word Sense Disambiguation, the problem of discerning the correct sense of a word in a given context, is no exception. State-of-the-art approaches in WSD today leverage lexical information along with pre-trained embeddings from these models to achieve results comparable to human inter-annotator agreement on standard evaluation benchmarks. In the same vein, we experiment with several strategies to optimize bi-encoders for this specific task and propose alternative methods of presenting lexical information to our model. Through our multi-stage pre-training and fine-tuning pipeline we further the state of the art in Word Sense Disambiguation.