pwc-archive/papers-with-abstracts huggingface dataset:
Budgeted Online Selection of Candidate IoT Clients to Participate in Federated Learning
Authors: Ihab Mohammed, Shadha Tabatabai, Ala Al-Fuqaha, Faissal El Bouanani, Junaid Qadir, Basheer Qolomany, Mohsen Guizani
Date: 2020-11-16 00:00:00
Tasks: Federated Learning

Machine Learning (ML), and Deep Learning (DL) in particular, play a vital role in providing smart services to the industry. These techniques however suffer from privacy and security concerns since data is collected from clients and then stored and processed at a central location. Federated Learning (FL), an architecture in which model parameters are exchanged instead of client data, has been proposed as a solution to these concerns. Nevertheless, FL trains a global model by communicating with clients over communication rounds, which introduces more traffic on the network and increases the convergence time to the target accuracy. In this work, we solve the problem of optimizing accuracy in stateful FL with a budgeted number of candidate clients by selecting the best candidate clients in terms of test accuracy to participate in the training process. Next, we propose an online stateful FL heuristic to find the best candidate clients. Additionally, we propose an IoT client alarm application that utilizes the proposed heuristic in training a stateful FL global model based on IoT device type classification to alert clients about unauthorized IoT devices in their environment. To test the efficiency of the proposed online heuristic, we conduct several experiments using a real dataset and compare the results against state-of-the-art algorithms. Our results indicate that the proposed heuristic outperforms the online random algorithm with up to 27% gain in accuracy. Additionally, the performance of the proposed online heuristic is comparable to the performance of the best offline algorithm.
A Deep Learning Approach to Predict Hamburg Rutting Curve
Authors: Hamed Majidifard, Behnam Jahangiri, Punyaslok Rath, Amir H. Alavi, William G. Buttlar
Date: 2020-11-12 00:00:00
Tasks: Deep Learning

Rutting continues to be one of the principal distresses in asphalt pavements worldwide. This type of distress is caused by permanent deformation and shear failure of the asphalt mix under the repetition of heavy loads. The Hamburg wheel tracking test (HWTT) is a widely used testing procedure designed to accelerate, and to simulate the rutting phenomena in the laboratory. Rut depth, as one of the outputs of the HWTT, is dependent on a number of parameters related to mix design and testing conditions. This study introduces a new model for predicting the rutting depth of asphalt mixtures using a deep learning technique - the convolution neural network (CNN). A database containing a comprehensive collection of HWTT results was used to develop a CNN-based machine learning prediction model. The database includes 10,000 rutting depth data points measured across a large variety of asphalt mixtures. The model has been formulated in terms of known influencing mixture variables such as asphalt binder high temperature performance grade, mixture type, aggregate size, aggregate gradation, asphalt content, total asphalt binder recycling content, and testing parameters, including testing temperature and number of wheel passes. A rigorous validation process was used to assess the accuracy of the model to predict total rut depth and the HWTT rutting curve. A sensitivity analysis is presented, which evaluates the effect of the investigated variables on rutting depth predictions by the CNN model. The model can be used as a tool to estimate the rut depth in asphalt mixtures when laboratory testing is not feasible, or for cost saving, pre-design trials.
Low-latency Federated Learning and Blockchain for Edge Association in Digital Twin empowered 6G Networks
Authors: Yunlong Lu, Xiaohong Huang, Ke Zhang, Sabita Maharjan, Yan Zhang
Date: 2020-11-17 00:00:00
Tasks: Federated Learning, Multi-agent Reinforcement Learning

Emerging technologies such as digital twins and 6th Generation mobile networks (6G) have accelerated the realization of edge intelligence in Industrial Internet of Things (IIoT). The integration of digital twin and 6G bridges the physical system with digital space and enables robust instant wireless connectivity. With increasing concerns on data privacy, federated learning has been regarded as a promising solution for deploying distributed data processing and learning in wireless networks. However, unreliable communication channels, limited resources, and lack of trust among users, hinder the effective application of federated learning in IIoT. In this paper, we introduce the Digital Twin Wireless Networks (DTWN) by incorporating digital twins into wireless networks, to migrate real-time data processing and computation to the edge plane. Then, we propose a blockchain empowered federated learning framework running in the DTWN for collaborative computing, which improves the reliability and security of the system, and enhances data privacy. Moreover, to balance the learning accuracy and time cost of the proposed scheme, we formulate an optimization problem for edge association by jointly considering digital twin association, training data batch size, and bandwidth allocation. We exploit multi-agent reinforcement learning to find an optimal solution to the problem. Numerical results on real-world dataset show that the proposed scheme yields improved efficiency and reduced cost compared to benchmark learning method.
LOss-Based SensiTivity rEgulaRization: towards deep sparse neural networks
Authors: Enzo Tartaglione, Andrea Bragagnolo, Attilio Fiandrotti, Marco Grangetto
Date: 2020-11-16 00:00:00
Tasks: Sensitivity

LOBSTER (LOss-Based SensiTivity rEgulaRization) is a method for training neural networks having a sparse topology. Let the sensitivity of a network parameter be the variation of the loss function with respect to the variation of the parameter. Parameters with low sensitivity, i.e. having little impact on the loss when perturbed, are shrunk and then pruned to sparsify the network. Our method allows to train a network from scratch, i.e. without preliminary learning or rewinding. Experiments on multiple architectures and datasets show competitive compression ratios with minimal computational overhead.
Towards Learning Controllable Representations of Physical Systems
Authors: Kevin Haninger, Raul Vicente Garcia, Joerg Krueger
Date: 2020-11-16 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Learned representations of dynamical systems reduce dimensionality, potentially supporting downstream reinforcement learning (RL). However, no established methods predict a representation's suitability for control and evaluation is largely done via downstream RL performance, slowing representation design. Towards a principled evaluation of representations for control, we consider the relationship between the true state and the corresponding representations, proposing that ideally each representation corresponds to a unique true state. This motivates two metrics: temporal smoothness and high mutual information between true state/representation. These metrics are related to established representation objectives, and studied on Lagrangian systems where true state, information requirements, and statistical properties of the state can be formalized for a broad class of systems. These metrics are shown to predict reinforcement learning performance in a simulated peg-in-hole task when comparing variants of autoencoder-based representations.
Multi-objective semi-supervised clustering to identify health service patterns for injured patients
Authors: Hadi Akbarzadeh Khorshidi, Uwe Aickelin, Gholamreza Haffari, Behrooz Hassani-Mahmooei
Date: 2020-11-16 00:00:00
Tasks: Clustering

This study develops a pattern recognition method that identifies patterns based on their similarity and their association with the outcome of interest. The practical purpose of developing this pattern recognition method is to group patients, who are injured in transport accidents, in the early stages post-injury. This grouping is based on distinctive patterns in health service use within the first week post-injury. The groups also provide predictive information towards the total cost of medication process. As a result, the group of patients who have undesirable outcomes are identified as early as possible based health service use patterns.
Imputation techniques on missing values in breast cancer treatment and fertility data
Authors: Xuetong Wu, Hadi Akbarzadeh Khorshidi, Uwe Aickelin, Zobaida Edib, Michelle Peate
Date: 2020-11-16 00:00:00
Tasks: Imputation, Missing Values

Clinical decision support using data mining techniques offers more intelligent way to reduce the decision error in the last few years. However, clinical datasets often suffer from high missingness, which adversely impacts the quality of modelling if handled improperly. Imputing missing values provides an opportunity to resolve the issue. Conventional imputation methods adopt simple statistical analysis, such as mean imputation or discarding missing cases, which have many limitations and thus degrade the performance of learning. This study examines a series of machine learning based imputation methods and suggests an efficient approach to in preparing a good quality breast cancer (BC) dataset, to find the relationship between BC treatment and chemotherapy-related amenorrhoea, where the performance is evaluated with the accuracy of the prediction.
EventDetectR -- An Open-Source Event Detection System
Authors: Sowmya Chandrasekaran, Margarita Rebolledo, Thomas Bartz-Beielstein
Date: 2020-11-16 00:00:00
Tasks: Event Detection

EventDetectR: An efficient Event Detection System (EDS) capable of detecting unexpected water quality conditions. This approach uses multiple algorithms to model the relationship between various multivariate water quality signals. Then the residuals of the models were utilized in constructing the event detection algorithm, which provides a continuous measure of the probability of an event at every time step. The proposed framework was tested for water contamination events with industrial data from automated water quality sensors. The results showed that the framework is reliable with better performance and is highly suitable for event detection.
Passive detection of behavioral shifts for suicide attempt prevention
Authors: Pablo Moreno-Muñoz, Lorena Romero-Medrano, Ángela Moreno, Jesús Herrera-López, Enrique Baca-García, Antonio Artés-Rodríguez
Date: 2020-11-14 00:00:00

More than one million people commit suicide every year worldwide. The costs of daily cares, social stigma and treatment issues are still hard barriers to overcome in mental health. Most symptoms of mental disorders are related to the behavioral state of a patient, such as the mobility or social activity. Mobile-based technologies allow the passive collection of patients data, which supplements conventional assessments that rely on biased questionnaires and occasional medical appointments. In this work, we present a non-invasive machine learning (ML) model to detect behavioral shifts in psychiatric patients from unobtrusive data collected by a smartphone app. Our clinically validated results shed light on the idea of an early detection mobile tool for the task of suicide attempt prevention.
An experiment on the mechanisms of racial bias in ML-based credit scoring in Brazil
Authors: Ramon Vilarino, Renato Vicente
Date: 2020-11-11 00:00:00
Tasks: BIG-bench Machine Learning, counterfactual

We dissect an experimental credit scoring model developed with real data and demonstrate - without access to protected attributes - how the use of location information introduces racial bias. We analyze the tree gradient boosting model with the aid of a game-theoretic inspired machine learning explainability technique, counterfactual experiments and Brazilian census data. By exposing algorithmic racial bias explaining the trained machine learning model inner mechanisms, this experiment comprises an interesting artifact to aid the endeavor of theoretical understanding of the emergence of racial bias in machine learning systems. Without access to individuals' racial categories, we show how classification parity measures using geographically defined groups could carry information about model racial bias. The experiment testifies to the need for methods and language that do not presuppose access to protected attributes when auditing ML models, the importance of considering regional specifics when addressing racial issues, and the central role of census data in the AI research community. To the best of our knowledge, this is the first documented case of algorithmic racial bias in ML-based credit scoring in Brazil, the country with the second largest Black population in the world.
Unsupervised stance detection for arguments from consequences
Authors: Jonathan Kobbe, Ioana Hulpu{\textcommabelow{s}}, Heiner Stuckenschmidt
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Stance Detection

Social media platforms have become an essential venue for online deliberation where users discuss arguments, debate, and form opinions. In this paper, we propose an unsupervised method to detect the stance of argumentative claims with respect to a topic. Most related work focuses on topic-specific supervised models that need to be trained for every emergent debate topic. To address this limitation, we propose a topic independent approach that focuses on a frequently encountered class of arguments, specifically, on arguments from consequences. We do this by extracting the effects that claims refer to, and proposing a means for inferring if the effect is a good or bad consequence. Our experiments provide promising results that are comparable to, and in particular regards even outperform BERT. Furthermore, we publish a novel dataset of arguments relating to consequences, annotated with Amazon Mechanical Turk.
Statistical Power and Translationese in Machine Translation Evaluation
Authors: Yvette Graham, Barry Haddow, Philipp Koehn
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Machine Translation, Translation

The term translationese has been used to describe features of translated text, and in this paper, we provide detailed analysis of potential adverse effects of translationese on machine translation evaluation. Our analysis shows differences in conclusions drawn from evaluations that include translationese in test data compared to experiments that tested only with text originally composed in that language. For this reason we recommend that reverse-created test data be omitted from future machine translation test sets. In addition, we provide a re-evaluation of a past machine translation evaluation claiming human-parity of MT. One important issue not previously considered is statistical power of significance tests applied to comparison of human and machine translation. Since the very aim of past evaluations was investigation of ties between human and MT systems, power analysis is of particular importance, to avoid, for example, claims of human parity simply corresponding to Type II error resulting from the application of a low powered test. We provide detailed analysis of tests used in such evaluations to provide an indication of a suitable minimum sample size for future studies.
KERMIT: Complementing Transformer Architectures with Encoders of Explicit Syntactic Interpretations
Authors: Fabio Massimo Zanzotto, Andrea Santilli, Leonardo Ranaldi, Dario Onorati, Pierfrancesco Tommasino, Francesca Fallucchi
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Representation Learning, Sentence, Syntax Representation, Text Classification

Syntactic parsers have dominated natural language understanding for decades. Yet, their syntactic interpretations are losing centrality in downstream tasks due to the success of large-scale textual representation learners. In this paper, we propose KERMIT (Kernel-inspired Encoder with Recursive Mechanism for Interpretable Trees) to embed symbolic syntactic parse trees into artificial neural networks and to visualize how syntax is used in inference. We experimented with KERMIT paired with two state-of-the-art transformer-based universal sentence encoders (BERT and XLNet) and we showed that KERMIT can indeed boost their performance by effectively embedding human-coded universal syntactic representations in neural networks
A Spectral Method for Unsupervised Multi-Document Summarization
Authors: Kexiang Wang, Baobao Chang, Zhifang Sui
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Combinatorial Optimization, Document Summarization, Multi-Document Summarization

Multi-document summarization (MDS) aims at producing a good-quality summary for several related documents. In this paper, we propose a spectral-based hypothesis, which states that the goodness of summary candidate is closely linked to its so-called spectral impact. Here spectral impact considers the perturbation to the dominant eigenvalue of affinity matrix when dropping the summary candidate from the document cluster. The hypothesis is validated by three theoretical perspectives: semantic scaling, propagation dynamics and matrix perturbation. According to the hypothesis, we formulate the MDS task as the combinatorial optimization of spectral impact and propose an accelerated greedy solution based on a surrogate of spectral impact. The evaluation results on various datasets demonstrate: (1) The performance of the summary candidate is positively correlated with its spectral impact, which accords with our hypothesis; (2) Our spectral-based method has a competitive result as compared to state-of-the-art MDS systems.
Friendly Topic Assistant for Transformer Based Abstractive Summarization
Authors: Zhengjue Wang, Zhibin Duan, Hao Zhang, Chaojie Wang, Long Tian, Bo Chen, Mingyuan Zhou
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Abstractive Text Summarization, Document Summarization, document understanding, Topic Models

Abstractive document summarization is a comprehensive task including document understanding and summary generation, in which area Transformer-based models have achieved the state-of-the-art performance. Compared with Transformers, topic models are better at learning explicit document semantics, and hence could be integrated into Transformers to further boost their performance. To this end, we rearrange and explore the semantics learned by a topic model, and then propose a topic assistant (TA) including three modules. TA is compatible with various Transformer-based models and user-friendly since i) TA is a plug-and-play model that does not break any structure of the original Transformer network, making users easily fine-tune Transformer+TA based on a well pre-trained model; ii) TA only introduces a small number of extra parameters. Experimental results on three datasets demonstrate that TA is able to improve the performance of several Transformer-based models.
Condolence and Empathy in Online Communities
Authors: Naitian Zhou, David Jurgens
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11

Offering condolence is a natural reaction to hearing someone{'}s distress. Individuals frequently express distress in social media, where some communities can provide support. However, not all condolence is equal{---}trite responses offer little actual support despite their good intentions. Here, we develop computational tools to create a massive dataset of 11.4M expressions of distress and 2.8M corresponding offerings of condolence in order to examine the dynamics of condolence online. Our study reveals widespread disparity in what types of distress receive supportive condolence rather than just engagement. Building on studies from social psychology, we analyze the language of condolence and develop a new dataset for quantifying the empathy in a condolence using appraisal theory. Finally, we demonstrate that the features of condolence individuals find most helpful online differ substantially in their features from those seen in interpersonal settings.
An Embedding Model for Estimating Legislative Preferences from the Frequency and Sentiment of Tweets
Authors: Gregory Spell, Brian Guay, Sunshine Hillygus, Lawrence Carin
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11

Legislator preferences are typically represented as measures of general ideology estimated from roll call votes on legislation, potentially masking important nuances in legislators{'} political attitudes. In this paper we introduce a method of measuring more specific legislator attitudes using an alternative expression of preferences: tweeting. Specifically, we present an embedding-based model for predicting the frequency and sentiment of legislator tweets. To illustrate our method, we model legislators{'} attitudes towards President Donald Trump as vector embeddings that interact with embeddings for Trump himself constructed using a neural network from the text of his daily tweets. We demonstrate the predictive performance of our model on tweets authored by members of the U.S. House and Senate related to the president from November 2016 to February 2018. We further assess the quality of our learned representations for legislators by comparing to traditional measures of legislator preferences.
Connecting the Dots: Event Graph Schema Induction with Path Language Modeling
Authors: Manling Li, Qi Zeng, Ying Lin, Kyunghyun Cho, Heng Ji, Jonathan May, Nathanael Chambers, Clare Voss
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Language Modeling, Language Modelling

Event schemas can guide our understanding and ability to make predictions with respect to what might happen next. We propose a new Event Graph Schema, where two event types are connected through multiple paths involving entities that fill important roles in a coherent story. We then introduce Path Language Model, an auto-regressive language model trained on event-event paths, and select salient and coherent paths to probabilistically construct these graph schemas. We design two evaluation metrics, instance coverage and instance coherence, to evaluate the quality of graph schema induction, by checking when coherent event instances are covered by the schema graph. Intrinsic evaluations show that our approach is highly effective at inducing salient and coherent schemas. Extrinsic evaluations show the induced schema repository provides significant improvement to downstream end-to-end Information Extraction over a state-of-the-art joint neural extraction model, when used as additional global features to unfold instance graphs.
Incremental Event Detection via Knowledge Consolidation Networks
Authors: Pengfei Cao, Yubo Chen, Jun Zhao, Taifeng Wang
Date: 2020-11-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Event Detection, Incremental Learning

Conventional approaches to event detection usually require a fixed set of pre-defined event types. Such a requirement is often challenged in real-world applications, as new events continually occur. Due to huge computation cost and storage budge, it is infeasible to store all previous data and re-train the model with all previous data and new data, every time new events arrive. We formulate such challenging scenarios as incremental event detection, which requires a model to learn new classes incrementally without performance degradation on previous classes. However, existing incremental learning methods cannot handle semantic ambiguity and training data imbalance problems between old and new classes in the task of incremental event detection. In this paper, we propose a Knowledge Consolidation Network (KCN) to address the above issues. Specifically, we devise two components, prototype enhanced retrospection and hierarchical distillation, to mitigate the adverse effects of semantic ambiguity and class imbalance, respectively. Experimental results demonstrate the effectiveness of the proposed method, outperforming the state-of-the-art model by 19{\%} and 13.4{\%} of whole F1 score on ACE benchmark and TAC KBP benchmark, respectively.