pwc-archive/papers-with-abstracts huggingface dataset:
Improving Arabic Diacritization by Learning to Diacritize and Translate
Authors: Brian Thompson, Ali Alshehri
Date: 2021-09-29 00:00:00
Proceeding: IWSLT (ACL) 2022 5
Tasks: Translation

We propose a novel multitask learning method for diacritization which trains a model to both diacritize and translate. Our method addresses data sparsity by exploiting large, readily available bitext corpora. Furthermore, translation requires implicit linguistic and semantic knowledge, which is helpful for resolving ambiguities in the diacritization task. We apply our method to the Penn Arabic Treebank and report a new state-of-the-art word error rate of 4.79%. We also conduct manual and automatic analysis to better understand our method and highlight some of the remaining challenges in diacritization.
Second Order WinoBias (SoWinoBias) Test Set for Latent Gender Bias Detection in Coreference Resolution
Authors: Hillary Dawkins
Date: 2021-09-28 00:00:00
Proceeding: ACL (GeBNLP) 2021 8
Tasks: Bias Detection, coreference-resolution, Coreference Resolution, Gender Bias Detection

We observe an instance of gender-induced bias in a downstream application, despite the absence of explicit gender words in the test cases. We provide a test set, SoWinoBias, for the purpose of measuring such latent gender bias in coreference resolution systems. We evaluate the performance of current debiasing methods on the SoWinoBias test set, especially in reference to the method's design and altered embedding space properties. See https://github.com/hillarydawkins/SoWinoBias.
Marked Attribute Bias in Natural Language Inference
Authors: Hillary Dawkins
Date: 2021-09-28 00:00:00
Proceeding: Findings (ACL) 2021 8
Tasks: Attribute, Natural Language Inference, Word Embeddings

Reporting and providing test sets for harmful bias in NLP applications is essential for building a robust understanding of the current problem. We present a new observation of gender bias in a downstream NLP application: marked attribute bias in natural language inference. Bias in downstream applications can stem from training data, word embeddings, or be amplified by the model in use. However, focusing on biased word embeddings is potentially the most impactful first step due to their universal nature. Here we seek to understand how the intrinsic properties of word embeddings contribute to this observed marked attribute effect, and whether current post-processing methods address the bias successfully. An investigation of the current debiasing landscape reveals two open problems: none of the current debiased embeddings mitigate the marked attribute error, and none of the intrinsic bias measures are predictive of the marked attribute effect. By noticing that a new type of intrinsic bias measure correlates meaningfully with the marked attribute effect, we propose a new postprocessing debiasing scheme for static word embeddings. The proposed method applied to existing embeddings achieves new best results on the marked attribute bias test set. See https://github.com/hillary-dawkins/MAB.
Shaking Syntactic Trees on the Sesame Street: Multilingual Probing with Controllable Perturbations
Authors: Ekaterina Taktasheva, Vladislav Mikhailov, Ekaterina Artemova
Date: 2021-09-28 00:00:00
Proceeding: EMNLP (MRL) 2021 11
Tasks: Sensitivity

Recent research has adopted a new experimental field centered around the concept of text perturbations which has revealed that shuffled word order has little to no impact on the downstream performance of Transformer-based language models across many NLP tasks. These findings contradict the common understanding of how the models encode hierarchical and structural information and even question if the word order is modeled with position embeddings. To this end, this paper proposes nine probing datasets organized by the type of \emph{controllable} text perturbation for three Indo-European languages with a varying degree of word order flexibility: English, Swedish and Russian. Based on the probing analysis of the M-BERT and M-BART models, we report that the syntactic sensitivity depends on the language and model pre-training objectives. We also find that the sensitivity grows across layers together with the increase of the perturbation granularity. Last but not least, we show that the models barely use the positional information to induce syntactic trees from their intermediate self-attention and contextualized representations.
Text Simplification for Comprehension-based Question-Answering
Authors: Tanvi Dadu, Kartikey Pant, Seema Nagar, Ferdous Ahmed Barbhuiya, Kuntal Dey
Date: 2021-09-28 00:00:00
Proceeding: WNUT (ACL) 2021 11
Tasks: Machine Translation, Question Answering, Semantic Role Labeling, Sentence, Style Transfer, Text Simplification

Text simplification is the process of splitting and rephrasing a sentence to a sequence of sentences making it easier to read and understand while preserving the content and approximating the original meaning. Text simplification has been exploited in NLP applications like machine translation, summarization, semantic role labeling, and information extraction, opening a broad avenue for its exploitation in comprehension-based question-answering downstream tasks. In this work, we investigate the effect of text simplification in the task of question-answering using a comprehension context. We release Simple-SQuAD, a simplified version of the widely-used SQuAD dataset. Firstly, we outline each step in the dataset creation pipeline, including style transfer, thresholding of sentences showing correct transfer, and offset finding for each answer. Secondly, we verify the quality of the transferred sentences through various methodologies involving both automated and human evaluation. Thirdly, we benchmark the newly created corpus and perform an ablation study for examining the effect of the simplification process in the SQuAD-based question answering task. Our experiments show that simplification leads to up to 2.04% and 1.74% increase in Exact Match and F1, respectively. Finally, we conclude with an analysis of the transfer process, investigating the types of edits made by the model, and the effect of sentence length on the transfer model.
Double framed moduli spaces of quiver representations
Authors: Marco Armenta, Thomas Brüstle, Souheila Hassoun, Markus Reineke
Date: 2021-09-29 00:00:00

Motivated by problems in the neural networks setting, we study moduli spaces of double framed quiver representations and give both a linear algebra description and a representation theoretic description of these moduli spaces. We define a network category whose isomorphism classes of objects correspond to the orbits of quiver representations, in which neural networks map input data. We then prove that the output of a neural network depends only on the corresponding point in the moduli space. Finally, we present a different perspective on mapping neural networks with a specific activation function, called ReLU, to a moduli space using the symplectic reduction approach to quiver moduli.
Improvements for mlrose applied to the Traveling Salesperson Problem
Authors: Stefan Wintersteller, Martin Uray, Michael Lehenauer, Stefan Huber
Date: 2021-09-29 00:00:00

In this paper we discuss the application of Artificial Intelligence (AI) to the exemplary industrial use case of the two-dimensional commissioning problem in a high-bay storage, which essentially can be phrased as an instance of Traveling Salesperson Problem (TSP). We investigate the mlrose library that provides an TSP optimizer based on various heuristic optimization techniques. Our focus is on two methods, namely Genetic Algorithm (GA) and Hill Climbing (HC), which are provided by mlrose. We present improvements for both methods that yield shorter tour lengths, by moderately exploiting the problem structure of TSP. That is, the proposed improvements have a generic character and are not limited to TSP only.
Stability Analysis of Fractional Order Memristor Synapse-coupled Hopfield Neural Network with Ring Structure
Authors: Leila Eftekhari, Mohammad M. Amirian
Date: 2021-09-29 00:00:00

A memristor is a nonlinear two-terminal electrical element that incorporates memory features and nanoscale properties, enabling us to design very high-density artificial neural networks. To enhance the memory property, we should use mathematical frameworks like fractional calculus, which is capable of doing so. Here, we first present a fractional-order memristor synapse-coupling Hopfield neural network on two neurons and then extend the model to a neural network with a ring structure that consists of n sub-network neurons, increasing the synchronization in the network. Necessary and sufficient conditions for the stability of equilibrium points are investigated, highlighting the dependency of the stability on the fractional-order value and the number of neurons. Numerical simulations and bifurcation analysis, along with Lyapunov exponents, are given in the two-neuron case that substantiates the theoretical findings, suggesting possible routes towards chaos when the fractional order of the system increases. In the n-neuron case also, it is revealed that the stability depends on the structure and number of sub-networks.
Influence of Binomial Crossover on Approximation Error of Evolutionary Algorithms
Authors: Cong Wang, Jun He, Yu Chen, Xiufen Zou
Date: 2021-09-29 00:00:00
Tasks: Evolutionary Algorithms

Although differential evolution (DE) algorithms perform well on a large variety of complicated optimization problems, only a few theoretical studies are focused on the working principle of DE algorithms. To make the first attempt to reveal the function of binomial crossover, this paper aims to answer whether it can reduce the approximation error of evolutionary algorithms. By investigating the expected approximation error and the probability of not finding the optimum, we conduct a case study comparing two evolutionary algorithms with and without binomial crossover on two classical benchmark problems: OneMax and Deceptive. It is proven that using binomial crossover leads to the dominance of transition matrices. As a result, the algorithm with binomial crossover asymptotically outperforms that without crossover on both OneMax and Deceptive, and outperforms on OneMax, however, not on Deceptive. Furthermore, an adaptive parameter strategy is proposed which can strengthen the superiority of binomial crossover on Deceptive.
New Hybrid Techniques for Business Recommender Systems
Authors: Charuta Pande, Hans Friedrich Witschel, Andreas Martin
Date: 2021-09-27 00:00:00
Tasks: Recommendation Systems

Besides the typical applications of recommender systems in B2C scenarios such as movie or shopping platforms, there is a rising interest in transforming the human-driven advice provided e.g. in consultancy via the use of recommender systems. We explore the special characteristics of such knowledge-based B2B services and propose a process that allows to incorporate recommender systems into them. We suggest and compare several recommender techniques that allow to incorporate the necessary contextual knowledge (e.g. company demographics). These techniques are evaluated in isolation on a test set of business intelligence consultancy cases. We then identify the respective strengths of the different techniques and propose a new hybridisation strategy to combine these strengths. Our results show that the hybridisation leads to a substantial performance improvement over the individual methods.
Efficient Two-Step Networks for Temporal Action Segmentation
Authors: Yunheng Li, Zhuben Dong, Kaiyuan Liu, Lin Feng, Lianyu Hu, Jie Zhu, Li Xu, YuHan Wang, Shenglan Liu
Date: 2021-04-30 00:00:00
Proceeding: Neurocomputing 2021 4
Tasks: Action Segmentation, Segmentation, Temporal Action Segmentation, Vocal Bursts Valence Prediction

Due to boundary ambiguity and over-segmentation issues, identifying all the frames in long untrimmed videos is still challenging. To address these problems, we present the Efficient Two-Step Network (ETSN) with two components. The first step of ETSN is Efficient Temporal Series Pyramid Networks (ETSPNet) that capture both local and global frame-level features and provide accurate predictions of segmentation boundaries. The second step is a novel unsupervised approach called Local Burr Suppression (LBS), which significantly reduces the over-segmentation errors. Our empirical evaluations on the benchmarks including 50Salads, GTEA and Breakfast dataset demonstrate that ETSN outperforms the current state-of-the-art methods by a large margin.
Modeling and Control of bittide Synchronization
Authors: Sanjay Lall, Calin Cascaval, Martin Izzard, Tammo Spalink
Date: 2021-09-29 00:00:00
Tasks: Common Sense Reasoning

Distributed system applications rely on a fine-grain common sense of time. Existing systems maintain the common sense of time by keeping each independent machine as close as possible to wall-clock time through a combination of software protocols like NTP and GPS signals and/or precision references like atomic clocks. This approach is expensive and has tolerance limitations that require protocols to deal with asynchrony and its performance consequences. Moreover, at data-center scale it is impractical to distribute a physical clock as is done on a chip or printed circuit board. In this paper we introduce a distributed system design that removes the need for physical clock distribution or mechanisms for maintaining close alignment to wall-clock time, and instead provides applications with a perfectly synchronized logical clock. We discuss the abstract frame model (AFM), a mathematical model that underpins the system synchronization. The model is based on the rate of communication between nodes in a topology without requiring a global clock. We show that there are families of controllers that satisfy the properties required for existence and uniqueness of solutions to the AFM, and give examples.
Adversarial Linear-Quadratic Mean-Field Games over Multigraphs
Authors: Muhammad Aneeq uz Zaman, Sujay Bhatt, Tamer Başar
Date: 2021-09-29 00:00:00

In this paper, we propose a game between an exogenous adversary and a network of agents connected via a multigraph. The multigraph is composed of (1) a global graph structure, capturing the virtual interactions among the agents, and (2) a local graph structure, capturing physical/local interactions among the agents. The aim of each agent is to achieve consensus with the other agents in a decentralized manner by minimizing a local cost associated with its local graph and a global cost associated with the global graph. The exogenous adversary, on the other hand, aims to maximize the average cost incurred by all agents in the multigraph. We derive Nash equilibrium policies for the agents and the adversary in the Mean-Field Game setting, when the agent population in the global graph is arbitrarily large and the ``homogeneous mixing" hypothesis holds on local graphs. This equilibrium is shown to be unique and the equilibrium Markov policies for each agent depend on the local state of the agent, as well as the influences on the agent by the local and global mean fields.
A Universal Deep Room Acoustics Estimator
Authors: Paula Sánchez López, Paul Callens, Milos Cernak
Date: 2021-09-29 00:00:00
Tasks: Room Impulse Response (RIR)

Speech audio quality is subject to degradation caused by an acoustic environment and isotropic ambient and point noises. The environment can lead to decreased speech intelligibility and loss of focus and attention by the listener. Basic acoustic parameters that characterize the environment well are (i) signal-to-noise ratio (SNR), (ii) speech transmission index, (iii) reverberation time, (iv) clarity, and (v) direct-to-reverberant ratio. Except for the SNR, these parameters are usually derived from the Room Impulse Response (RIR) measurements; however, such measurements are often not available. This work presents a universal room acoustic estimator design based on convolutional recurrent neural networks that estimate the acoustic environment measurement blindly and jointly. Our results indicate that the proposed system is robust to non-stationary signal variations and outperforms current state-of-the-art methods.
Objective-oriented method for uniformation of various directivity representations
Authors: Adam Szwajcowski
Date: 2021-09-29 00:00:00

Over recent years, numerous attempts were taken to provide efficient methods of directivity representation, either regarding sound sources or head-related transfer functions. Because of the wide variety of programming tools and scripts used by different researchers, the resulting representations are inconvevnient to reproduce and compare with each other, hampering the development of the subject. Within this paper, an objective-oriented method is proposed to deal with this issue. The suggested approach bases on defining classes for different directivity models that share some general properties of directivity functions, allowing for easy comparison between different representations. A basic Matlab toolbox utlizing this method is presented alongside exemplary implementations of directivity models based on spherical and hyperspherical harmonics.
Comparison of Self-Supervised Speech Pre-Training Methods on Flemish Dutch
Authors: Jakob Poncelet, Hugo Van hamme
Date: 2021-09-29 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Representation Learning, speech-recognition, Speech Recognition

Recent research in speech processing exhibits a growing interest in unsupervised and self-supervised representation learning from unlabelled data to alleviate the need for large amounts of annotated data. We investigate several popular pre-training methods and apply them to Flemish Dutch. We compare off-the-shelf English pre-trained models to models trained on an increasing amount of Flemish data. We find that the most important factors for positive transfer to downstream speech recognition tasks include a substantial amount of data and a matching pre-training domain. Ideally, we also finetune on an annotated subset in the target language. All pre-trained models improve linear phone separability in Flemish, but not all methods improve Automatic Speech Recognition. We experience superior performance with wav2vec 2.0 and we obtain a 30% WER improvement by finetuning the multilingually pre-trained XLSR-53 model on Flemish Dutch, after integration into an HMM-DNN acoustic model.
Strategic formation of collaborative networks
Authors: Philip Solimine, Luke Boosey
Date: 2021-09-29 00:00:00
Tasks: counterfactual

We examine behavior in an experimental collaboration game that incorporates endogenous network formation. The environment is modeled as a generalization of the voluntary contributions mechanism. By varying the information structure in a controlled laboratory experiment, we examine the underlying mechanisms of reciprocity that generate emergent patterns in linking and contribution decisions. Providing players more detailed information about the sharing behavior of others drastically increases efficiency, and positively affects a number of other key outcomes. To understand the driving causes of these changes in behavior we develop and estimate a structural model for actions and small network panels and identify how social preferences affect behavior. We find that the treatment reduces altruism but stimulates reciprocity, helping players coordinate to reach mutually beneficial outcomes. In a set of counterfactual simulations, we show that increasing trust in the community would encourage higher average contributions at the cost of mildly increased free-riding. Increasing overall reciprocity greatly increases collaborative behavior when there is limited information but can backfire in the treatment, suggesting that negative reciprocity and punishment can reduce efficiency. The largest returns would come from an intervention that drives players away from negative and toward positive reciprocity.
On the properties of Laplacian pseudoinverses
Authors: Angela Fontan, Claudio Altafini
Date: 2021-09-29 00:00:00

The pseudoinverse of a graph Laplacian is used in many applications and fields, such as for instance in the computation of the effective resistance in electrical networks, in the calculation of the hitting/commuting times for a Markov chain and in continuous-time distributed averaging problems. In this paper we show that the Laplacian pseudoinverse is in general not a Laplacian matrix but rather a signed Laplacian with the property of being an eventually exponentially positive matrix, i.e., of obeying a strong Perron-Frobenius property. We show further that the set of signed Laplacians with this structure (i.e., eventual exponential positivity) is closed with respect to matrix pseudoinversion. This is true even for signed digraphs, and provided that we restrict to Laplacians that are weight balanced also stability is guaranteed.
The Impact of Network Design Interventions on CPS Security
Authors: Pradeep Sharma Oruganti, Parinaz Naghizadeh, Qadeer Ahmed
Date: 2021-09-29 00:00:00

We study a game-theoretic model of the interactions between a Cyber-Physical System's (CPS) operator (the defender) against an attacker who launches stepping-stone attacks to reach critical assets within the CPS. We consider that, in addition to optimally allocating its security budget to protect the assets, the defender may choose to modify the CPS through network design interventions. In particular, we propose and motivate four ways in which the defender can introduce additional nodes in the CPS: these nodes may be intended as additional safeguards, be added for functional or structural redundancies, or introduce additional functionalities in the system. We analyze the security implications of each of these design interventions, and evaluate their impacts on the security of an automotive network as our case study. We motivate the choice of the attack graph for this case study and elaborate how the parameters in the resulting security game are selected using the CVSS metrics and the ISO-26262 ASIL ratings as guidance. We then use numerical experiments to verify and evaluate how our proposed network interventions may be used to guide improvements in automotive security.
Distributed Optimal Allocation with Quantized Communication and Privacy-Preserving Guarantees
Authors: Jakob Nylöf, Apostolos I. Rikos, Sebin Gracy, Karl H. Johansson
Date: 2021-09-29 00:00:00
Tasks: Privacy Preserving

In this paper, we analyze the problem of optimally allocating resources in a distributed and privacy-preserving manner. We propose a novel distributed optimal resource allocation algorithm with privacy-preserving guarantees, which operates over a directed communication network. Our algorithm converges in finite time and allows each node to process and transmit quantized messages. Our algorithm utilizes a distributed quantized average consensus strategy combined with a privacy-preserving mechanism. We show that the algorithm converges in finite-time, and we prove that, under specific conditions on the network topology, nodes are able to preserve the privacy of their initial state. Finally, to illustrate the results, we consider an example where test kits need to be optimally allocated proportionally to the number of infections in a region. It is shown that the proposed privacy-preserving resource allocation algorithm performs well with an appropriate convergence rate under privacy guarantees.