pwc-archive/papers-with-abstracts huggingface dataset:
Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translation
Authors: Eva Vanmassenhove, Dimitar Shterionov, Matthew Gwilliam
Date: 2021-01-30 00:00:00
Proceeding: EACL 2021 2
Tasks: Machine Translation, NMT, Translation

Recent studies in the field of Machine Translation (MT) and Natural Language Processing (NLP) have shown that existing models amplify biases observed in the training data. The amplification of biases in language technology has mainly been examined with respect to specific phenomena, such as gender bias. In this work, we go beyond the study of gender in MT and investigate how bias amplification might affect language in a broader sense. We hypothesize that the 'algorithmic bias', i.e. an exacerbation of frequently observed patterns in combination with a loss of less frequent ones, not only exacerbates societal biases present in current datasets but could also lead to an artificially impoverished language: 'machine translationese'. We assess the linguistic richness (on a lexical and morphological level) of translations created by different data-driven MT paradigms - phrase-based statistical (PB-SMT) and neural MT (NMT). Our experiments show that there is a loss of lexical and morphological richness in the translations produced by all investigated MT paradigms for two language pairs (EN<=>FR and EN<=>ES).
Epistocracy Algorithm: A Novel Hyper-heuristic Optimization Strategy for Solving Complex Optimization Problems
Authors: Seyed Ziae Mousavi Mojab, Seyedmohammad Shams, Hamid Soltanian-Zadeh, Farshad Fotouhi
Date: 2021-01-30 00:00:00

This paper proposes a novel evolutionary algorithm called Epistocracy which incorporates human socio-political behavior and intelligence to solve complex optimization problems. The inspiration of the Epistocracy algorithm originates from a political regime where educated people have more voting power than the uneducated or less educated. The algorithm is a self-adaptive, and multi-population optimizer in which the evolution process takes place in parallel for many populations led by a council of leaders. To avoid stagnation in poor local optima and to prevent a premature convergence, the algorithm employs multiple mechanisms such as dynamic and adaptive leadership based on gravitational force, dynamic population allocation and diversification, variance-based step-size determination, and regression-based leadership adjustment. The algorithm uses a stratified sampling method called Latin Hypercube Sampling (LHS) to distribute the initial population more evenly for exploration of the search space and exploitation of the accumulated knowledge. To investigate the performance and evaluate the reliability of the algorithm, we have used a set of multimodal benchmark functions, and then applied the algorithm to the MNIST dataset to further verify the accuracy, scalability, and robustness of the algorithm. Experimental results show that the Epistocracy algorithm outperforms the tested state-of-the-art evolutionary and swarm intelligence algorithms in terms of performance, precision, and convergence.
Using Multiple Generative Adversarial Networks to Build Better-Connected Levels for Mega Man
Authors: Benjamin Capps, Jacob Schrum
Date: 2021-01-30 00:00:00

Generative Adversarial Networks (GANs) can generate levels for a variety of games. This paper focuses on combining GAN-generated segments in a snaking pattern to create levels for Mega Man. Adjacent segments in such levels can be orthogonally adjacent in any direction, meaning that an otherwise fine segment might impose a barrier between its neighbor depending on what sorts of segments in the training set are being most closely emulated: horizontal, vertical, or corner segments. To pick appropriate segments, multiple GANs were trained on different types of segments to ensure better flow between segments. Flow was further improved by evolving the latent vectors for the segments being joined in the level to maximize the length of the level's solution path. Using multiple GANs to represent different types of segments results in significantly longer solution paths than using one GAN for all segment types, and a human subject study verifies that these levels are more fun and have more human-like design than levels produced by one GAN.
An Unsupervised Language-Independent Entity Disambiguation Method and its Evaluation on the English and Persian Languages
Authors: Majid Asgari-Bidhendi, Behrooz Janfada, Amir Havangi, Sayyed Ali Hossayni, Behrouz Minaei-Bidgoli
Date: 2021-01-31 00:00:00
Tasks: Entity Disambiguation, Entity Linking, Natural Language Understanding

Entity Linking is one of the essential tasks of information extraction and natural language understanding. Entity linking mainly consists of two tasks: recognition and disambiguation of named entities. Most studies address these two tasks separately or focus only on one of them. Moreover, most of the state-of-the -art entity linking algorithms are either supervised, which have poor performance in the absence of annotated corpora or language-dependent, which are not appropriate for multi-lingual applications. In this paper, we introduce an Unsupervised Language-Independent Entity Disambiguation (ULIED), which utilizes a novel approach to disambiguate and link named entities. Evaluation of ULIED on different English entity linking datasets as well as the only available Persian dataset illustrates that ULIED in most of the cases outperforms the state-of-the-art unsupervised multi-lingual approaches.
Superposition with Lambdas
Authors: Alexander Bentkamp, Jasmin Blanchette, Sophie Tourret, Petar Vukmirović, Uwe Waldmann
Date: 2021-01-31 00:00:00

We designed a superposition calculus for a clausal fragment of extensional polymorphic higher-order logic that includes anonymous functions but excludes Booleans. The inference rules work on $\beta\eta$-equivalence classes of $\lambda$-terms and rely on higher-order unification to achieve refutational completeness. We implemented the calculus in the Zipperposition prover and evaluated it on TPTP and Isabelle benchmarks. The results suggest that superposition is a suitable basis for higher-order reasoning.
Adversarial Contrastive Pre-training for Protein Sequences
Authors: Matthew B. A. McDermott, Brendan Yap, Harry Hsu, Di Jin, Peter Szolovits
Date: 2021-01-31 00:00:00
Tasks: Language Modeling, Language Modelling

Recent developments in Natural Language Processing (NLP) demonstrate that large-scale, self-supervised pre-training can be extremely beneficial for downstream tasks. These ideas have been adapted to other domains, including the analysis of the amino acid sequences of proteins. However, to date most attempts on protein sequences rely on direct masked language model style pre-training. In this work, we design a new, adversarial pre-training method for proteins, extending and specializing similar advances in NLP. We show compelling results in comparison to traditional MLM pre-training, though further development is needed to ensure the gains are worth the significant computational cost.
Polyphone Disambiguation in Mandarin Chinese with Semi-Supervised Learning
Authors: Yi Shi, Congyi Wang, Yu Chen, Bin Wang
Date: 2021-02-01 00:00:00
Tasks: Polyphone disambiguation

The majority of Chinese characters are monophonic, while a special group of characters, called polyphonic characters, have multiple pronunciations. As a prerequisite of performing speech-related generative tasks, the correct pronunciation must be identified among several candidates. This process is called Polyphone Disambiguation. Although the problem has been well explored with both knowledge-based and learning-based approaches, it remains challenging due to the lack of publicly available labeled datasets and the irregular nature of polyphone in Mandarin Chinese. In this paper, we propose a novel semi-supervised learning (SSL) framework for Mandarin Chinese polyphone disambiguation that can potentially leverage unlimited unlabeled text data. We explore the effect of various proxy labeling strategies including entropy-thresholding and lexicon-based labeling. Qualitative and quantitative experiments demonstrate that our method achieves state-of-the-art performance. In addition, we publish a novel dataset specifically for the polyphone disambiguation task to promote further research.
Human Perceptions on Moral Responsibility of AI: A Case Study in AI-Assisted Bail Decision-Making
Authors: Gabriel Lima, Nina Grgić-Hlača, Meeyoung Cha
Date: 2021-02-01 00:00:00
Tasks: Attribute, Decision Making

How to attribute responsibility for autonomous artificial intelligence (AI) systems' actions has been widely debated across the humanities and social science disciplines. This work presents two experiments ($N$=200 each) that measure people's perceptions of eight different notions of moral responsibility concerning AI and human agents in the context of bail decision-making. Using real-life adapted vignettes, our experiments show that AI agents are held causally responsible and blamed similarly to human agents for an identical task. However, there was a meaningful difference in how people perceived these agents' moral responsibility; human agents were ascribed to a higher degree of present-looking and forward-looking notions of responsibility than AI agents. We also found that people expect both AI and human decision-makers and advisors to justify their decisions regardless of their nature. We discuss policy and HCI implications of these findings, such as the need for explainable AI in high-stakes scenarios.
Autonomous Navigation through intersections with Graph ConvolutionalNetworks and Conditional Imitation Learning for Self-driving Cars
Authors: Xiaodong Mei, Yuxiang Sun, Yuying Chen, Congcong Liu, Ming Liu
Date: 2021-02-01 00:00:00
Tasks: Autonomous Driving, Autonomous Navigation, Imitation Learning, Self-Driving Cars

In autonomous driving, navigation through unsignaled intersections with many traffic participants moving around is a challenging task. To provide a solution to this problem, we propose a novel branched network G-CIL for the navigation policy learning. Specifically, we firstly represent such dynamic environments as graph-structured data and propose an effective strategy for edge definition to aggregate surrounding information for the ego-vehicle. Then graph convolutional neural networks are used as the perception module to capture global and geometric features from the environment. To generate safe and efficient navigation policy, we further incorporate it with conditional imitation learning algorithm, to learn driving behaviors directly from expert demonstrations. Our proposed network is capable of handling a varying number of surrounding vehicles and generating optimal control actions (e.g., steering angle and throttle) according to the given high-level commands (e.g., turn left towards the global goal). Evaluations on unsignaled intersections with various traffic densities demonstrate that our end-to-end trainable neural network outperforms the baselines with higher success rate and shorter navigation time.
Many Hands Make Light Work: Using Essay Traits to Automatically Score Essays
Authors: Rahul Kumar, Sandeep Mathias, Sriparna Saha, Pushpak Bhattacharyya
Date: 2021-02-01 00:00:00
Proceeding: NAACL 2022 7
Tasks: Automated Essay Scoring, Multi-Task Learning

Most research in the area of automatic essay grading (AEG) is geared towards scoring the essay holistically while there has also been some work done on scoring individual essay traits. In this paper, we describe a way to score essays holistically using a multi-task learning (MTL) approach, where scoring the essay holistically is the primary task, and scoring the essay traits is the auxiliary task. We compare our results with a single-task learning (STL) approach, using both LSTMs and BiLSTMs. We also compare our results of the auxiliary task with such tasks done in other AEG systems. To find out which traits work best for different types of essays, we conduct ablation tests for each of the essay traits. We also report the runtime and number of training parameters for each system. We find that MTL-based BiLSTM system gives the best results for scoring the essay holistically, as well as performing well on scoring the essay traits.
Implicit Feedback-based Group Recommender System for Internet of Thing Applications
Authors: Zhiwei Guo, Keping Yu, Tan Guo, Ali Kashif Bashir, Muhammad Imran, Mohsen Guizani
Date: 2021-01-29 00:00:00
Tasks: Recommendation Systems

With the prevalence of Internet of Things (IoT)-based social media applications, the distance among people has been greatly shortened. As a result, recommender systems in IoT-based social media need to be developed oriented to groups of users rather than individual users. However, existing methods were highly dependent on explicit preference feedbacks, ignoring scenarios of implicit feedback. To remedy such gap, this paper proposes an implicit feedback-based group recommender system using probabilistic inference and non-cooperative game(GREPING) for IoT-based social media. Particularly, unknown process variables can be estimated from observable implicit feedbacks via Bayesian posterior probability inference. In addition, the globally optimal recommendation results can be calculated with the aid of non-cooperative game. Two groups of experiments are conducted to assess the GREPING from two aspects: efficiency and robustness. Experimental results show obvious promotion and considerable stability of the GREPING compared to baseline methods.
Deep Evolutionary Learning for Molecular Design
Authors: Yifeng Li, Hsu Kiang Ooi, Alain Tchagang
Date: 2020-12-28 00:00:00
Tasks: Bayesian Optimization

In this paper, we propose a deep evolutionary learning (DEL) process that integrates fragment-based deep generative model and multi-objective evolutionary computation for molecular design. Our approach enables (1) evolutionary operations in the latent space of the generative model, rather than the structural space, to generate novel promising molecular structures for the next evolutionary generation, and (2) generative model fine-tuning using newly generated high-quality samples. Thus, DEL implements a data-model co-evolution concept which improves both sample population and generative model learning. Experiments on two public datasets indicate that sample population obtained by DEL exhibits improved property distributions, and dominates samples generated by multi-objective Bayesian optimization algorithms.
Evolutionary Algorithms for Fuzzy Cognitive Maps
Authors: Stefanos Tsimenidis
Date: 2020-12-19 00:00:00
Tasks: Evolutionary Algorithms

Fuzzy Cognitive Maps (FCMs) is a complex systems modeling technique which, due to its unique advantages, has lately risen in popularity. They are based on graphs that represent the causal relationships among the parameters of the system to be modeled, and they stand out for their interpretability and flexibility. With the late popularity of FCMs, a plethora of research efforts have taken place to develop and optimize the model. One of the most important elements of FCMs is the learning algorithm they use, and their effectiveness is largely determined by it. The learning algorithms learn the node weights of an FCM, with the goal of converging towards the desired behavior. The present study reviews the genetic algorithms used for training FCMs, as well as gives a general overview of the FCM learning algorithms, putting evolutionary computing into the wider context.
Niching Diversity Estimation for Multi-modal Multi-objective Optimization
Authors: Yiming Peng, Hisao Ishibuchi
Date: 2021-01-31 00:00:00
Tasks: Diversity

Niching is an important and widely used technique in evolutionary multi-objective optimization. Its applications mainly focus on maintaining diversity and avoiding early convergence to local optimum. Recently, a special class of multi-objective optimization problems, namely, multi-modal multi-objective optimization problems (MMOPs), started to receive increasing attention. In MMOPs, a solution in the objective space may have multiple inverse images in the decision space, which are termed as equivalent solutions. Since equivalent solutions are overlapping (i.e., occupying the same position) in the objective space, standard diversity estimators such as crowding distance are likely to select one of them and discard the others, which may cause diversity loss in the decision space. In this study, a general niching mechanism is proposed to make standard diversity estimators more efficient when handling MMOPs. In our experiments, we integrate our proposed niching diversity estimation method into SPEA2 and NSGA-II and evaluate their performance on several MMOPs. Experimental results show that the proposed niching mechanism notably enhances the performance of SPEA2 and NSGA-II on various MMOPs.
Demonstrating the Evolution of GANs through t-SNE
Authors: Victor Costa, Nuno Lourenço, João Correia, Penousal Machado
Date: 2021-01-31 00:00:00
Tasks: Evolutionary Algorithms

Generative Adversarial Networks (GANs) are powerful generative models that achieved strong results, mainly in the image domain. However, the training of GANs is not trivial, presenting some challenges tackled by different strategies. Evolutionary algorithms, such as COEGAN, were recently proposed as a solution to improve the GAN training, overcoming common problems that affect the model, such as vanishing gradient and mode collapse. In this work, we propose an evaluation method based on t-distributed Stochastic Neighbour Embedding (t-SNE) to assess the progress of GANs and visualize the distribution learned by generators in training. We propose the use of the feature space extracted from trained discriminators to evaluate samples produced by generators and from the input dataset. A metric based on the resulting t-SNE maps and the Jaccard index is proposed to represent the model quality. Experiments were conducted to assess the progress of GANs when trained using COEGAN. The results show both by visual inspection and metrics that the Evolutionary Algorithm gradually improves discriminators and generators through generations, avoiding problems such as mode collapse.
Challenges in Automated Debiasing for Toxic Language Detection
Authors: Xuhui Zhou, Maarten Sap, Swabha Swayamdipta, Noah A. Smith, Yejin Choi
Date: 2021-01-29 00:00:00
Proceeding: EACL 2021 2
Tasks: Fairness, text-classification, Text Classification

Biased associations have been a challenge in the development of classifiers for detecting toxic language, hindering both fairness and accuracy. As potential solutions, we investigate recently introduced debiasing methods for text classification datasets and models, as applied to toxic language detection. Our focus is on lexical (e.g., swear words, slurs, identity mentions) and dialectal markers (specifically African American English). Our comprehensive experiments establish that existing methods are limited in their ability to prevent biased behavior in current toxicity detectors. We then propose an automatic, dialect-aware data correction method, as a proof-of-concept. Despite the use of synthetic labels, this method reduces dialectal associations with toxicity. Overall, our findings show that debiasing a model trained on biased toxic language data is not as effective as simply relabeling the data to remove existing biases.
Can We Automate Scientific Reviewing?
Authors: Weizhe Yuan, PengFei Liu, Graham Neubig
Date: 2021-01-30 00:00:00
Tasks: Review Generation

The rapid development of science and technology has been accompanied by an exponential growth in peer-reviewed scientific publications. At the same time, the review of each paper is a laborious process that must be carried out by subject matter experts. Thus, providing high-quality reviews of this growing number of papers is a significant challenge. In this work, we ask the question "can we automate scientific reviewing?", discussing the possibility of using state-of-the-art natural language processing (NLP) models to generate first-pass peer reviews for scientific papers. Arguably the most difficult part of this is defining what a "good" review is in the first place, so we first discuss possible evaluation measures for such reviews. We then collect a dataset of papers in the machine learning domain, annotate them with different aspects of content covered in each review, and train targeted summarization models that take in papers to generate reviews. Comprehensive experimental results show that system-generated reviews tend to touch upon more aspects of the paper than human-written reviews, but the generated text can suffer from lower constructiveness for all aspects except the explanation of the core ideas of the papers, which are largely factually correct. We finally summarize eight challenges in the pursuit of a good review generation system together with potential solutions, which, hopefully, will inspire more future research on this subject. We make all code, and the dataset publicly available: https://github.com/neulab/ReviewAdvisor, as well as a ReviewAdvisor system: http://review.nlpedia.ai/.
Learning From Human Correction
Authors: Tong Guo
Date: 2021-01-30 00:00:00
Tasks: Deep Learning, General Classification, text-classification, Text Classification

In industry NLP application, our manually labeled data has a certain number of noisy data. We present a simple method to find the noisy data and re-label them manually, meanwhile we collect the correction information. Then we present novel method to incorporate the human correction information into deep learning model. Human know how to correct noisy data. So the correction information can be inject into deep learning model. We do the experiment on our own text classification dataset, which is manually labeled, because we re-label the noisy data in our dataset for our industry application. The experiment result shows that our method improve the classification accuracy from 91.7% to 92.5%. The 91.7% accuracy is trained on the corrected dataset, which improve the baseline from 83.3% to 91.7%.
Triple M: A Practical Text-to-speech Synthesis System With Multi-guidance Attention And Multi-band Multi-time LPCNet
Authors: Shilun Lin, Fenglong Xie, Li Meng, Xinhui Li, Li Lu
Date: 2021-01-30 00:00:00
Tasks: CPU, Sentence, Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis

In this work, a robust and efficient text-to-speech (TTS) synthesis system named Triple M is proposed for large-scale online application. The key components of Triple M are: 1) A sequence-to-sequence model adopts a novel multi-guidance attention to transfer complementary advantages from guiding attention mechanisms to the basic attention mechanism without in-domain performance loss and online service modification. Compared with single attention mechanism, multi-guidance attention not only brings better naturalness to long sentence synthesis, but also reduces the word error rate by 26.8%. 2) A new efficient multi-band multi-time vocoder framework, which reduces the computational complexity from 2.8 to 1.0 GFLOP and speeds up LPCNet by 2.75x on a single CPU.