pwc-archive/papers-with-abstracts huggingface dataset:
Crowd, Lending, Machine, and Bias
Authors: Runshan Fu, Yan Huang, Param Vir Singh
Date: 2020-07-20 00:00:00
Tasks: Fairness

Big data and machine learning (ML) algorithms are key drivers of many fintech innovations. While it may be obvious that replacing humans with machine would increase efficiency, it is not clear whether and where machines can make better decisions than humans. We answer this question in the context of crowd lending, where decisions are traditionally made by a crowd of investors. Using data from Prosper.com, we show that a reasonably sophisticated ML algorithm predicts listing default probability more accurately than crowd investors. The dominance of the machine over the crowd is more pronounced for highly risky listings. We then use the machine to make investment decisions, and find that the machine benefits not only the lenders but also the borrowers. When machine prediction is used to select loans, it leads to a higher rate of return for investors and more funding opportunities for borrowers with few alternative funding options. We also find suggestive evidence that the machine is biased in gender and race even when it does not use gender and race information as input. We propose a general and effective "debasing" method that can be applied to any prediction focused ML applications, and demonstrate its use in our context. We show that the debiased ML algorithm, which suffers from lower prediction accuracy, still leads to better investment decisions compared with the crowd. These results indicate that ML can help crowd lending platforms better fulfill the promise of providing access to financial resources to otherwise underserved individuals and ensure fairness in the allocation of these resources.
An Energy Ontology for Global City Indicators (ISO 37120)
Authors: Alanna Komisar, Mark S. Fox
Date: 2020-07-19 00:00:00
Tasks: General Knowledge

To create tomorrow's smarter cities, today's initiatives will need to create measurable improvements. However, a city is a complex system and measuring its performance generates a breadth of issues. Specifically, determining what criteria should be measured, how indications should be defined, and how should the identified indicators be derived. This working paper is one in series that addresses the creation of a Semantic Web based representation of the 17 different themes of ISO 37120 indicators as part of the larger PolisGnosis Project (Fox, 2017). We define a standard ontology for representing general knowledge for the Energy Theme indicators, and for representing both the definition and data used to derive the Energy indicators.
On Controllability of AI
Authors: Roman V. Yampolskiy
Date: 2020-07-19 00:00:00

Invention of artificial general intelligence is predicted to cause a shift in the trajectory of human civilization. In order to reap the benefits and avoid pitfalls of such powerful technology it is important to be able to control it. However, possibility of controlling artificial general intelligence and its more advanced version, superintelligence, has not been formally established. In this paper, we present arguments as well as supporting evidence from multiple domains indicating that advanced AI can't be fully controlled. Consequences of uncontrollability of AI are discussed with respect to future of humanity and research on AI, and AI safety and security.
AI Failures: A Review of Underlying Issues
Authors: Debarag Narayan Banerjee, Sasanka Sekhar Chanda
Date: 2020-07-18 00:00:00

Instances of Artificial Intelligence (AI) systems failing to deliver consistent, satisfactory performance are legion. We investigate why AI failures occur. We address only a narrow subset of the broader field of AI Safety. We focus on AI failures on account of flaws in conceptualization, design and deployment. Other AI Safety issues like trade-offs between privacy and security or convenience, bad actors hacking into AI systems to create mayhem or bad actors deploying AI for purposes harmful to humanity and are out of scope of our discussion. We find that AI systems fail on account of omission and commission errors in the design of the AI system, as well as upon failure to develop an appropriate interpretation of input information. Moreover, even when there is no significant flaw in the AI software, an AI system may fail because the hardware is incapable of robust performance across environments. Finally an AI system is quite likely to fail in situations where, in effect, it is called upon to deliver moral judgments -- a capability AI does not possess. We observe certain trade-offs in measures to mitigate a subset of AI failures and provide some recommendations.
Working Memory for Online Memory Binding Tasks: A Hybrid Model
Authors: Seyed Mohammad Mahdi Heidarpoor Yazdi, Abdolhossein Abbassian
Date: 2020-08-05 00:00:00

Working Memory is the brain module that holds and manipulates information online. In this work, we design a hybrid model in which a simple feed-forward network is coupled to a balanced random network via a read-write vector called the interface vector. Three cases and their results are discussed similar to the n-back task called, first-order memory binding task, generalized first-order memory task, and second-order memory binding task. The important result is that our dual-component model of working memory shows good performance with learning restricted to the feed-forward component only. Here we take advantage of the random network property without learning. Finally, a more complex memory binding task called, a cue-based memory binding task, is introduced in which a cue is given as input representing a binding relation that prompts the network to choose the useful chunk of memory. To our knowledge, this is the first time that random networks as a flexible memory is shown to play an important role in online binding tasks. We may interpret our results as a candidate model of working memory in which the feed-forward network learns to interact with the temporary storage random network as an attentional-controlling executive system.
Creative AI Through Evolutionary Computation: Principles and Examples
Authors: Risto Miikkulainen
Date: 2020-08-04 00:00:00

The main power of artificial intelligence is not in modeling what we already know, but in creating solutions that are new. Such solutions exist in extremely large, high-dimensional, and complex search spaces. Population-based search techniques, i.e. variants of evolutionary computation, are well suited to finding them. These techniques make it possible to find creative solutions to practical problems in the real world, making creative AI through evolutionary computation the likely "next deep learning."
Boosting Ant Colony Optimization via Solution Prediction and Machine Learning
Authors: Yuan Sun, Sheng Wang, Yunzhuang Shen, Xiao-Dong Li, Andreas T. Ernst, Michael Kirley
Date: 2020-07-29 00:00:00
Tasks: BIG-bench Machine Learning, Combinatorial Optimization

This paper introduces an enhanced meta-heuristic (ML-ACO) that combines machine learning (ML) and ant colony optimization (ACO) to solve combinatorial optimization problems. To illustrate the underlying mechanism of our ML-ACO algorithm, we start by describing a test problem, the orienteering problem. In this problem, the objective is to find a route that visits a subset of vertices in a graph within a time budget to maximize the collected score. In the first phase of our ML-ACO algorithm, an ML model is trained using a set of small problem instances where the optimal solution is known. Specifically, classification models are used to classify an edge as being part of the optimal route, or not, using problem-specific features and statistical measures. The trained model is then used to predict the probability that an edge in the graph of a test problem instance belongs to the corresponding optimal route. In the second phase, we incorporate the predicted probabilities into the ACO component of our algorithm, i.e., using the probability values as heuristic weights or to warm start the pheromone matrix. Here, the probability values bias sampling towards favoring those predicted high-quality edges when constructing feasible routes. We have tested multiple classification models including graph neural networks, logistic regression and support vector machines, and the experimental results show that our solution prediction approach consistently boosts the performance of ACO. Further, we empirically show that our ML model trained on small synthetic instances generalizes well to large synthetic and real-world instances. Our approach integrating ML with a meta-heuristic is generic and can be applied to a wide range of optimization problems.
Integrating Variable Reduction Strategy with Evolutionary Algorithm for Solving Nonlinear Equations Systems
Authors: Aijuan Song, Guohua Wu, Witold Pedrycz
Date: 2020-07-13 00:00:00

Nonlinear equations systems (NESs) are widely used in real-world problems while they are also difficult to solve due to their characteristics of nonlinearity and multiple roots. Evolutionary algorithm (EA) is one of the methods for solving NESs, given their global search capability and an ability to locate multiple roots of a NES simultaneously within one run. Currently, the majority of research on using EAs to solve NESs focuses on transformation techniques and improving the performance of the used EAs. By contrast, the problem domain knowledge of NESs is particularly investigated in this study, using which we propose to incorporate the variable reduction strategy (VRS) into EAs to solve NESs. VRS makes full use of the systems of expressing a NES and uses some variables (i.e., core variable) to represent other variables (i.e., reduced variables) through the variable relationships existing in the equation systems. It enables to reduce partial variables and equations and shrink the decision space, thereby reducing the complexity of the problem and improving the search efficiency of the EAs. To test the effectiveness of VRS in dealing with NESs, this paper integrates VRS into two existing state-of-the-art EA methods (i.e., MONES and DRJADE), respectively. Experimental results show that, with the assistance of VRS, the EA methods can significantly produce better results than the original methods and other compared methods.
Supervised Machine Learning Techniques: An Overview with Applications to Banking
Authors: Linwei Hu, Jie Chen, Joel Vaughan, Hanyu Yang, Kelly Wang, Agus Sudjianto, Vijayan N. Nair
Date: 2020-07-28 00:00:00
Tasks: BIG-bench Machine Learning

This article provides an overview of Supervised Machine Learning (SML) with a focus on applications to banking. The SML techniques covered include Bagging (Random Forest or RF), Boosting (Gradient Boosting Machine or GBM) and Neural Networks (NNs). We begin with an introduction to ML tasks and techniques. This is followed by a description of: i) tree-based ensemble algorithms including Bagging with RF and Boosting with GBMs, ii) Feedforward NNs, iii) a discussion of hyper-parameter optimization techniques, and iv) machine learning interpretability. The paper concludes with a comparison of the features of different ML algorithms. Examples taken from credit risk modeling in banking are used throughout the paper to illustrate the techniques and interpret the results of the algorithms.
Mastering high-dimensional dynamics with Hamiltonian neural networks
Authors: Scott T. Miller, John F. Lindner, Anshul Choudhary, Sudeshna Sinha, William L. Ditto
Date: 2020-07-28 00:00:00
Tasks: Relation, Vocal Bursts Intensity Prediction

We detail how incorporating physics into neural network design can significantly improve the learning and forecasting of dynamical systems, even nonlinear systems of many dimensions. A map building perspective elucidates the superiority of Hamiltonian neural networks over conventional neural networks. The results clarify the critical relation between data, dimension, and neural network learning performance.
Accuracy of neural networks for the simulation of chaotic dynamics: precision of training data vs precision of the algorithm
Authors: S. Bompas, B. Georgeot, D. Guéry-Odelin
Date: 2020-07-08 00:00:00
Tasks: Time Series, Time Series Analysis

We explore the influence of precision of the data and the algorithm for the simulation of chaotic dynamics by neural networks techniques. For this purpose, we simulate the Lorenz system with different precisions using three different neural network techniques adapted to time series, namely reservoir computing (using ESN), LSTM and TCN, for both short and long time predictions, and assess their efficiency and accuracy. Our results show that the ESN network is better at predicting accurately the dynamics of the system, and that in all cases the precision of the algorithm is more important than the precision of the training data for the accuracy of the predictions. This result gives support to the idea that neural networks can perform time-series predictions in many practical applications for which data are necessarily of limited precision, in line with recent results. It also suggests that for a given set of data the reliability of the predictions can be significantly improved by using a network with higher precision than the one of the data.
The Cone epsilon-Dominance: An Approach for Evolutionary Multiobjective Optimization
Authors: Lucas S. Batista, Felipe Campelo, Frederico G. Guimarães, Jaime A. Ramírez
Date: 2020-07-14 00:00:00
Tasks: Diversity, Evolutionary Algorithms, Multiobjective Optimization

We propose the cone epsilon-dominance approach to improve convergence and diversity in multiobjective evolutionary algorithms (MOEAs). A cone-eps-MOEA is presented and compared with MOEAs based on the standard Pareto relation (NSGA-II, NSGA-II*, SPEA2, and a clustered NSGA-II) and on the epsilon-dominance (eps-MOEA). The comparison is performed both in terms of computational complexity and on four performance indicators selected to quantify the quality of the final results obtained by each algorithm: the convergence, diversity, hypervolume, and coverage of many sets metrics. Sixteen well-known benchmark problems are considered in the experimental section, including the ZDT and the DTLZ families. To evaluate the possible differences amongst the algorithms, a carefully designed experiment is performed for the four performance metrics. The results obtained suggest that the cone-eps-MOEA is capable of presenting an efficient and balanced performance over all the performance metrics considered. These results strongly support the conclusion that the cone-eps-MOEA is a competitive approach for obtaining an efficient balance between convergence and diversity to the Pareto front, and as such represents a useful tool for the solution of multiobjective optimization problems.
A Comparison of Machine Learning Algorithms Applied to American Legislature Polarization
Authors: Gabriel Mersy, Vincent Santore, Isaac Rand, Corrine Kleinman, Grant Wilson, Jason Bonsall, Tyler Edwards
Date: 2020-07-18 00:00:00
Tasks: BIG-bench Machine Learning, regression

We present a novel approach to the measurement of American state legislature polarization with an experimental comparison of three different machine learning algorithms. Our approach strictly relies on public data sources and open source software. The results suggest that artificial neural network regression has the best outcome compared to both support vector machine and ordinary least squares regression in the prediction of both state House and state Senate legislature polarization. In addition to the technical outcomes of our study, broader implications are assessed as a means of highlighting the importance of accessible information for the higher purpose of promoting civic responsibility.
Data-Driven Sentence Simplification: Survey and Benchmark
Authors: Fern Alva-Manchego, o, Carolina Scarton, Lucia Specia
Date: 2020-03-01 00:00:00
Proceeding: CL 2020 3
Tasks: Sentence, Survey

Sentence Simplification (SS) aims to modify a sentence in order to make it easier to read and understand. In order to do so, several rewriting transformations can be performed such as replacement, reordering, and splitting. Executing these transformations while keeping sentences grammatical, preserving their main idea, and generating simpler output, is a challenging and still far from solved problem. In this article, we survey research on SS, focusing on approaches that attempt to learn how to simplify using corpora of aligned original-simplified sentence pairs in English, which is the dominant paradigm nowadays. We also include a benchmark of different approaches on common data sets so as to compare them and highlight their strengths and limitations. We expect that this survey will serve as a starting point for researchers interested in the task and help spark new ideas for future developments.
Corpora Annotated with Negation: An Overview
Authors: Salud Mar{\'\i}a Jim{\'e}nez-Zafra, Roser Morante, Mar{\'\i}a Teresa Mart{\'\i}n-Valdivia, L. Alfonso Ure{\~n}a-L{\'o}pez
Date: 2020-03-01 00:00:00
Proceeding: CL 2020 3
Tasks: Negation, Semantic Role Labeling

Negation is a universal linguistic phenomenon with a great qualitative impact on natural language processing applications. The availability of corpora annotated with negation is essential to training negation processing systems. Currently, most corpora have been annotated for English, but the presence of languages other than English on the Internet, such as Chinese or Spanish, is greater every day. In this study, we present a review of the corpora annotated with negation information in several languages with the goal of evaluating what aspects of negation have been annotated and how compatible the corpora are. We conclude that it is very difficult to merge the existing corpora because we found differences in the annotation schemes used, and most importantly, in the annotation guidelines: the way in which each corpus was tokenized and the negation elements that have been annotated. Differently than for other well established tasks like semantic role labeling or parsing, for negation there is no standard annotation scheme nor guidelines, which hampers progress in its treatment.
Multilingual and Interlingual Semantic Representations for Natural Language Processing: A Brief Introduction
Authors: Marta R. Costa-juss{\`a}, Cristina Espa{\~n}a-Bonet, Pascale Fung, Noah A. Smith
Date: 2020-06-01 00:00:00
Proceeding: CL 2020 6
Tasks: Articles

We introduce the Computational Linguistics special issue on Multilingual and Interlingual Semantic Representations for Natural Language Processing. We situate the special issue{'}s five articles in the context of our fast-changing field, explaining our motivation for this project. We offer a brief summary of the work in the issue, which includes developments on lexical and sentential semantic representations, from symbolic and neural perspectives.
Unsupervised Word Translation with Adversarial Autoencoder
Authors: Tasnim Mohiuddin, Shafiq Joty
Date: 2020-06-01 00:00:00
Proceeding: CL 2020 6
Tasks: Machine Translation, Transfer Learning, Translation, Word Embeddings, Word Translation

Crosslingual word embeddings learned from monolingual embeddings have a crucial role in many downstream tasks, ranging from machine translation to transfer learning. Adversarial training has shown impressive success in learning crosslingual embeddings and the associated word translation task without any parallel data by mapping monolingual embeddings to a shared space. However, recent work has shown superior performance for non-adversarial methods in more challenging language pairs. In this article, we investigate adversarial autoencoder for unsupervised word translation and propose two novel extensions to it that yield more stable training and improved results. Our method includes regularization terms to enforce cycle consistency and input reconstruction, and puts the target encoders as an adversary against the corresponding discriminator. We use two types of refinement procedures sequentially after obtaining the trained encoders and mappings from the adversarial training, namely, refinement with Procrustes solution and refinement with symmetric re-weighting. Extensive experimentations with high- and low-resource languages from two different data sets show that our method achieves better performance than existing adversarial and non-adversarial approaches and is also competitive with the supervised system. Along with performing comprehensive ablation studies to understand the contribution of different components of our adversarial model, we also conduct a thorough analysis of the refinement procedures to understand their effects.
LessLex: Linking Multilingual Embeddings to SenSe Representations of LEXical Items
Authors: Davide Colla, Enrico Mensa, Daniele P. Radicioni
Date: 2020-06-01 00:00:00
Proceeding: CL 2020 6
Tasks: text similarity

We present LESSLEX, a novel multilingual lexical resource. Different from the vast majority of existing approaches, we ground our embeddings on a sense inventory made available from the BabelNet semantic network. In this setting, multilingual access is governed by the mapping of terms onto their underlying sense descriptions, such that all vectors co-exist in the same semantic space. As a result, for each term we have thus the {``}blended{''} terminological vector along with those describing all senses associated to that term. LESSLEX has been tested on three tasks relevant to lexical semantics: conceptual similarity, contextual similarity, and semantic text similarity. We experimented over the principal data sets for such tasks in their multilingual and crosslingual variants, improving on or closely approaching state-of-the-art results. We conclude by arguing that LESSLEX vectors may be relevant for practical applications and for research on conceptual and lexical access and competence.
A Systematic Study of Inner-Attention-Based Sentence Representations in Multilingual Neural Machine Translation
Authors: Ra{\'u}l V{\'a}zquez, Aless Raganato, ro, Mathias Creutz, J{\"o}rg Tiedemann
Date: 2020-06-01 00:00:00
Proceeding: CL 2020 6
Tasks: Machine Translation, Sentence, Transfer Learning, Translation

Neural machine translation has considerably improved the quality of automatic translations by learning good representations of input sentences. In this article, we explore a multilingual translation model capable of producing fixed-size sentence representations by incorporating an intermediate crosslingual shared layer, which we refer to as attention bridge. This layer exploits the semantics from each language and develops into a language-agnostic meaning representation that can be efficiently used for transfer learning. We systematically study the impact of the size of the attention bridge and the effect of including additional languages in the model. In contrast to related previous work, we demonstrate that there is no conflict between translation performance and the use of sentence representations in downstream tasks. In particular, we show that larger intermediate layers not only improve translation quality, especially for long sentences, but also push the accuracy of trainable classification tasks. Nevertheless, shorter representations lead to increased compression that is beneficial in non-trainable similarity tasks. Similarly, we show that trainable downstream tasks benefit from multilingual models, whereas additional language signals do not improve performance in non-trainable benchmarks. This is an important insight that helps to properly design models for specific applications. Finally, we also include an in-depth analysis of the proposed attention bridge and its ability to encode linguistic properties. We carefully analyze the information that is captured by individual attention heads and identify interesting patterns that explain the performance of specific settings in linguistic probing tasks.