pwc-archive/papers-with-abstracts huggingface dataset:
Text Generation with Text-Editing Models
Authors: Eric Malmi, Yue Dong, Jonathan Mallinson, Aleksandr Chuklin, Jakub Adamek, Daniil Mirylenka, Felix Stahlberg, Sebastian Krause, Shankar Kumar, Aliaksei Severyn
Date: 2022-06-14 00:00:00
Proceeding: NAACL (ACL) 2022 7
Tasks: Grammatical Error Correction, Hallucination, Style Transfer, Text Generation

Text-editing models have recently become a prominent alternative to seq2seq models for monolingual text-generation tasks such as grammatical error correction, simplification, and style transfer. These tasks share a common trait - they exhibit a large amount of textual overlap between the source and target texts. Text-editing models take advantage of this observation and learn to generate the output by predicting edit operations applied to the source sequence. In contrast, seq2seq models generate outputs word-by-word from scratch thus making them slow at inference time. Text-editing models provide several benefits over seq2seq models including faster inference speed, higher sample efficiency, and better control and interpretability of the outputs. This tutorial provides a comprehensive overview of text-editing models and current state-of-the-art approaches, and analyzes their pros and cons. We discuss challenges related to productionization and how these models can be used to mitigate hallucination and bias, both pressing challenges in the field of text generation.
Computational linguistics and Natural Language Processing
Authors: Saturnino Luz
Date: 2022-06-14 00:00:00
Tasks: Translation

This chapter provides an introduction to computational linguistics methods, with focus on their applications to the practice and study of translation. It covers computational models, methods and tools for collection, storage, indexing and analysis of linguistic data in the context of translation, and discusses the main methodological issues and challenges in this field. While an exhaustive review of existing computational linguistics methods and tools is beyond the scope of this chapter, we describe the most representative approaches, and illustrate them with descriptions of typical applications.
An Experimental Investigation of Part-Of-Speech Taggers for Vietnamese
Authors: Tuan-Phong Nguyen, Quoc-Tuan Truong, Xuan-Nam Nguyen, Anh-Cuong Le
Date: 2022-06-14 00:00:00
Tasks: Chunking, Dependency Parsing, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Part-Of-Speech Tagging, POS, POS Tagging

Part-of-speech (POS) tagging plays an important role in Natural Language Processing (NLP). Its applications can be found in many NLP tasks such as named entity recognition, syntactic parsing, dependency parsing and text chunking. In the investigation conducted in this paper, we utilize the technologies of two widely-used toolkits, ClearNLP and Stanford POS Tagger, as well as develop two new POS taggers for Vietnamese, then compare them to three well-known Vietnamese taggers, namely JVnTagger, vnTagger and RDRPOSTagger. We make a systematic comparison to find out the tagger having the best performance. We also design a new feature set to measure the performance of the statistical taggers. Our new taggers built from Stanford Tagger and ClearNLP with the new feature set can outperform all other current Vietnamese taggers in term of tagging accuracy. Moreover, we also analyze the affection of some features to the performance of statistical taggers. Lastly, the experimental results also reveal that the transformation-based tagger, RDRPOSTagger, can run significantly faster than any other statistical tagger.
OSN Dashboard Tool For Sentiment Analysis
Authors: Andreas Kilde Lien, Lars Martin Randem, Hans Petter Fauchald Taralrud, Maryam Edalati
Date: 2022-06-14 00:00:00
Tasks: Classification, Sentiment Analysis, Sentiment Classification

The amount of opinionated data on the internet is rapidly increasing. More and more people are sharing their ideas and opinions in reviews, discussion forums, microblogs and general social media. As opinions are central in all human activities, sentiment analysis has been applied to gain insights in this type of data. There are proposed several approaches for sentiment classification. The major drawback is the lack of standardized solutions for classification and high-level visualization. In this study, a sentiment analyzer dashboard for online social networking analysis is proposed. This, to enable people gaining insights in topics interesting to them. The tool allows users to run the desired sentiment analysis algorithm in the dashboard. In addition to providing several visualization types, the dashboard facilitates raw data results from the sentiment classification which can be downloaded for further analysis.
The Maximum Linear Arrangement Problem for trees under projectivity and planarity
Authors: Lluís Alemany-Puig, Juan Luis Esteban, Ramon Ferrer-i-Cancho
Date: 2022-06-14 00:00:00

A linear arrangement is a mapping $\pi$ from the $n$ vertices of a graph $G$ to $n$ distinct consecutive integers. Linear arrangements can be represented by drawing the vertices along a horizontal line and drawing the edges as semicircles above said line. In this setting, the length of an edge is defined as the absolute value of the difference between the positions of its two vertices in the arrangement, and the cost of an arrangement as the sum of all edge lengths. Here we study two variants of the Maximum Linear Arrangement problem (MaxLA), which consists of finding an arrangement that maximizes the cost. In the planar variant for free trees, vertices have to be arranged in such a way that there are no edge crossings. In the projective variant for rooted trees, arrangements have to be planar and the root of the tree cannot be covered by any edge. In this paper we present algorithms that are linear in time and space to solve planar and projective MaxLA for trees. We also prove several properties of maximum projective and planar arrangements, and show that caterpillar trees maximize planar MaxLA over all trees of a fixed size thereby generalizing a previous extremal result on trees.
CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generation
Authors: Daoguang Zan, Bei Chen, Dejian Yang, Zeqi Lin, Minsu Kim, Bei guan, Yongji Wang, Weizhu Chen, Jian-Guang Lou
Date: 2022-06-14 00:00:00
Tasks: Code Generation, Library-Oriented Code Generation

Code generation is a longstanding challenge, aiming to generate a code snippet based on a natural language description. Usually, expensive text-code paired data is essential for training a code generation model. Recently, thanks to the success of pre-training techniques, large language models are trained on large-scale unlabelled code corpora and perform well in code generation. In this paper, we investigate how to leverage an unlabelled code corpus to train a model for library-oriented code generation. Since it is a common practice for programmers to reuse third-party libraries, in which case the text-code paired data are harder to obtain due to the huge number of libraries. We observe that library-oriented code snippets are more likely to share similar code sketches. Hence, we present CERT with two steps: a sketcher generates the sketch, then a generator fills the details in the sketch. Both the sketcher and the generator are continually pre-trained upon a base model using unlabelled data. Furthermore, we craft two benchmarks named PandasEval and NumpyEval to evaluate library-oriented code generation. Experimental results demonstrate the impressive performance of CERT. For example, it surpasses the base model by an absolute 15.67% improvement in terms of pass@1 on PandasEval. Our work is available at https://github.com/microsoft/PyCodeGPT.
"hasSignification()": une nouvelle fonction de distance pour soutenir la détection de données personnelles
Authors: Amine Mrabet, Ali Hassan, Patrice Darmon
Date: 2022-06-14 00:00:00
Tasks: Attribute

Today with Big Data and data lakes, we are faced of a mass of data that is very difficult to manage it manually. The protection of personal data in this context requires an automatic analysis for data discovery. Storing the names of attributes already analyzed in a knowledge base could optimize this automatic discovery. To have a better knowledge base, we should not store any attributes whose name does not make sense. In this article, to check if the name of an attribute has a meaning, we propose a solution that calculate the distances between this name and the words in a dictionary. Our studies on the distance functions like N-Gram, Jaro-Winkler and Levenshtein show limits to set an acceptance threshold for an attribute in the knowledge base. In order to overcome these limitations, our solution aims to strengthen the score calculation by using an exponential function based on the longest sequence. In addition, a double scan in dictionary is also proposed in order to process the attributes which have a compound name.
Computing Real Numbers with Large-Population Protocols Having a Continuum of Equilibria
Authors: Xiang Huang, Rachel N. Huls
Date: 2022-06-14 00:00:00

Bournez, Fraigniaud, and Koegler defined a number in [0,1] as computable by their Large-Population Protocol (LPP) model, if the proportion of agents in a set of marked states converges to said number over time as the population grows to infinity. The notion, however, restricts the ordinary differential equations (ODEs) associated with an LPP to have only finitely many equilibria. This restriction places an intrinsic limitation on the model. As a result, a number is computable by an LPP if and only if it is algebraic, namely, not a single transcendental number can be computed under this notion. In this paper, we lift the finitary requirement on equilibria. That is, we consider systems with a continuum of equilibria. We show that essentially all numbers in [0,1] that are computable by bounded general-purpose analog computers (GPACs) or chemical reaction networks (CRNs) can also be computed by LPPs under this new definition. This implies a rich series of numbers (e.g., the reciprocal of Euler's constant, $\pi/4$, Euler's $\gamma$, Catalan's constant, and Dottie number) are all computable by LPPs. Our proof is constructive: We develop an algorithm that transfers bounded GPACs/CRNs into LPPs. Our algorithm also fixes a gap in Bournez et al.'s construction of LPPs designed to compute any arbitrary algebraic number in [0,1].
FreeTransfer-X: Safe and Label-Free Cross-Lingual Transfer from Off-the-Shelf Models
Authors: Yinpeng Guo, Liangyou Li, Xin Jiang, Qun Liu
Date: 2022-06-14 00:00:00
Proceeding: Findings (NAACL) 2022 7
Tasks: Cross-Lingual Transfer, Diagnostic, Knowledge Distillation, Machine Translation, NMT, Transfer Learning

Cross-lingual transfer (CLT) is of various applications. However, labeled cross-lingual corpus is expensive or even inaccessible, especially in the fields where labels are private, such as diagnostic results of symptoms in medicine and user profiles in business. Nevertheless, there are off-the-shelf models in these sensitive fields. Instead of pursuing the original labels, a workaround for CLT is to transfer knowledge from the off-the-shelf models without labels. To this end, we define a novel CLT problem named FreeTransfer-X that aims to achieve knowledge transfer from the off-the-shelf models in rich-resource languages. To address the problem, we propose a 2-step knowledge distillation (KD, Hinton et al., 2015) framework based on multilingual pre-trained language models (mPLM). The significant improvement over strong neural machine translation (NMT) baselines demonstrates the effectiveness of the proposed method. In addition to reducing annotation cost and protecting private labels, the proposed method is compatible with different networks and easy to be deployed. Finally, a range of analyses indicate the great potential of the proposed method.
CHQ-Summ: A Dataset for Consumer Healthcare Question Summarization
Authors: Shweta Yadav, Deepak Gupta, Dina Demner-Fushman
Date: 2022-06-14 00:00:00
Tasks: Community Question Answering, Descriptive, Natural Language Understanding, Question Answering

The quest for seeking health information has swamped the web with consumers' health-related questions. Generally, consumers use overly descriptive and peripheral information to express their medical condition or other healthcare needs, contributing to the challenges of natural language understanding. One way to address this challenge is to summarize the questions and distill the key information of the original question. To address this issue, we introduce a new dataset, CHQ-Summ that contains 1507 domain-expert annotated consumer health questions and corresponding summaries. The dataset is derived from the community question-answering forum and therefore provides a valuable resource for understanding consumer health-related posts on social media. We benchmark the dataset on multiple state-of-the-art summarization models to show the effectiveness of the dataset.
Hate Speech and Counter Speech Detection: Conversational Context Does Matter
Authors: Xinchen Yu, Eduardo Blanco, Lingzi Hong
Date: 2022-06-13 00:00:00
Proceeding: NAACL 2022 7

Hate speech is plaguing the cyberspace along with user-generated content. This paper investigates the role of conversational context in the annotation and detection of online hate and counter speech, where context is defined as the preceding comment in a conversation thread. We created a context-aware dataset for a 3-way classification task on Reddit comments: hate speech, counter speech, or neutral. Our analyses indicate that context is critical to identify hate and counter speech: human judgments change for most comments depending on whether we show annotators the context. A linguistic analysis draws insights into the language people use to express hate and counter speech. Experimental results show that neural networks obtain significantly better results if context is taken into account. We also present qualitative error analyses shedding light into (a) when and why context is beneficial and (b) the remaining errors made by our best model when context is taken into account.
Generating Exact Optimal Designs via Particle Swarm Optimization: Assessing Efficacy and Efficiency via Case Study
Authors: Stephen J. Walsh, John J. Borkowski
Date: 2022-06-14 00:00:00

In this study we address existing deficiencies in the literature on applications of Particle Swarm Optimization to generate optimal designs. We present the results of a large computer study in which we bench-mark both efficiency and efficacy of PSO to generate high quality candidate designs for small-exact response surface scenarios commonly encountered by industrial practitioners. A preferred version of PSO is demonstrated and recommended. Further, in contrast to popular local optimizers such as the coordinate exchange, PSO is demonstrated to, even in a single run, generate highly efficient designs with large probability at small computing cost. Therefore, it appears beneficial for more practitioners to adopt and use PSO as tool for generating candidate experimental designs.
Development of a hybrid method for stock trading based on TOPSIS, EMD and ELM
Authors: Elivelto Ebermam, Helder Knidel, Renato A. Krohling
Date: 2022-06-14 00:00:00

Deciding when to buy or sell a stock is not an easy task because the market is hard to predict, being influenced by political and economic factors. Thus, methodologies based on computational intelligence have been applied to this challenging problem. In this work, every day the stocks are ranked by technique for order preference by similarity to ideal solution (TOPSIS) using technical analysis criteria, and the most suitable stock is selected for purchase. Even so, it may occur that the market is not favorable to purchase on certain days, or even, the TOPSIS make an incorrect selection. To improve the selection, another method should be used. So, a hybrid model composed of empirical mode decomposition (EMD) and extreme learning machine (ELM) is proposed. The EMD decomposes the series into several sub-series, and thus the main omponent (trend) is extracted. This component is processed by the ELM, which performs the prediction of the next element of component. If the value predicted by the ELM is greater than the last value, then the purchase of the stock is confirmed. The method was applied in a universe of 50 stocks in the Brazilian market. The selection made by TOPSIS showed promising results when compared to the random selection and the return generated by the Bovespa index. Confirmation with the EMD-ELM hybrid model was able to increase the percentage of profit tradings.
A novel MDPSO-SVR hybrid model for feature selection in electricity consumption forecasting
Authors: Yukun Bao, Liang Shen, Xiaoyuan Zhang, Yanmei Huang, Changrui Deng
Date: 2022-06-14 00:00:00
Tasks: feature selection

Electricity consumption forecasting has vital importance for the energy planning of a country. Of the enabling machine learning models, support vector regression (SVR) has been widely used to set up forecasting models due to its superior generalization for unseen data. However, one key procedure for the predictive modeling is feature selection, which might hurt the prediction accuracy if improper features were selected. In this regard, a modified discrete particle swarm optimization (MDPSO) was employed for feature selection in this study, and then MDPSO-SVR hybrid mode was built to predict future electricity consumption. Compared with other well-established counterparts, MDPSO-SVR model consistently performs best in two real-world electricity consumption datasets, which indicates that MDPSO for feature selection can improve the prediction accuracy and the SVR equipped with the MDPSO can be a promised alternative for electricity consumption forecasting.
A Local Optima Network Analysis of the Feedforward Neural Architecture Space
Authors: Isak Potgieter, Christopher W. Cleghorn, Anna S. Bosman
Date: 2022-06-02 00:00:00

This study investigates the use of local optima network (LON) analysis, a derivative of the fitness landscape of candidate solutions, to characterise and visualise the neural architecture space. The search space of feedforward neural network architectures with up to three layers, each with up to 10 neurons, is fully enumerated by evaluating trained model performance on a selection of data sets. Extracted LONs, while heterogeneous across data sets, all exhibit simple global structures, with single global funnels in all cases but one. These results yield early indication that LONs may provide a viable paradigm by which to analyse and optimise neural architectures.
Two-Timescale Stochastic Approximation for Bilevel Optimisation Problems in Continuous-Time Models
Authors: Louis Sharrock
Date: 2022-06-14 00:00:00

We analyse the asymptotic properties of a continuous-time, two-timescale stochastic approximation algorithm designed for stochastic bilevel optimisation problems in continuous-time models. We obtain the weak convergence rate of this algorithm in the form of a central limit theorem. We also demonstrate how this algorithm can be applied to several continuous-time bilevel optimisation problems.
Fast Computation of Highly G-optimal Exact Designs via Particle Swarm Optimization
Authors: Stephen J. Walsh, John J. Borkowski
Date: 2022-06-13 00:00:00

Computing proposed exact $G$-optimal designs for response surface models is a difficult computation that has received incremental improvements via algorithm development in the last two-decades. These optimal designs have not been considered widely in applications in part due to the difficulty and cost involved with computing them. Three primary algorithms for constructing exact $G$-optimal designs are presented in the literature: the coordinate exchange (CEXCH), a genetic algorithm (GA), and the relatively new $G$-optimal via $I_\lambda$-optimality algorithm ($G(I_\lambda)$-CEXCH) which was developed in part to address large computational cost. Particle swarm optimization (PSO) has achieved widespread use in many applications, but to date, its broad-scale success notwithstanding, has seen relatively few applications in optimal design problems. In this paper we develop an extension of PSO to adapt it to the optimal design problem. We then employ PSO to generate optimal designs for several scenarios covering $K = 1, 2, 3, 4, 5$ design factors, which are common experimental sizes in industrial experiments. We compare these results to all $G$-optimal designs published in last two decades of literature. Published $G$-optimal designs generated by GA for $K=1, 2, 3$ factors have stood unchallenged for 14 years. We demonstrate that PSO has found improved $G$-optimal designs for these scenarios, and it does this with comparable computational cost to the state-of-the-art algorithm $G(I_\lambda)$-CEXCH. Further, we show that PSO is able to produce equal or better $G$-optimal designs for $K= 4, 5$ factors than those currently known. These results suggest that PSO is superior to existing approaches for efficiently generating highly $G$-optimal designs.
An analysis of retracted papers in Computer Science
Authors: Martin Shepperd, Leila Yousefi
Date: 2022-06-14 00:00:00

Context: The retraction of research papers, for whatever reason, is a growing phenomenon. However, although retracted paper information is publicly available via publishers, it is somewhat distributed and inconsistent. Objective: The aim is to assess: (i) the extent and nature of retracted research in Computer Science (CS) (ii) the post-retraction citation behaviour of retracted works and (iii) the potential impact on systematic reviews and mapping studies. Method: We analyse the Retraction Watch database and take citation information from the Web of Science and Google scholar. Results: We find that of the 33,955 entries in the Retraction watch database (16 May 2022), 2,816 are classified as CS, i.e., approximately 8.3%. For CS, 56% of retracted papers, provide little or no information as to the reasons. This contrasts with 26% for other disciplines. There is also a remarkable disparity between different publishers, a tendency for multiple versions of a retracted paper over and above the Version of Record (VoR), and for new citations long after a paper is officially retracted. Conclusions: Unfortunately retraction seems to be a sufficiently common outcome for a scientific paper that we as a research community need to take it more seriously, e.g., standardising procedures and taxonomies across publishers and the provision of appropriate research tools. Finally, we recommend particular caution when undertaking secondary analyses and meta-analyses which are at risk of becoming contaminated by these problem primary studies.
Memory-Oriented Design-Space Exploration of Edge-AI Hardware for XR Applications
Authors: Vivek Parmar, Syed Shakib Sarwar, Ziyun Li, Hsien-Hsin S. Lee, Barbara De Salvo, Manan Suri
Date: 2022-06-08 00:00:00
Tasks: CPU, Hand Detection, Quantization

Low-Power Edge-AI capabilities are essential for on-device extended reality (XR) applications to support the vision of Metaverse. In this work, we investigate two representative XR workloads: (i) Hand detection and (ii) Eye segmentation, for hardware design space exploration. For both applications, we train deep neural networks and analyze the impact of quantization and hardware specific bottlenecks. Through simulations, we evaluate a CPU and two systolic inference accelerator implementations. Next, we compare these hardware solutions with advanced technology nodes. The impact of integrating state-of-the-art emerging non-volatile memory technology (STT/SOT/VGSOT MRAM) into the XR-AI inference pipeline is evaluated. We found that significant energy benefits (>=24%) can be achieved for hand detection (IPS=10) and eye segmentation (IPS=0.1) by introducing non-volatile memory in the memory hierarchy for designs at 7nm node while meeting minimum IPS (inference per second). Moreover, we can realize substantial reduction in area (>=30%) owing to the small form factor of MRAM compared to traditional SRAM.