Cartography of Natural Language Processing for Social Good ( NLP 4 SG ): Searching for Definitions, Statistics and White Spots (2021)

Abstract: The range of works that can be considered as developing NLP for social good (NLP4SG) is enormous. While many of them target the identification of hate speech or fake news, there are others that address, e.g., text simplification to alleviate consequences of dyslexia, or coaching strategies to fight depression. However, so far, there is no clear picture of what areas are targeted by NLP4SG, who are the actors, which are the main scenarios and what are the topics that have been left aside. In order to obtain a clearer view in this respect, we first propose a working definition of NLP4SG and identify some primary aspects that are crucial for NLP4SG, including, e.g., areas, ethics, privacy and bias. Then, we draw upon a corpus of around 50,000 articles downloaded from the ACL Anthology. Based on a list of keywords retrieved from the literature and revised in view of the task, we select from this corpus articles that can be considered to be on NLP4SG according to our definition and analyze them in terms of trends along the time line, etc. The result is a map of the current NLP4SG research and insights concerning the white spots on this map.

## introduction measuring the social impact of nlp is not a trivial task. a priori, the range of works that can be considered as developing nlp for social good (nlp4sg) is enormous. it goes from more theoretical works @xcite , language resources @xcite and models @xcite to concrete technologies of which many target the identification of hate speech @xcite or fake news @xcite . but there are also others that address, e.g., text simplification or paraphrasing, which can be used to alleviate consequences of dyslexia @xcite , conversational agents for mental health treatment @xcite , or elearning applications, which support students with specific learning disabilities @xcite . in general, many nlp technologies can be used for good but also for bad; at a larger scale, they may affect the lives of many people, and it is difficult to predict in the first place all the potential positive or negative sides resulting from the application of these technologies. in order to discard at this stage uncontrolled "collateral" positive or negative technology influence, we can assume that social good does not come as a side effect when researching certain fields and developing technologies. even more: if we do not address directly, measure and intentionally promote and control social good, we can cause more harm than good. therefore, it is of paramount importance to define what we mean when we say "nlp for social good", what aspects of peoples' lives are improved by nlp4sg and how, and what suitable strategies are to promote and measure the impact of technological solutions related to nlp. however, so far, there is no clear picture of what areas are targeted by nlp4sg, who are the actors, which are the main scenarios and what are the topics that have been left aside. in this paper, we discuss what nlp for social good (nlp4sg) is, and how we can promote the development of more socially positive technologies. the contribution of this paper is twofold: the remainder of this paper is structured as follows. section 2 defines nlp4sg and introduces some other central aspects of it -the applications, collaboration, and ethics. section 3 details the data, methodology, and results of our evaluation of the social impact in the nlp field. section 4 elaborates on how to improve the current state of affairs. section 5 addresses the limitations and ethical concerns, and section 6, finally, summarizes the implications of our work and draws some conclusions. ## defining nlp4sg before we set out to provide an overview of the nlp4sg research and explore its characteristics, we need to define what we mean by nlp4sg. let us start by analysing what is "social good". in the context of social science, barak (2020) proposes a conceptual "social good" model according to which there are three elements needed to promote social good: innovative technologies, social good domains, and engaging unconventional systems of change, which in this work we also refer to as "collaborations". in the following subsections, we focus on each of these dimensions and dig into other nlp4sg related aspects. ## evaluation of social good in nlp evaluating the current state of nlp for social good is a crucial step towards the identification of the gaps and promotion of a more impactful technology development. for this purpose, we build upon the nlp scholar dataset (mohammad, 2020) and analyse existent features together with new classifications on social good aspects. in what follows, we describe in detail the data and the procedure of our analysis. we make the code available to the community 1 . ## improving the current state of affairs as shown in the previous section, the nlp field is recently more attentive to social good related issues. nevertheless, we do believe that there are certain aspects that need further attention by the community. in what follows, we enumerate these aspects, along with some hints on how to address them. metrics explicit nlp4sg non-explicit nlp4sg total number of papers in percentage 9.63% 90.37% average number of google scholar citations per paper 25.87 42.03 from the total authors publishing in nlp 24.02% 75.98% mean number of authors per paper 3.65 2.97 more than social good areas. although we follow previous research in an attempt to measure nlp4sg by matching keywords of certain areas @xcite , we must be cautious when look-ing at the obtained results: while research in a certain area may imply social good, it may also imply social harm, depending on how a certain technology is going to be used (e.g., a fake news detector may be used to detect, but also to generate fake news). another analysis over the same data may aid to interpret the achieved results and help to understand whether the approaches of the previous work to address social good areas lead to positive or negative outcomes. with this in mind, and for the sake of a broader analysis of research impact, we include into our consideration other nlp4sg dimensions such as ethics, social good terms and systems of change and collaboration. as a guideline for future research, we may conclude that providing data statements and terms of use for the developed technologies would help preventing potential misuses. other social good dimensions. when we look at the explicit nlp4sg, papers from the considered areas are more frequent than papers related to other dimensions, and it is only in recent years that other dimensions-related papers are increasing in number and have more weight. we believe that research in nlp would benefit from a wider discussion on social good dimensions such as ethics, positive impact and collaborations. in particular, questions such as how the development of nlp applications may involve end-users and include knowledge about their context of use require more attention. social good should not be the researcher's enemy. we show that explicit nlp4sg publications tend to have less citations in average and are published in smaller venues. the reduced number of authors of explicit nlp4sg papers suggests that there is a smaller nlp4sg community within the larger nlp community. we believe that it is urgent to actively encourage research to tackle social good areas, and, in particular, also to promote the formation of an interconnected social good community across different academic disciplines. pushing towards this objective will benefit both the field and the society that is impacted by the technology that we produce. ## limitations and ethical concerns as mentioned in section 3.2, our keyword matching approach to the identification of nlp publications as being relevant to nlp4sg is robust and performative enough only for a subset of the publications, namely those that contain one of the keywords that we are looking for. the term explicit nlp4sg intends to highlight this limitation. it is possible (or even likely) that it misses some papers tackling nlp4sg in a more subtle manner. this means that the results presented in this paper serve as a lower bound baseline. as far as the collection of the used keyword is concerned, we started by using an initial sample of terms specifically conceived for artificial intelligence. we tried then to add some nlp related expressions and remove terms that were bringing misleading results. however, in the course of the presented analysis it became clear that a more systematic method could have revealed more social good nlp related terms. furthermore, when discussing definitions of social good, we should bear in mind that what is considered to be a "positive impact" depends on the context and set of values. for instance, ethical concerns and guidelines are different according to different countries @xcite and are not absent of social and political interests (washington and kuo, 2020). as a consequence, we must acknowledge the limitation of our analysis in this regard since we follow an eurocentric perspective and focus only on acl publications. ## conclusions the goals of this paper have been to help to draw a clearer picture of what nlp4sg is and where we stand in the current state of nlp. we established working definitions of nlp4sg and identified some aspects that are crucial for the analysis of nlp publications with respect to their relevance to nlp4sg, namely technologies, areas, collaborations. nlp-specific ethical aspects formed another perspective of our analysis. we drew upon the acl anthology corpus and annotated papers in terms of explicit vs. non-explicit nlp4sg to show a clearer view of the evolution of the field. we identified social good-relevant nlp areas with less research, as well as other social good dimensions that are important to address, and proposed a non-exhaustive list of aspects that need further attention by the community. the results of the research in nlp have a huge impact on the whole society, and we strongly believe that it is urgent for the community to potentiate and encourage research that not only includes ethical consideration, but also actively addresses social good. tecnologia (fct), within the scope of operational program human capital (poch), supported by the european social fund and by national funds from mctes. the work of the juan soler-company and leo wanner has been supported by the european commission in the context of the h2020 research program under the contract numbers 700024 and 786731.