From Text to Context: Contextualizing Language with Humans, Groups, and Communities for Socially Aware NLP (2024)

Abstract: Aimed at the NLP researchers or practitioners who would like to integrate human - individual, group, or societal level factors into their analyses, this tutorial will cover recent techniques and libraries for doing so at each level of analysis. Starting with human-centered techniques that provide benefit to traditional document- or word-level NLP tasks (Garten et al., 2019; Lynn et al., 2017), we undertake a thorough exploration of critical human-level aspects as they pertain to NLP, gradually moving up to higher levels of analysis: individual persons, individual with agent (chat/dialogue), groups of people, and finally communities or societies.

## type of tutorial the tutorial will introduce research that has successfully integrated personal and social factors into traditional nlp as a foundation for cutting-edge research in the field. this multidisciplinary work has not been presented at prior *cl tutorials and is timely, given recent excitement in the *cl community for human-aware nlp systems. unique aspects of this tutorial will include 1) interdisciplinary methods woven together into a coherent framework for human-centered nlp, 2) theory and domain expertise from an interdisciplinary team of presenters, and 3) hands-on demonstrations that facilitate immediate uptake and application by at-tendees foot_0 . ## target audience & pre-requisites our intended audience for this tutorial is experienced as well as upcoming nlp researchers looking to add human and social contexts to traditional nlp tasks. we expect this tutorial will attract 70-100 attendees. we expect that attendees will arrive with a practical baseline knowledge of machine learning and computational linguistics. specifically, we anticipate that our audience will be familiar with transformer based nlp models, and canonical tasks that the field has been applied to such as: document classification, stance detection, etc. ## outline introduction (15 minutes) the 3 hour tutorial will begin with a brief overview of the entire session organized from the individualto the societal levels of context. we will also introduce the key concepts in behavioral and social science that motivate the techniques that will be discussed in the subsequent sections. ## reading list owing to the diverse nature of the sessions and the presenters' backgrounds, about two-thirds of the materials will encompass contemporary research works from other teams, with the other third coming from our works for this tutorial @xcite @xcite @xcite . ## diversity considerations we are an interdisciplinary team composed of computer scientists and psychologist across 3 institutions. we intend to leverage multiple levels of expertise to be accessible to an audience with varied fluency. we have 4 highly-experienced researchers (3 professors, 1 data scientist at nih) and 5 rising researchers (each with one or more *cl publications). presenters span multiple demographics, ethnicities, and non-neurotypical backgrounds. this tutorial is aimed at encouraging more human-aware nlp systems through the incorporation of personal, demographic and cultural attributes of the speaker. salvatore giorgi is a senior data scientist for the national institute of drug abuse and the world well being project at university of pennsylvania. his research focuses on multi-level nlp and bias mitigation. webpage: https://sjgiorgi. github.io/ joão sedoc is an assistant professor in the department of technology, operations and statistics at new york university stern school of business. joão's research areas are at the intersection of machine learning and natural language processing. his interests include conversational agents, model evaluation, deep learning, and crowdsourcing. webpage: https://stern.nyu.edu/ faculty/bio/joao-sedoc h. andrew schwartz is an associate professor at stony brook university and director of the human language analysis lab. his research focuses on interdisciplinary human-centered nlp, publishing in both computational linguistics and psychological science venues. webpage: https: //www3.cs.stonybrook.edu/~has/ ryan l. boyd is a psychologist and computational social scientist. his research uses behavioral science methods to understand how verbal behavior provides clues to how we think, feel, and behave, focusing on domains ranging from personality to society, mental health, human sexuality, and storytelling (e.g., boyd et al., 2015, 2020). webpage: https://www.ryanboyd.io adithya v ganesan is a computer science phd student at the stony brook university, with research focusing on building nlp systems for psychological applications. webpage: https: //adithya8.github.io siddharth mangalik is a computer science phd student at stony brook university. his research work focuses on methods for examining the language of large-scale communities across time. webpage: https://smangalik.github.io/ vasudha varadarajan is a computer science phd student at stony brook university. her research focuses on using discourse-level nlp for understanding cognitive styles, and also on improving language-based mental health assessments. webpage: https://vasevarad.github.io nikita soni is a computer science phd student at stony brook university. her research focuses on large language modeling in the additional context of the human behind the language. webpage: https://www3.cs.stonybrook.edu/~nisoni/ swanie juhng is a computer science phd student at stony brook university. her research focuses on developing nlp and ml systems to understand the context of psychological conditions. webpage: https://swaniejuhng.github.io ## ethics statement as with most human centered nlp tasks, one must carefully consider issues of privacy and consent, as well as social context and unintended downstream applications. human level data, which encompasses text as well as non-linguistic data such as self-reports (surveys or health records, for example) and inferred factors (such as language-based estimates of gender or personality), may contain sensitive or identifying information. thus, care must be taken when collecting, storing, and analyzing data, as well as presenting results (e.g., directly quoting text), in order to not publicize private data or identify individuals. for example, reddit forums are often self-moderated intimate communities where users may anonymously discuss private and sensitive details related to, among others, mental and physical health, substance use and recovery, and parenting. identifying personal accounts in such contexts may be especially harmful to individuals @xcite . similarly, many studies which use publicly available social media data are classified as not involving human subjects and exempt from institutional review board approval. thus, the humans behind the social media accounts do not explicitly consent to research studies @xcite . there are also ethical issues around inferring human factors using nlp or machine learning methods. common tasks such as inferring sociodemographics can suffer from limited representation in data sets (sample biases) or narrow definitions of social constructs (e.g., binary gender). misclassifications can have unintended downstream consequences which, as more automated systems are deployed in real world situations, are becoming increasingly consequential @xcite . many algorithms designed to address such issues and remove biases often further marginalize vulnerable groups @xcite . on the other hand, incorporating human factors may help alleviate biases. for example, when removing selection biases from population-level estimates one must know the socio-demographics of the people within the sample. in the current context, for example, this could mean estimating human factors, such as age and income, at scale across millions of twitter users. dialog agents, as another example, can run the risk of mimicking the social and cultural biases in their training data. thus, forcing diverse ranges of human factors on agents may make them more diverse. given this range of concerns, addressing ethical issues will be woven into each section of the tutorial.