Large Language Models as S ocio T echnical Systems (2023)

Abstract: The expectation of Large Language Models (LLMs) to solve various societal problems has ignored the larger socio-technical frame of reference under which they operate. From a socio-technical perspective, LLMs are necessary to look at separately from other ML models as they have radically different implications in society never witnessed before. In this article, we ground Selbst et al.(2019)’s five abstraction traps – The Framing Trap, The Portability Trap, The Formalism Trap, The Ripple Effect Trap and the Solutionism Trap in the context of LLMs discussing the problems associated with the abstraction and fairness of LLMs. Through learnings from previous studies and examples, we discuss each trap that LLMs fall into, and propose ways to address the points of LLM failure by gauging them from a socio-technical lens. We believe the discussions would provide a broader perspective of looking at LLMs through a sociotechnical lens and our recommendations could serve as baselines to effectively demarcate responsibilities among the various technical and social stakeholders and inspire future LLM research.

## introduction machine learning's allied fields like natural language processing and computer vision have been thriving on abstraction to achieve powerful generalisation -by delineating the surface form from generalised patterns through neural network and transformer based approximation functions. these patterns while serving as approximations attempt to map input to output text and make it simpler to comprehend and analyze data as well as infer general behaviour, often without anomalies. specifically large language models (llms)' abstractive nature helps represent the essential characteristics of large pieces of text @xcite without including all of its specific details. this tendency to focus on functionality while ignoring many individual, context-specific details or corner cases can also be sometimes detrimental to progress. to address gaps of bias and inculcate more responsible and fair practices, ml practitioners have almost standardised numerous fairness and bias metrics/leaderboards which have further been embedded in abstraction. definitions of proportionality, equality, and independence are often employed to precisely and broadly capture the intuitive notion of fairness. due to inherent abstraction, many of these definitions fall short of accounting the specific social context in which the ml models would be deployed @xcite . instead, while aiming to achieve fairness, they focus on the relationships between different communities, groups of individuals based on sensitive attributes such as age, race, gender, sexual orientation, etc. and model predictions for those individuals. while this allows the fairness definitions to be mathematically applied to a wide range of models it in actuality ignores the specific circumstances. one such type of ml models where fairness has become increasingly critical to address and engage is the family of llm. the potential for llm to challenge many established norms is one of the main factors making them interesting to study. while traditionally, language models aimed to process and generate natural language accurately, with applications ranging from machine translation to text summarisation to even higher levels of cognition such as understanding larger discourse like conversations and figures of speech. post the mainstreaming of transformers @xcite , llms are rarely attributed to attempting to cater only to linguistic tasks. much of their success has been extended beyond language related tasks -essentially and arguably, any type of data with sequential properties like speech, music, etc. does not appear too hard to model in theory given sufficient data and compute power @xcite . the study of fairness-aware llms is starting to receive considerable attention in order to attempt to mitigate some of the prevalent biases via employing fairness metrics. a plethora of fairness metrics, such as demographic parity, equal opportunity @xcite and predictive parity are commonly used to evaluate language models @xcite . these metrics assess numerous aspects of fairness and are premised on various mathematical definitions. demographic parity, for example, considers the overall distribution of outcomes across different communities, whereas equal opportunity focuses on outcomes for individuals who belong to a specific sensitive group, such as those of a certain race or gender. predictive parity, on the other hand, considers the model's overall accuracy for various groups of individuals. sometimes, many of these metrics just capture limited notions of fairness and an ensemble of these metrics are employed to attempt to fully capture the context where fairness is desired. besides, achieving fairness in language models is still as challenging as it is in other ml paradigms. apart from the lack of consensus over the definitions of fairness, fairness is frequently at odds with other goals, such as model performance and accuracy and sometimes even at odds with legal concepts of fairness themselves @xcite leading to researchers ignoring aspects of fairness. selbst et al. (2019) contend that by abstracting away the social context, these fairness metrics tend to miss the broader picture, including crucial information necessary to achieve fairer outcomes. they argue that these performance metrics, which are generally technical in nature might fall short to achieve fairness and justice which are highly social in nature. while abstract and contextual concepts like fairness and justice are properties of social and legal systems, technical systems are subsystems, and hence to treat fairness (and justice) devoid of social context is to make a category error or an abstraction error @xcite . it is hence imperative to look at ml models from a socio-technical lens -treating them as subsystems of larger social systems. @xcite further explicate this abstraction error in terms of five failure modes -framing trap, portability trap, formalism trap, ripple effect trap and solutionism trap and argue for viewing these models as socio-technical lens. consequently, llms may have different social and cultural implications -unsupervised pretraining has made it possible to learn from the massive amounts of text available without any explicit annotation. such rapid scale of generalisation is unique to llms. language models are unsurprisingly used towards building crucial high social impact applications, like news summariseriation, legal guidance @xcite , as virtual assistants @xcite @xcite @xcite , science writing, health and medical consulation @xcite etc. besides, llms are not as easy to train as they are to use. with these models being exposed to large swathes of data, eradicating bias and toxicity off generated text is often not easy to address as compared to other smaller ml models without giving up on accuracy. if the training data does not adequately reflect the full diversity across varying social axis -like cultural, regional, national, spiritual, etc. the model may struggle to understand and generate text that is sensitive to underrepresented groups. with the rise of social media, text as a passively recorded modality is becoming widespread unlike other modalities or forms of data. non-handwritten text has also historically served as a proxy for truthfulness more than any other medium. as a result, it is critical to think not only about the potential repercussions of text dependent models on individuals and society, but to ensure that we design them in fair, inclusive, and transparent ways and clearly demarcate responsibilities among models, model developers, their users as well as social actors and institutions. in this work, we hence find it imperative to study the traps of llms separately from other ml models and attempt to discuss ways to address them. our focus is specifically on grounding selbst et al. (2019)'s abstraction traps in the context of llms. ## the abstraction traps our contributions in this paper are as follows: ## conclusion the field of large language models (llms) is rapidly advancing, furthering the prediction of outcomes that were previously unpredictable or considered exclusively under the domain of human expertise. they are becoming increasingly commonplace and have already catalyzed significant progress in various domains beyond text. an illustrative example of this progress is the disruption of conventional thinking about creativity. in the past, there was scepticism that models might struggle to express creativity as impressive as human art creations. however, recent successes have given rise to ai art models that challenge these assumptions, ushering in a new era of commercial artistry -redefining the boundaries of human-machine collaboration @xcite . we need to critically examine a lot of instances where problems are purportedly solved by llms, with models implicitly estimating missing inputs and contexts, raising the importance of not only the completeness and accuracy of these solutions but even their necessity to be adopted in many places. we established @xcite 's abstraction traps in the context of large language models. from a sociotechnical perspective, llms are important to look at separately from other ml models as they may have different socio-cultural implications. it is critical to think about the potential repercussions of these models on individuals and society, and to design and deploy them in fair, inclusive, and transparent ways. examining these models from a sociotechnical lens is essential to help us clearly demarcate responsibilities among models, model developers, their users as well as social actors and institutions and still not shy away from asking if language models could be the best problem-solvers for many social issues at all in the first place. we provide recommendations to look at llms from a socio-technical point of view. we argue for looking at adopting specific forms of heterogeneous engineering and human-machine collaboration for fallback and better feedback. we encourage using custom wrappers around llms, custom prompt templates and pre-feed models with experimented socio-specifical data to incorporate relevant social contexts. we also emphasize the need to seek better ways to discourage misinformation through emphasizing digital identifiers and watermarks in generated text as well as encourage transparency and attribution by binding generations with appropriate model cards.