The Generation Gap: Exploring Age Bias in the Value Systems of Large Language Models (2024)

Abstract: We explore the alignment of values in Large Language Models (LLMs) with specific age groups, leveraging data from the World Value Survey across thirteen categories. Through a diverse set of prompts tailored to ensure response robustness, we find a general inclination of LLM values towards younger demographics, especially when compared to the US population. Although a general inclination can be observed, we also found that this inclination toward younger groups can be different across different value categories. Additionally, we explore the impact of incorporating age identity information in prompts and observe challenges in mitigating value discrepancies with different age cohorts. Our findings highlight the age bias in LLMs and provide insights for future work. Materials for our analysis will be available via https://github.com/anonymous

## introduction widely used large language models (llms) should be reflective of all age groups @xcite @xcite . age statistics estimate that by 2030, 44.8% of the us population will be over 45 years old @xcite , and one in six people worldwide will be aged 60 years or over @xcite . analyzing how the values (e.g., religious values) in llms align with different age groups can enhance our understanding of the experience that users of different ages have with an llm. for instance, for an older group that may exhibit less inclination towards new technologies @xcite , an llm that embodies the values of a tech-savvy individual may lead to less empathetic interactions. minimizing the value disparities between llms and the older population has the potential to lead to better communication between these demographics and the digital products they engage with. in this paper, we investigate whether and which values in llms are more aligned with specific age groups. specifically, by using the world value survey @xcite , we prompt various llms to elicit their values on thirteen categories, employing eight format variations in prompts for robust testing. we observe a general inclination of llm values towards younger demographics, as shown in fig 1 . we also demonstrate the specific categories of value and example inquiries where llms exhibit such age preferences (see sec 4). furthermore, we study the effect of adding age identity information when prompting llms. specifically, we instruct llms to use an age and country identity before requesting their responses. surprisingly, we find that adding age identity fails to eliminate the value discrepancies with targeted age groups on eight out of thirteen categories (see fig 4 ), despite occasional success in specific instances (see sec 5). we advocate for increased awareness within the research community regarding the potential age bias inherent in llms, particularly concerning their predisposition towards certain values. we also emphasize the complexities involved in calibrating prompts to effectively address this bias. ## related work due to the rapid advancements in llms across various tasks @xcite , there is a growing concern regarding the presence of social bias in these models @xcite . recent research has shown that llms exhibit "preferences" for certain demographic groups, such as white and female individuals @xcite , and political inclination @xcite . however, the age-related preferences of llms remain less explored. prior work has mentioned age as one of multi-facets of bias in llm performance @xcite @xcite @xcite while lacking a direct study on the age aspect. recent research @xcite publishes an evaluation for well-known llms on age bias through 50 multi-choice questions; unlike it focuses on discriminatory narratives towards specific age groups, our investigation is running at an implicit level. we argue that understanding the underlying value systems is crucial, as the value discrepancies between users and llms can significantly impact their adoption of llms, even when the explicit discrimination is rectified, as exemplified in technology attitudes discussed in sec 1. ## aligning with which age on which trend observation. fig 2 exemplifies the bias for llms across six age groups in several countries. due to the limited paper pages, results on other llms and countries can be found in appx d and e. as it is not intuitive to see a bias towards younger people in these decoupled results, we summarize the performance of all llms in the us, as shown in fig 1. then we observe a general inclination of popular llms favoring the values of younger demographics in the us on different value categories, indicated by the trend coefficient. significance testing procedure is available in appx f. we observe that in the us and china, as countries with large populations, the models tend to have a higher alignment rank on younger groups on most categories, despite few exceptions (e.g., happiness and well-being). however, in ethiopia and nigeria (tab 15), the inclina-tion is less evident. we leave this phenomenon for future study. case study. in fig 3, we show two representative prompts and their responses from chatgpt and human groups, to exemplify values where chatgpt displays a clear inclination toward a specific age group. note llm values can be far away from all human age groups, as depicted in the second sub-figure. we discuss this point in appx g.2. 5 the effect of adding identity in prompts prompt adjustment. to analyze if adding age identity in the prompt helps to align values of llm with the targeted age groups, we adjust our prompts by adding a sentence like "suppose you are from [country] and your age is between [lowerbound] and [upperbound]." at the beginning of the required component of the original prompt and get responses that correspond with six age groups. observation on gap change. we illustrate the change of euclidean distance between values of llm and different age groups after adding identity information. as is presented in fig 4, in eight out of thirteen categories (no.1,2,4,5,7,8,11,12) no improvement is observed. case study. we also showcase a successful calibration example for a question about the source of acquiring information in fig 5. the value pyramid illustrates llms' responses for different age ranges compared to the answers from the u.s. population. when age is factored into the llm prompt, the llm's views are more aligned with the u.s. population of that respective age group, as it reports higher frequency using radio news for the older group. ## further discussion on the age bias observed in llms in this study, we have shown how llms are not representative of the value systems of older adults. although further validation is necessary for a solid conclusion, we believe there may be several potential harms arising from this bias: ## suggestions on age-aware alignment for future work although we have shown that llms are not representative of the value systems of older adults, our study is not intended to promote a naive copy of the values of different age groups to achieve alignment. simplistically applying statistical knowledge of the values of a particular age group might reinforce stereotypes rather than promote genuine alignment. for example, consider whether llms should adopt the value that the older generation is less tech-savvy and thus develop the stereotype that an older user would primarily obtain news from the radio rather than social media. however, as illustrated in fig 6 , while fewer older adults rely on social media for information, a significant portion still does. therefore, llms must be aware of statistical discrepancies but should avoid brute-force applying statistics to any individual, as a brute-force application often only considers the mean instead of other qualities, such as variance, outliers, and so on. thus, to facilitate a true age-aware alignment, we recommend researchers to rely on the following rules of thumb: achieving age-aware alignment requires llms to be sensitive to value differences across age groups and to build on these insights to offer helpful and harmless responses. for example, when engaging with older users, instead of brute-force assuming they are lagging behind new technology, a well-aligned system should keep tracking their understanding of the ongoing topics, offering more detailed explanations and minimizing the use of neologisms only when confusion arises. to achieve such age-sensitive interactions, exploring an effective feedback-acquiring method during interactions that complies with the real agetailored connotation of helpfulness and harmlessness is meaningful. although challenging, we believe this is a vital direction for future research. ## conclusion in this paper, we investigated the alignment of values in llms with specific age groups using data from the world value survey. our findings suggest a general inclination of llm values towards younger demographics. our study contributes to raising attention to the potential age bias in llms and advocates continued efforts from the community to address this issue. moving forward, efforts to calibrate value inclinations in llms should consider the complexities involved in prompt engineering and strive for equitable representation across diverse age groups.