Sentiment Analysis of Nakba Oral Histories: A Critical Study of Large Language Models (2025)

Abstract: This study explores the use of Large Language Models (LLMs), specifically ChatGPT, for sentiment analysis of Nakba oral histories, which document the experiences of Palestinian refugees. The study compares sentiment analysis results from full testimonies (average 2500 words) and their summarized versions (300 words). The findings reveal that summarization increased positive sentiment and decreased negative sentiment, suggesting that the process may highlight more hopeful themes while oversimplifying emotional complexities. The study highlights both the potential and limitations of using LLMs for analyzing sensitive, trauma-based narratives and calls for further research to improve sentiment analysis in such contexts.

## introduction the nakba, meaning "catastrophe" in arabic, refers to the forced displacement and dispossession of palestinians during the 1948 occupation by the israeli colonial power, a pivotal moment that reshaped the social, cultural, and political landscape of palestine and the middle east @xcite . narratives surrounding the nakba are deeply embedded in the collective memory of palestinian communities, transmitted through generations via oral histories, personal testimonies, and cultural expressions @xcite . these narratives, rich in emotional depth, political significance, and historical context, have become essential language resources for understanding the human impact of this traumatic event @xcite . however, the challenge of interpreting, preserving, and analyzing such complex narratives has intensified in the age of artificial intelligence (ai) and large language models (llms), which have emerged as powerful tools for analyzing text at scale @xcite . sentiment analysis, a key subfield of natural language processing (nlp), offers the potential to systematically evaluate the emotional content of texts @xcite , thereby providing insights into the emotional and psychological dimensions of the nakba narratives. by applying sentiment analysis to nakba oral histories, we gain the ability to quantify and explore emotions like grief, loss, resistance, and resilience within these stories @xcite . however, while llms have demonstrated exceptional capabilities in processing and analyzing vast datasets, they are not without their limitations, particularly when applied to sensitive, culturally charged, and historically complex narratives such as those related to the nakba @xcite . these models often struggle with the subtleties of language, historical context, and the lived experiences embedded in oral histories, raising questions about their adequacy in capturing the full emotional and cultural resonance of such narratives @xcite . this paper critically examines the application of sentiment analysis to nakba oral histories, exploring both the potential and the pitfalls of using llms to analyze such narratives. it argues that while llms can offer valuable insights into the emotional tone of texts, they must be used with caution, considering the unique cultural and historical context of the nakba. the study positions nakba narratives as crucial language resources, emphasizing their role in shaping collective memory and identity, while also critiquing the limitations of ai tools in fully capturing the intricacies of human experience and emotion. by doing so, the paper highlights the importance of interdisciplinary approaches-combining computational methods with cultural sensitivitywhen applying ai-driven tools to historically significant and emotionally complex language resources such as nakba oral histories. ## analysis and results in this study, sentiment analysis was conducted on both the full and summarized versions of ten nakba oral testimonies, with the results broken down into positive, neutral, and negative sentiment categories for each individual testimony. the following analysis provides a comparison between the sentiment distribution of the full testimonies and their corresponding summaries, along with a critical assessment of the findings. ## discussion the results of the sentiment analysis on the full and summarized nakba oral histories offer valuable insights into the capabilities and limitations of llms such as chatgpt. specifically, the observed shifts in sentiment-particularly the increase in positive sentiment and decrease in negative sentiment in the summarized versions-raise important questions about how llms process, condense, and represent emotional content in complex narratives. this discussion will explore what these shifts reveal about the behavior of llms in sentiment analysis and summarization tasks. the increase in positive sentiment in the summarized testimonies, compared to the full versions, can be attributed to several factors inherent in the summarization process. in the context of the nakba testimonies, this shift could reflect the llm's tendency to neglect aspects such as fear, sad, or grief. instead, llms emphasized positive emotions and hope, which are central to the palestinian refugee experience but may not be as prominent in the more traumatic or sorrowful details of the longer narratives. since summarization inherently involves reducing the complexity of emotional expression, llms might emphasize elements that allow for a more coherent and cohesive portrayal, possibly skewing the sentiment toward the positive end of the spectrum. moreover, llms like chatgpt are trained on vast datasets with a significant amount of positive, optimistic language. this bias could influence the model to unintentionally highlight positive aspects of the narrative, even if those sentiments are less central in the full testimony. the summarization process could amplify this tendency, resulting in summaries that appear more positive in sentiment, even when the original narrative is emotionally complex or predominantly negative. the reduction in negative sentiment observed in the summarized testimonies can also be understood through the lens of llm behavior. negative emotions, particularly those related to trauma, grief, and loss, are often more nuanced and detailed in the full testimonies. when tasked with summarizing these narratives, the llm might omit or condense the detailed expressions of pain, anger, or sorrow to meet the constraints of brevity and focus on key events or themes. in some cases, the model may unintentionally downplay the intensity of negative emotions by rephrasing or generalizing painful experiences. this could occur due to the model's propensity to avoid overly dramatic language or its tendency to reduce emotional complexity due to low resources of the nakba and palestinian narrative in the training dataset of llms. furthermore, negative sentiments that are less immediately apparent or require more context might be excluded from summaries, leading to a shift in sentiment toward the neutral or positive end of the spectrum. it is important to note that while the summarization process may reduce the explicit negativity in the text, this does not necessarily reflect a change in the underlying emotional experience of the refugees. instead, it highlights the limitations of llms in capturing the full emotional depth of complex, traumatic narratives. in reducing the complexity of the text, the model may inadvertently present a version of the testimony that appears less negative, even if the full testimony conveys a much more emotionally charged story. ## conclusion this study aimed to evaluate the performance and behavior of llms in performing sentiment analysis on nakba oral histories, with a particular focus on how summarization of complex, emotionally charged narratives influences sentiment results. the primary objective was to examine whether llms, specifically chatgpt, could accurately capture and analyze the emotional tone of testimonies from palestinian refugees, and how summarization might affect the representation of these sentiments. the contribution of this work lies in its novel application of llms to sensitive historical narratives, providing valuable insights into the potential and limitations of these models for sentiment analysis in emotionally complex contexts. the study highlights several limitations. first, the sentiment analysis conducted by chatgpt may have been influenced by biases in the model's training data, leading to a misrepresentation of the emotional tone in the palestinian refugee testimonies. additionally, the summarization process itself likely oversimplified the complexity of the full narratives, which may have distorted the sentiment analysis results. these limitations point to the need for more specialized models capable of understanding the cultural and historical contexts of sensitive narratives like the nakba. the future work will also compare different llms to see if the conclusions hold across them.