Old Reviews, New Aspects: Aspect Based Sentiment Analysis and Entity Typing for Book Reviews with LLM s (2025)

Abstract: This paper faces the problem of the limited availability of datasets for Aspect-Based Sentiment Analysis (ABSA) in the Cultural Heritage domain. Currently, the main datasets for ABSA are product or restaurant reviews. We expand this to book reviews. Our methodology employs an LLM to maintain domain relevance while preserving the linguistic authenticity and natural variations found in genuine reviews. Entity types are annotated through the tool Text2AMR2FRED and evaluated manually. Additionally, we finetuned Llama 3.1 8B as a baseline model that not only performs ABSA, but also performs Entity Typing (ET) with a set of classes from DOLCE foundational ontology, enabling precise categorization of target aspects within book reviews. We present three key contributions as a step forward expanding ABSA: 1) a semi-synthetic set of book reviews, 2) an evaluation of Llama-3-1-Instruct 8B on the ABSA task, and 3) a fine-tuned version of Llama-3-1-Instruct 8B for ABSA.

## introduction knowledge graphs (kgs) have emerged as a fundamental framework for representing structured information extracted from diverse natural language processing (nlp) tasks @xcite . the concept of a kg encompasses everything from basic subject-predicate-object triples to complex, semantically-rich rdf graphs that adhere to semantic web standards @xcite . while numerous approaches exist for general kg extraction, specialized nlp tasks can be strategically integrated into pipelines that generate domain-specific kgs. aspect-based sentiment analysis (absa) represents one such application, enabling the creation of opinion-centric knowledge graphs where opinion holders serve as subject nodes, with the aspects they discuss and associated sentiments functioning as object nodes in the resulting graph structure @xcite . current absa research faces significant domain limitations, with datasets predominantly concentrated in two areas: restaurant and product reviews @xcite . this narrow focus creates a substantial gap in the cultural heritage (ch) domain, where opinions typically exhibit greater complexity and require specialized aspect categories and opinion frameworks. with the exception of limited book review datasets, this domain remains largely unexplored through the lens of absa. traditional absa datasets typically capture three key elements: the aspects being evaluated, the sentiments expressed toward those aspects, and the categorical classification of those aspects. to enhance the semantic richness of absa outputs, entity typing (et) can be integrated to expand the ontological coverage. this approach goes beyond identifying an aspect's contextual role in an opinion by assigning more granular type classifications. for example, in the statement: "the portrayal of levantine people in the book was colonialist" absa and et would not only identify "portrayal of levantine people" as belonging to the "topic" category but would further classify "levantine people" as a "group" (or collection, following the dolce ontology), providing deeper semantic understanding of the entities, concepts and events being discussed. in this work, we present three contributions: ## data and resources in this section, we discuss the dataset and resources. the dataset used for fine-tuning llama is available on huggingface @xcite . the code used to fetch the public data, generate the prompts for the semi-synthetic dataset, annotate the dolce types, fine-tune and evaluate the model are available in our github repository at https://github. com/aschimmenti/absa_et_book_reviews . ## evaluation the evaluation of the fine-tuned model was performed over three iterations and compared with the base instruction model. the evaluation was performed using the same bit precision (16-bit). being a generative model, the annotated dataset can only work as ground truth (gt). true positives for precision, recall and f 1 score were calculated only on matches between the model's output and the gt. the evaluation was performed three times on the test split of the dataset (2, 000 reviews). our evaluation reveals that the fine-tuned llama 3.1 8b model achieves promising performance on the challenging task of literary absa with integrated et. the model demonstrates: ## conclusions and future work in this paper, we presented three main contributions to advance aspect-based sentiment analysis in the literary domain: (1) a semi-synthetic dataset of 10.000 book reviews with aspects typed according to dolce ontology classes, (2) a comprehensive evaluation of llama 3.1-instruct 8b on this dataset, and (3) a fine-tuned model that simultaneously performs absa and entity typing. our approach addresses a large gap in ch sentiment analysis, where traditional absa datasets have focused primarily on consumer reviews (restaurants and products). by introducing a semantically rich pipeline to generate synthetic reviews, we managed to integrate the tasks of et with absa. this represents a step toward simplifying the extraction of kgs centered around opinions. the performance of our fine-tuned model (@xmath0.72 for aspect identification, @xmath1.56 for full matching) demonstrates the viability of our approach, especially considering the relatively small model size (8b parameters). the model's strong recall (0.83) indicates effective coverage of relevant aspects, while its precision (0.64) reflects the challenges of defining exact aspect boundaries in nuanced contexts. the error analysis revealed several patterns that inform future work. first, the model shows particular strength with explicit mentions of agentive elements (characters, authors) while struggling with implicit references and temporal aspects. second, sentiment classification remains a significant challenge for smaller llms, especially for aspects with mixed or nuanced sentiment expressions. building on these findings, we identify several promising directions for future research: these future directions aim to enhance both the technical capabilities and practical applications of our absa+et approach. by integrating advanced sentiment analysis with ontology-grounded entity typing, we envision a powerful framework for analyzing opinions in complex cultural contexts, supporting applications ranging from digital humanities research to automated ke from scholarly discourse.