BARTABSA ++: Revisiting BARTABSA with Decoder LLM s (2025)

Abstract: We revisit the BARTABSA framework for aspect-based sentiment analysis with modern decoder LLMs to assess the importance of explicit structure modeling today. Our updated implementation - BARTABSA++ - features architectural enhancements that boost performance and training stability.Systematic testing with various encoder-decoder architectures shows that BARTABSA++ with BART-Large achieves state-of-the-art results, even surpassing a finetuned GPT-4o model.Our analysis indicates the encoder’s representational quality is vital, while the decoder’s role is minimal, explaining the limited benefits of scaling decoder-only LLMs for this task. These findings underscore the complementary roles of explicit structured modeling and large language models, indicating structured approaches remain competitive for tasks requiring precise relational information extraction.

## introduction in this work, we revisit bartabsa @xcite , a pointer network-based method for aspect-based sentiment analysis (absa) that achieved state-of-the-art performance with the bart encoder-decoder transformer -considered small by today's standards. bartabsa models aspect sentiment triplet extraction (aste) @xcite as a constrained generation task, using a pointer network @xcite to explicitly copy input tokens into strictly structured outputs. in the meantime, advances in large language models (llms) have transformed natural language processing (nlp) by demonstrating that implicit knowledge acquired during pretraining can often address tasks that previously demanded explicit structured representations @xcite . this shift prompts the question: is explicit structured output modeling still these authors contributed equally to this work. 1 https://github.com/lsx-uniwue/bartabsa-plusplus relevant in the llm era, especially in structured representation-critical tasks like absa? to evaluate this, we employ the methodology of @xcite , which constructs encoderdecoder architectures by reusing pretrained encoder or decoder checkpoints. we reimplement bartabsa using modern libraries and incorporate architectural enhancements like: (1) feature normalization to balance embedding spaces and stabilize training @xcite ; (2) cross-attention mechanisms reusing weights from bart's decoder layers @xcite ; with this enhanced framework, we systematically evaluate different architectural configurations to examine potential scaling effects of modernsized llms within structured language modeling. this research direction is particularly valuable as it combines the implicit knowledge of modern decoder llms with the transparency offered by the explicit copying mechanism of pointer networksa property important for interpretable nlp systems. experiments with bart @xcite , bert @xcite , and gpt-2 @xcite variants reveal performance hinges on the encoder, with minimal decoder impact. counterintuitively, scaling gpt-2 does not yield performance gains, emphasizing the encoder's pretraining importance for pointer networks (and other encoder-decoder architectures). our study shows large autoregressive models do not universally surpass structured approaches in nlp and contributes to the dialogue on explicit structure modeling, like pointer networks, in the drivers updated okay but the bios update froze the system . llm era. it explores scaling structured generation with llm-sized models to merge their implicit knowledge with structured copying mechanisms. ## methodology we will combine yan et al.'s (2021a) bartabsa framework, with rothe et al.'s (2020) encoderdecoder model construction methodology in section 3.3. this requires us to first reimplement bartabsa in section 3.1, before stabilizing training with additional features and optimizations in section 3.2. ## experiments we evaluate our approaches on four popular se-meval absa benchmark datasets: 14lap, 14res, 15res, and 16res @xcite @xcite , using the refined version from @xcite and commonly used tuple-level metrics @xcite . our backbone models are bart-base and bart-large, with full implementation details in appendix a. we compare our results against the original bartabsa @xcite and the state-ofthe-art by @xcite , who achieve their results using a mixture of sequence tagging and sequence generation. to assess how the advancing capabilities of llms might impact the aste task, we additionally establish an llm-based baseline by finetuning openai's gpt-4o model 4 on each absa dataset. using the prompt template shown in appendix b, we train for 3 epochs with consistent hyperparameters across all datasets. this (methodologically) very simple baseline achieves an average f 1 score of 70.95, roughly on par with the current sota results. in particular, the llm-based approach exhibits a distinctive pattern of trading precision for recall, with significantly higher recall (74.94) compared to other methods. ## conclusion in this work, we revisited the bartabsa framework in the context of modern decoder llms. our enhanced implementation-bartabsa++demonstrates that explicit structured models remain highly competitive for tasks requiring precise extraction of relational information, even outperforming a finetuned gpt-4o model. our systematic experiments reveal a fundamental insight: while our architectural enhancements enable effective scaling with pretrained encoderdecoder models like bart, this scaling behavior does not transfer encoder-decoder models based on decoder-only llms. the critical factor appears to be the encoder's representational quality at the token level, which decoder llms struggle to match when repurposed as encoders. these findings highlight the complementary nature of structured approaches and large language models in nlp. while llms excel at tasks leveraging their implicit knowledge, structured pointer networks seem to be able to provide superior performance and interpretability for precise relational extraction tasks.