Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance
Authors: name: Xueqing Peng; name: Triantafillos Papadopoulos; name: Efstathia Soufleri; name: Polydoros Giannouris; name: Ruoyu Xiang; name: Yan Wang; name: Lingfei Qian; name: Jimin Huang; name: Qianqian Xie; name: Sophia Ananiadou
Published: 2025-02-26 03:04:01

Abstract: Despite Greece's pivotal role in the global economy, large language models
(LLMs) remain underexplored for Greek financial context due to the linguistic
complexity of Greek and the scarcity of domain-specific datasets. Previous
efforts in multilingual financial natural language processing (NLP) have
exposed considerable performance disparities, yet no dedicated Greek financial
benchmarks or Greek-specific financial LLMs have been developed until now. To
bridge this gap, we introduce Plutus-ben, the first Greek Financial Evaluation
Benchmark, and Plutus-8B, the pioneering Greek Financial LLM, fine-tuned with
Greek domain-specific data. Plutus-ben addresses five core financial NLP tasks
in Greek: numeric and textual named entity recognition, question answering,
abstractive summarization, and topic classification, thereby facilitating
systematic and reproducible LLM assessments. To underpin these tasks, we
present three novel, high-quality Greek financial datasets, thoroughly
annotated by expert native Greek speakers, augmented by two existing resources.
Our comprehensive evaluation of 22 LLMs on Plutus-ben reveals that Greek
financial NLP remains challenging due to linguistic complexity, domain-specific
terminology, and financial reasoning gaps. These findings underscore the
limitations of cross-lingual transfer, the necessity for financial expertise in
Greek-trained models, and the challenges of adapting financial LLMs to Greek
text. We release Plutus-ben, Plutus-8B, and all associated datasets publicly to
promote reproducible research and advance Greek financial NLP, fostering
broader multilingual inclusivity in finance.