Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance
Authors: name: Xueqing Peng, name: Triantafillos Papadopoulos, name: Efstathia Soufleri, name: Polydoros Giannouris, name: Ruoyu Xiang, name: Yan Wang, name: Lingfei Qian, name: Jimin Huang, name: Qianqian Xie, name: Sophia Ananiadou
Organization: The Fin AI

Despite Greece's pivotal role in the global economy, large language models
(LLMs) remain underexplored for Greek financial context due to the linguistic
complexity of Greek and the scarcity of domain-specific datasets. Previous
efforts in multilingual financial natural language processing (NLP) have
exposed considerable performance disparities, yet no dedicated Greek financial
benchmarks or Greek-specific financial LLMs have been developed until now. To
bridge this gap, we introduce Plutus-ben, the first Greek Financial Evaluation
Benchmark, and Plutus-8B, the pioneering Greek Financial LLM, fine-tuned with
Greek domain-specific data. Plutus-ben addresses five core financial NLP tasks
in Greek: numeric and textual named entity recognition, question answering,
abstractive summarization, and topic classification, thereby facilitating
systematic and reproducible LLM assessments. To underpin these tasks, we
present three novel, high-quality Greek financial datasets, thoroughly
annotated by expert native Greek speakers, augmented by two existing resources.
Our comprehensive evaluation of 22 LLMs on Plutus-ben reveals that Greek
financial NLP remains challenging due to linguistic complexity, domain-specific
terminology, and financial reasoning gaps. These findings underscore the
limitations of cross-lingual transfer, the necessity for financial expertise in
Greek-trained models, and the challenges of adapting financial LLMs to Greek
text. We release Plutus-ben, Plutus-8B, and all associated datasets publicly to
promote reproducible research and advance Greek financial NLP, fostering
broader multilingual inclusivity in finance.