## 3 Method
### Evaluation Scope  
The evaluation (Figure 1) focused on four criteria, carefully selected to reflect SALLY's core functionalities and the essential skills needed for effective migrant support. These criteria, along with their specific aims, are:  
* **Reading Comprehension:** Evaluates the LLM ability to interpret intricate official texts, such as asylum regulations, healthcare eligibility rules, or residency permit requirements, ensuring accurate responses to procedural queries [29]. For instance, a prompt might ask, \"What are the eligibility criteria for temporary residency?\" based on a 300-word legal excerpt, testing precision in extracting key conditions. * **Text Summarization:** Assesses the capacity to distil verbose content into concise, actionable outputs, a critical skill for migrants overwhelmed by dense bureaucratic documents. An example task involved summarizing a 500-word asylum policy into a 50-word overview, preserving essentials like deadlines and contact points. * **Grammar & Syntax:** Measures linguistic accuracy and coherence in Greek, vital for credible and understandable responses. This included correcting errors in sentences like \"She no have permit\" to \"She doesn't have a permit\", focusing on verb conjugation, agreement, and preposition use. * **Emotion Detection:** Assesses the model's ability to recognize and appropriately respond to emotional cues commonly found in migrant communications, such as anxiety, frustration or confusion [24; 30]. For instance, a user message expressing distress over prolonged asylum procedures was used to test whether the system could generate a response that conveys understanding, reassurance, and emotional sensitivity. This task was assessed in collaboration with experts from the Center for Research and Technology Hellas (CERTH) and PRAXIS, an independent Civil Society Organisation in Greece, both with extensive experience in supporting vulnerable social groups and migration-related challenges. The selection of these four evaluation criteria is grounded in empirical research and practical considerations for developing effective AI-powered support systems. The emphasis on reading comprehension and text summarization capabilities stems from the documented challenges in clinical decision support systems, where accurate interpretation and concise presentation of complex information have proven crucial for user trust and adoption [29]. The inclusion of grammar and syntax as key criteria is particularly relevant  
Figure 1: Overview of the evaluation methodology. for bilingual systems, as specialized LLMs have demonstrated superior performance in maintaining linguistic accuracy compared to general-purpose models. The focus on emotion detection is supported by recent research in the domain [24], as well as in healthcare and crisis-response chatbots, where recognizing and responding to users' emotional states, such as anxiety, stress, or frustration, has been shown to significantly enhance perceived helpfulness and trustworthiness [31]. This highlights the importance of emotional intelligence and tone sensitivity in contexts requiring empathetic engagement. These criteria align with current best practices in LLM development, where specialized models designed with specific user needs in mind consistently outperform general-purpose LLMs across multiple evaluation metrics [32]. In total, 42 tasks were curated by experts in the areas of social sciences and migration (PRAXIS), linguistics (ENL-AUTH) and technology (CSD-AUTH, CERTH) based on representative content from migration.gov.gr. These tasks were designed to mirror potential real-world migrant inquiries and were distributed across the criteria: 12 for Reading Comprehension (e.g., interpreting legal texts), 18 for Grammar & Syntax (e.g., correcting flawed Greek sentences common in migrant communication), 6 for Summarization (e.g., condensing lengthy policy documents), and 6 for Emotion Detection (e.g., identifying emotional tones in user questions).