# Human-Centered AI for Migrant Integration Through LLM and RAG Optimization
## 6 Results and Discussion
### Results  
The analysis of the experimental results from integrating RAG into an LLM encompassed various evaluation metrics applied across different parameter configurations. These configurations included the chunk size, the value of K, the temperature, and semantic similarity. Additionally, linguistic performance metrics such as BLEU, ROUGE-1, ROUGE-2, and ROUGE-L were considered, along with the detection of stereotypes and hate speech. The detailed results of this proof of concept are available at the following link in [71]. The results presented demonstrate a complex interplay between the RAG system's parameters and the performance metrics, underscoring the critical factors for optimizing text generation. Our analysis revealed that a diminutive chunk size (e.g., 50) coupled with an incremental K from 1 to 10 precipitated a decline in the similarity and ROUGE metrics, suggesting that the integration of context within smaller processing units may compromise requisite textual cohesion (Tables 1 and 2). This may be due to the fact that a higher K value introduces noise by incorporating an excess of contextual information which the system cannot efficiently manage in small data blocks. Conversely, when the chunk size was increased to 100, it was observed that raising K to five resulted in an improvement in similarity, indicating a balanced compromise which allowed for effective context integration without overwhelming the model with irrelevant information (Figure 2). However, when the chunk size was expanded to ranges between 300 and 1000, a high K value combined with low temperatures tended to be detrimental to performance, likely because larger data blocks, when combined with excessive contextualization, overloaded the model, adversely affecting text quality and reducing the effectiveness of the semantic relationships in the generated content (see Figure 2). In this regard, intermediate values of K (two and five) appeared to offer an optimal balance within chunk sizes from 100 to 300, maximizing performance without introducing significant levels of noise or redundancy. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline
**K** & **Temp** & **Stereo** & **Anti-Stereo** & **Neutral** & **Non\_Hate** & **Hate** & **Perplexity** & **CO\({}_{2}\)e** \\ \hline  
\hline \end{tabular}
\end{table} 
Table 1: The results present the median values based on two hyperparameters: the value of K and the temperature (Temp), using a chunk size of 50 units for easy comparisons. The values indicate the metrics, which include the presence of stereotypes (Stereo), the absence of stereotypes (Anti-Stereo), the level of neutrality (Neutral), classification in terms of non-hate speech (Non_Hate) and hate speech (Hate), perplexity (Perplexity), and the carbon footprint, in terms of equivalent CO\({}_{2}\) emissions (CO\({}_{2}\)e) (g/Wh). Figure 2: Comparative evaluation of primary assessment metrics—ROUGE-1, ROUGE-2, ROUGE-L, BLEU, and similarity—in relation to their corresponding CO\({}_{2}\)e emissions as a function of the chunk size. The yellow line representing ROUGE-2 overlaps with the line for BLEU due to the similarity of their results. Regarding temperature, it can be observed that a low value (0.1) tended to stabilize the generated text, leading to higher similarity and BLEU scores with lower values of \"K\". This indicates a more controlled process which is less susceptible to irrelevant random deviations. In contrast, a temperature of 0.5 offered a balance between the model's accuracy and creativity, though it exhibited some volatility in configurations with larger chunk sizes. The results suggest that configurations with larger chunk sizes (such as 300 or 500), lower K values (two or five), and lower temperatures (0.1 or 0.5) tend to produce improved scores in metrics such as the BLEU, similarity, and certain ROUGE metrics while maintaining relatively lower computational costs and CO2e emissions.