## 5 Approaches  
During the experiments with the datasets described in Section 4, we tested a wide variety of approaches. These approaches were selected based on several criteria: (1) their ability to support all 58 languages of interest present in our dataset; (2) the goal to evaluate diverse types of methods, including trainable/fine-tunable, memory-based, and generative models, ensuring that at least one representative from each category would be tested:  
* _BERT+CNN_. Utilizing the BERT (Bidirectional Encoder Representations from Transformers) model [35] for vectorization, in conjunction with a CNN (Convolutional Neural Network) classifier [36], presents multiple benefits. BERT effectively encodes the semantics of individual words within their contextual environments. These encoded word representations are subsequently input into the CNN. The CNN's role is to delineate features from the n-gram patterns of these embeddings, enabling the identification of keywords or phrases within the text. This capability is particularly advantageous for topic detection. Our CNN's structure is similar to that shown in Figure 3 of [37]. For BERT, the bert-base-multilingual-cased model was chosen ([[https://huggingface.co/bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased)]([https://huggingface.co/bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased)) (accessed on 2 May 2022)), which supports 104 languages and is case-sensitive. The BERT+CNN training involved tuning approximately 735 thousand parameters, suggesting that this method could either solve our current problem effectively or establish a strong baseline. * _XLM-R_fine_tuning_. For our classification problem, we employed the XLM-R model as described in [38], making several targeted adjustments. All layers of the model were unfrozen, rendering approximately 278 million parameters trainable. Specifically, we utilized the xlm-roberta-base model ([[https://huggingface.co/xlm-roberta-base](https://huggingface.co/xlm-roberta-base)]([https://huggingface.co/xlm-roberta-base](https://huggingface.co/xlm-roberta-base)) (accessed on 3 March 2023)), a multilingual variant that supports 100 languages. * _LaBSE+FFNN_. The LaBSE model [34], a vectorizer, was used in combination with an FFNN classifier. Unlike BERT embeddings, which provide word-level representations, LaBSE specializes in sentence-level embeddings, vectorizing entire texts into single aggregated vectors. This approach effectively handles varying sentence structures with identical semantic meanings, making it ideal for languages featuring flexible sentence structures. The LaBSE model ([[https://huggingface.co/sentence-transformers/LaBSE](https://huggingface.co/sentence-transformers/LaBSE)]([https://huggingface.co/sentence-transformers/LaBSE](https://huggingface.co/sentence-transformers/LaBSE)) (accessed on 3 March 2023)) supports 109 languages and is both multilingual and crosslinkingal. The sentence vectors generated by LaBSE are input into the FFNN. The FFNN's architecture and hyperparameters were optimized using Hyperas ([[https://github.com/maxpumperla/hyperas](https://github.com/maxpumperla/hyperas)]([https://github.com/maxpumperla/hyperas](https://github.com/maxpumperla/hyperas)) (accessed on 3 March 2023)) and Hyperopt, which are Python libraries for hyperparameter optimization. We employed the Tree-structured Parzen Estimator (TPE) optimization algorithm, conducting 200 optimization trials to determine optimal settings, including discrete parameters like neuron counts and activation functions, continuous parameters like dropout rates, and conditional parameters such as additional FFNN layers. The training process required adjustments to approximately 1.3 thousand parameters. * _LaBSE_fine_tuning_. This process entailed two primary steps: (1) unfreezing all layers of LaBSE and optimizing all parameters, and (2) introducing an additional layer specifically tailored to our classification problem.