Examining and Adapting Time for Multilingual Classification via Mixture of Temporal Experts (2025)

Abstract: Time is implicitly embedded in classification process: classifiers are usually built on existing data while to be applied on future data whose distributions (e.g., label and token) may change. However, existing state-of-the-art classification models merely consider the temporal variations and primarily focus on English corpora, which leaves temporal studies less explored, let alone under multilingual settings. In this study, we fill the gap by treating time as domains (e.g., 2024 vs. 2025), examining temporal effects, and developing a domain adaptation framework to generalize classifiers over time on four languages, English, Danish, French, and German. Our framework proposes Mixture of Temporal Experts (MoTE) to leverage both semantic and data distributional shifts to learn and adapt temporal trends into classification models. Our analysis shows classification performance varies over time across different languages, and we experimentally demonstrate that MoTE can enhance classifier generalizability over temporal data shifts. Our study provides analytic insights and addresses the need for time-aware models that perform robustly in multilingual scenarios.

## introduction data, and therefore classification models built on the data, can change over time. written styles in social media platforms can change rapidly @xcite , and token and label distributions can shift over long periods of time @xcite . large language models @xcite (llms) pre-trained on large amount of data collections have assisted document classifiers capturing precise semantic representations of documents and achieving state-of-the-art performance on the classification task @xcite . common ways to train and evaluate classification models depend on randomly splitting data into training, development, and test sets. however, the train-ing and evaluation setting rarely considers and adapts the temporal nature of classification models --llms and classifiers are built on existing data while to be applied on future data subjecting to change. thus, a notable yet not solved question in the llm era is: does the temporal shifts effect classification model performance, and if so, how can we minimize the impacts? however, examining and encountering the impacts of temporal shifts on classification performance remain understudied. to estimate the impacts, studies developed classification models on news articles @xcite , legal documents @xcite , or medical notes @xcite ) by comparing performance variations across time intervals of yearly or multiple years. while retraining classifiers by new data and labels is straightforward @xcite @xcite , a substantial gap may prevent the method: labels of future data not be available at the time of model development. in addition, the existing studies @xcite @xcite primarily work on english corpora, little is known about if the principles are applicable to non-english languages. thus, a notable question yet unanswered is: can the principles of temporal shifts on english data and classifiers be applicable to other languages? in this study, we focus on a standard evaluation task, text classification, and answer the questions by 1) examining temporal shifts and their impacts on state-of-the-art classification models, 2) developing a domain adaptation approach, and 3) evaluate performance impacts on the gender groups under multilingual settings. specifically, we introduce a mixture of temporal experts (mote) by leveraging data shifts across time domains and developing a dynamic routing network to allow swiftly generalize classification models over time via two major modules: clustering-based shift evaluator and tem- the multilingual setting has rarely been studied particularly for the temporality and its impacts on classification models. we further conduct ablation studies to understand different mote modules and show that the dynamic routing mechanism plays the most critical role in promoting performance over time. our findings provide new insights and recommendations for both researchers and practitioners on the essential to consider time in designing and deploying models on time-varying corpora under the multilingual scenario. foot_0 ## data to better measure and understand time and its impacts on classification models, we select data from two distinct domains: user review data and legal documents. this selection enables us to compare between short, informal text (social media reviews) and longer, formal text (legal documents). legal data legal documents present a unique challenge due to their temporal nature, making models particularly susceptible to performance degradation over time. the eurlex dataset @xcite is a multilingual and multi-label dataset comprising legal documents from 23 european languages. the documents are annotated with eurovoc concepts (labels) by the publication office of eu. we use the first levels of the eurovoc taxonomy in our experiments, encompassing 21 concepts. prior studies have shown that temporal effects in eurlex significantly impact classification performance @xcite . we follow the original data split, with a consistent temporal division across all languages: training on documents from 1958 to 2010 and testing on documents from 2012 to 2016. we take english (en), spanish (es), croatian (hr) and maltese (mt) as representative examples, with their statistics shown in review data we retrieve a review data @xcite spanning years between 2007 and 2014 and covering four language corpora, danish (da), english (en), french (fr), and german (de). 2 the data sources user reviews from trustpilot website and contains 5-scale ratings, timestamps, and binary gender values (female and male). for each corpus, we divide the data into four time intervals, remove reviewers fewer than 10 tokens, and only keep data entries with complete user gender information, which aim to ensure data quality and integrity. ## mixture of temporal experts (mote) in this section, we introduce a multi-source domain adaptation approach, mixture of temporal experts (mote) in figure 2 , which leverages temporal data shift and mixture of temporal experts. the mote method contains two modules: clusteringbased shift evaluator and temporal router network. the clustering-based shift evaluator measures data shift by the difference between the data representation of input data and the established cluster centroids. the temporal router network module consists of a router and multiple experts. the router is trained to dispatch the data representations to experts based on it's distance to the cluster centroids. and then the experts integrate the data shift vectors produced by the shift evaluator before passing to the classification layer. finally, the predictor will take the time-aware representations for predictions. problem definition for each language corpus, we hold out the most recent data as target time domain data d target and the rest as source time domain d source , and divide d source to t time domains by time order: d t , @xmath0, ..., t }. the source model θ source is the model fine-tuned on source domain data.the performance decline when the source model θ source is tested on the target domain data d target indicates that the source model struggles to generalize effectively to the target data, and thus requires adaptation method to mitigate performance degradation. our goal is to develop a method that can adapt θ source to the target domain, which can be aware of the data shifts between d source and d target and improve model's performance on the target data, in which we achieve by the two major modules, clustering-based shift evaluator and temporal router network. ## experiments to examine the effectiveness of our proposed mixture of temporal experts (mote) method, we conduct experiments to compare our method with the source model (fine-tuned on source time domain) and state-of-the-art baselines on the target time domain data. we set our experimental result against three established baseline methods: self-labeling @xcite , anti-cf @xcite . we set the source model as a comparison to see whether the method improves the model's adaptation to the target domain. our assessment includes both performance evaluation and fairness evaluation. to validate domaingeneralizability of our mote method under multilingual setting, we also conduct experiments on the legal data eurlex @xcite containing 23 languages and compare our method with the best-performing baseline @xcite . in addition, our work includes an ablation study focusing on the proposed mote method, where we removed each component of our method. by assessing the performance resulting from the removal of each component, we measured its contribution to the overall effectiveness. ## results we present the macro-f1 score and auc along with fairness evaluation of the review data in and the danish corpus is the most gender-balanced corpus, with lowest gender-imbalance ratio of 1.14. in the legal domain, mote also shows substantial improvements in low-resource languages, such as maltese, where f1-ma improves by 11.9 percentage points, composes a relative increase of 36.9%. these may indicate: 1) mote is particularly effective for classimbalanced datasets (e.g. german corpus). the use of a mixture of temporal experts enhances predictions for minority classes, which are more impactful in the macro-f1 score. analysis 4. why anti-cf and self-labeling failed? we observe that methods like anti-cf and self-labeling lead to large degrees of performance decline compare to source model, which means the adaptation failed. these approaches depend heavily on the source model's outputs on the target data: anti-cf uses the source model's output to regularize the adapted model, while self-labeling relies on the source model's labels on the target data to train the new model. a possible explanation for this performance decline is that such reliance can introduce noise, particularly when there is substantial data shift in the minority classes. in this case, the source model's incorrect predictions may amplify errors in these classes, leading to reduced macro-f1 and macro-auc scores. this could explain why the performance decline is more pronounced in language corpora where the source model already exhibits lower macro-f1 and macro-auc scores, such as french and german. ## conclusion our study suggests that time can significantly impact classification performance and gender fairness under the multilingual setting, and model deployments should be aware of the temporal effects may vary across different languages and gender groups. the proposed mote approach outperform the stateof-the-art classifiers and time-aware baselines by a large margin and maintain gender fairness, as demonstrated both in review and lagal domain. our ablation analysis suggests that while the temporal expert and shifting evaluator modules both contribute to the performance improvements, the temporal expert module acts a more critical role, verifying our initial motivation. finally, the findings also provide a recommendation for practitioners following the insights above that it is important to slice test entries from future or new samples of any time-varying data, instead of only randomly slicing samples as a test set.