m S tyle D istance: Multilingual Style Embeddings and their Evaluation (2025)

Abstract: Style embeddings are useful for stylistic analysis and style transfer, yet they only exist for English. We introduce Multilingual StyleDistance (mStyleDistance), a method that can generate style embeddings in new languages using synthetic data and a contrastive loss. We create style embeddings in nine languages and a multilingual STEL-or-Content benchmark (Wegmann et al., 2022) that serves to assess their quality. We also employ our embeddings in an authorship verification task involving different languages. Our results show that mStyleDistance embeddings outperform existing style embeddings on these benchmarks and generalize well to unseen features and languages. We make our models and datasets publicly available.

## introduction style embedding models seek to embed texts with similar style closer in the embedding space regardless of their content. style embeddings are useful for tasks like style transfer and authorship attribution, but only exist for english @xcite . multilingual style embeddings could also serve to automatically evaluate style preservation in machine translation. models like xlm-roberta @xcite and e5 @xcite create multilingual representations for semantic tasks, but have not addressed style mainly due to the scarcity of style datasets. we propose a procedure, called multilingual styledistance (mstyledistance), to train style embeddings using contrastive learning with synthetic data in multiple languages. early work on style representations learning often involved unlabeled social media data @xcite @xcite et al., 2023), but patel et al. ( 2024b) showed that a contrastive learning objective with synthetic examples (sentence pairs with similar content and different style) can generate high quality style representations for english. we create msynthstel, a synthetic dataset of paraphrases addressing various style features in nine languages, and use it to create our multilingual style embeddings. in order to evaluate their quality, we contribute a new multilingual and cross-lingual stel-or-content (soc) evaluation benchmark which, following the original soc evaluation task @xcite , measures the ability of a model to embed sentences with the same style closer in the embedding space than sentences with the same content. we show that mstyledistance embeddings outperform other representations in these evaluations, and demonstrate their usefulness in a downstream setting addressing a multilingual authorship verification task. we publicly release our model, data, and evaluation benchmarks. ## multilingual synthetic data we extend the @xcite dataset to nine languages (l): arabic, german, spanish, french, hindi, japanese, korean, russian, and chinese. 1 style feature selection we use the set of 40 features addressed in @xcite , leaving out features not applicable to specific languages. for example, articles are not relevant for chinese and japanese so the corresponding features have been discarded. our set of features (f ) includes syntactic features (e.g., active/passive voice, contractions, frequent use of function words), emotional and cognitive features (e.g., words indicating sentiment or cognitive processes), stylistic and aesthetic features (e.g., metaphors, formal tone), social and interpersonal features (e.g., polite or offensive tone), graphical and digital features (e.g., capitalization, emojis, numerical digits), temporal and aspectual features (e.g., focus on present or future). a full list of the features that were considered for each language and details about the features that were omitted are given in appendix a. ## training mstyledistance following the contrastive training approach of @xcite , we construct feature-specific triples for each language @xmath0: an anchor text (a); a text with the same style as a but different content (pos); a distractor text (neg) which is a paraphrase of a or pos, but different in style from a. we use the multilingual xlm-roberta-base as our base model and train with a triplet loss @xcite . we ensure half of our triplets are cross-lingual, i.e. the pos and neg texts are randomly sampled from a different language than the anchor text. our triplet creation process ensures equal coverage of the languages we support. full training details can be found in appendix e. we also construct a cross-lingual soc benchmark that addresses embeddings' ability to capture style similarity across languages. this can be useful, for example, to evaluate style preservation in translations. we construct the benchmark with the xformal dataset @xcite , ## conclusion we introduced a novel approach to learning multilingual style embeddings from synthetic examples, and contribute a two benchmarks for evaluating the quality of multilingual style representations. we show that mstyledistance embeddings are able to distinguish style from content better than other english and multilingual embeddings, and generalize well to unseen features and languages. the authorship verification evaluation shows that mstyledistance embeddings also offer strong performance on multilingual downstream tasks.