ICLEF : In-Context Learning with Expert Feedback for Explainable Style Transfer (2024)

Abstract: While state-of-the-art large language models (LLMs) can excel at adapting text from one style to another, current work does not address the explainability of style transfer models. Recent work has explored generating textual explanations from larger teacher models and distilling them into smaller student models. One challenge with such approach is that LLM outputs may contain errors that require expertise to correct, but gathering and incorporating expert feedback is difficult due to cost and availability. To address this challenge, we propose ICLEF, a novel human-AI collaboration approach to model distillation that incorporates scarce expert human feedback by combining in-context learning and model self-critique. We show that our method leads to generation of high-quality synthetic explainable style transfer datasets for formality (E-GYAFC) and subjective bias (E-WNC). Via automatic and human evaluation, we show that specialized student models fine-tuned on our datasets outperform generalist teacher models on the explainable style transfer task in one-shot settings, and perform competitively compared to few-shot teacher models, highlighting the quality of the data and the role of expert feedback. In an extrinsic task of authorship attribution, we show that explanations generated by smaller models fine-tuned on E-GYAFC are more predictive of authorship than explanations generated by few-shot teacher models.

## introduction attribute style transfer is the task of transforming a given text along a particular style dimension, such as changing its formality, bias, or level of offensiveness @xcite @xcite . formality style transfer (e.g., @xmath0, while neutralizing text that contains subjective bias would be an important tool for wikipedia editors @xcite or journalists @xcite . style transfer approaches have primarily focused on the text re-writing task (e.g., @xmath1, subjective @xmath2. however, to our knowledge, no effort has focused on providing textual explanations for the style transfer task. for example, when transforming an informal sentence "i would throw them out asap !" into a formal paraphrase "i would dispose of them promptly" it would be useful to provide an explanation of the informal attributes in the input sentence (e.g., textese ("asap"), colloquialism ("throw out")), and formal attributes for the paraphrase (e.g., lexical sophistication ("promptly" and "dispose"); lack of abbreviations ("i would")). similarly, for neutralizing subjective bias in "orbis latinus, integral site of romance language" → "orbis latinus, comprehensive site of romance language", it would be useful to have an explanation about which word/phrase in the input is biased and why as well as the type of bias (e.g., framing ("integral" implies a subjective evaluation on the site's importance)). the model's explanations could help the user better assess the correctness of the style transfer system, could be used as features in downstream tasks such as authorship attribution (section 5), or could act as a defense against spurious correlations @xcite and annotation artifacts @xcite . to enable explainability in style transfer models, we provide the following contributions: gyafc informal (original s i ): i would throw them out asap ! e-gyafc llm paraphrase (synthetic s f ): i would dispose of them promptly. formal attributes (synthetic e f ): lexical sophistication ("promptly", "dispose") absence of contractions ("i would") fixed informal attributes (iclef-e i ): textese ("asap"), colloquialism ("throw out"), exclamation mark informal attributes (synthetic e i ): textese ("asap"), colloquialism ("throw out"), exclamation mark, abbreviated language ("i would") iclef llm critic expert feedback formal (original s f ): i would kick them out as soon as possible figure 1 : generating e-gyafc: formality style transfer dataset gyafc @xcite is augmented with semi-structured natural language explanations. the llm generates the informal attributes of the input sentence, a formal paraphrase, and the formal attributes of the resulting sentence. expert feedback is incorporated via in-context learning and self-critique to refine the initial generations. in context-learning with expert feedback (iclef) (see figure 1 , figure 2 , and §3.2). the approach combines model distillation for explanation generation @xcite with self-critique ability of llms @xcite @xcite , where the critic, unlike in prior work, is instantiated with expert demonstrations. we release the data, models, and code to encourage further research on explainability, learning from scarce human feedback, and style transfer. foot_0 textual explanations natural language explanations have been utilized for a variety of tasks @xcite , such as natural language inference @xcite , commonsense @xcite , social norm entailment @xcite . we focus on creating natural language explanations for the style transfer task, which has not been addressed before. style transfer style transfer approaches range from instruction-based methods @xcite and paraphrasing @xcite , to approaches focused on learning in low-resource settings @xcite . much of style transfer work focuses on style representations that decouple style and content @xcite , however most of these methods are not designed to be interpretable. interpretable approaches rely on constructing interpretable embeddings, such as liwc (tausczik and pennebaker, 2010) or lisa @xcite . @xcite proposed identifying biased segments in conjunction with neutralizing biased text. unlike these approaches, we propose to use natural language explanations to further enhance model interpretability. ## building datasets for explainable style transfer we build two explainable style transfer datasets by first augmenting existing datasets with synthetic textual explanations generated by a teacher model ( §3.1), and then improving the generated data using our in-context learning with expert feedback (iclef) framework ( §3.2). ## extrinsic evaluation of formality style transfer explanations we use authorship verification @xcite @xcite as an extrinsic task utilizing pan 2022 @xcite data. this is a binary classification task of deciding if two texts belong to the same author or not. the two input texts are two raw documents (e.g., paragraphs from blog posts), one of which is written by author a, and another one which is either written by the same author a biased (input) some cacti produce beautiful flowers, which like spines and branches arise from areoles. ## conclusion we propose a framework to augment two style transfer datasets with semi-structured textual explanations. to improve quality of model distillation and incorporate expert feedback, we propose iclef (in-context learning from expert feedback), a novel human-ai collaboration framework leveraging both in-context learning and self-critique abilities of llms. we evaluate smaller student models fine-tuned on the resulting datasets compared to large teacher models and conduct expert human evaluation. we also extrinsically evaluate the explanations for the formality style transfer on the downstream task of authorship attribution.