Continuous Language Model Interpolation yields Dynamic and Controllable Text Generation
Authors: Sara Kangaslahti
David Alvarez-Melis
Venue: Accepted by TMLR ()

Abstract: As large language models (LLMs) have gained popularity for a variety of use cases, making them adaptable and controllable has become increasingly important, especially for user-facing applications. In particular, linear interpolation between model parameters forms the backbone for many recent approaches to adapting models to user preferences. While the existing literature on LLM adaptation primarily focuses on finding methods that optimize for some set of performance criteria or user preferences, here we instead seek to better understand and characterize the behavior of dense, continuous interpolation between models. Specifically, we use low-rank updates to fine-tune a base model to various different domains, yielding a set of anchor models with distinct generation profiles. Then, we use the weight updates of these anchor models to parametrize the entire (infinite) class of models contained within their convex hull. We empirically show that varying the interpolation weights yields predictable and consistent change in the model outputs with respect to all of the controlled attributes simultaneously. We find that there is little entanglement between most attributes and identify and discuss the pairs of attributes for which this is not the case. Our results suggest that parameter merging facilitates flexible model adaptation due to its predictable behavior within the full interpolation region.

Published in Transactions on Machine Learning Research (08/2025)
Continuous Language Model Interpolation yields Dynamic and Controllable Text Generation
Sara Kangaslahti
aardhangaslattiig.hvorrr
School of Engineering and Applied Sciences
Horvard University
David Alviisz-Melis
dam&isens.. www.:lssnns.aaraarddd edu
Kempner Institude
Horvard University
Reviewed on OpenReview: https: // openreview. net/ formm? id= rd=aD9W22Whh
Abstract
As large language models (LLMs) have gained popularity for variety of use cases, making them adaptable and controllable has become increasingly important, especially for user- facing applications. In particular, linear interpolation between model parameters forms the backbone for many recent approaches to adapting models to user preferences. While the existing literature on LLM adaptation primarily focuses on finding methods that optimize for some set of performance criteria or user preferences, here we instead seek to better understand and characterize the behavior of dense, continuous interpolation between models. low-rank updates to fine-tune base model to various different domains, yielding set of anchor models with distinct generation profiles. Then, we use the weight updates of these anchor models to parametrize the entire (infinite) class of models contained within their convex hull. We empirically show that varying the interpolation weights yields predictable and consistent change in the model outputs with respect to all of the controlled attributes simultaneously. We find that there little entanglement between most attributes and identify and discuss the pairs of attributes for which this is not the case. Our results suggest that parameter merging facilitates ficxible model adaptation due to its predictable behavior within the full interpolation region.
Specifically,
1 Introduction
Large language models (LLMs) are used for a diverse set of applications due to their high performance across wide spectrum of (asks (Bubeck al., 2023). In many common LLM cases (such chatbots), different users often have distinct and continuously evolving preferences for the type of output they want. For example, user might want creative and verbose response for certain queries, but a concise and precise response for others. In practice, user may try different variations of the same query successively until they elicit a desired generation. This trial-and-error process can be time-consuming and lacks guaranteed results, especially since minor word changes in a prompt can have disproportionate impact on the output. Additionally, expressing fine- gra-ned continuous preferences (c.g., simplifying response by 25%) often difficult in inherenntly discrete- natural language. These challenges are exacerbated when the user has complex, multi-faceted preferences (e.g., specific combination of simplicity, formality, and verbosity) that they expect the generation to satisfy all at once. As result, it is critical to understand how to adapt LLMs to user preferences and constraints in fine-grained and predictable way.
Prior work in controllable text generation (CTG) has largely focused on optimizing for one set of control criteria through techniques such as instruction tuning (Zhou et al., 2023), modifying the output probability
Code: http:://ggthhbb.cm/stagggg//t/tt hnw.mooooo-iittttpllalaaiii interpolation

---

Published in Transactions on Machine Learning Research (08/2025)
User-specified Interpolation parameters
Continuous Language Model Interpolation
Attribute-level a weight Interpolation
A-weighted average of Interpolated models
Input prompt
6.
0:
xN
0g
Output text with N controls parametrized by and
ARE
6/y
xN
XN
Figure 1: Overview our continuous model interpolation framework. Given a collection of 'anchor" models fino-tuned on datasets at opposite ends of an attribute spectrum (c.g., 64st positive and 6-;: negative sentiment) for N different attributes, the user selects interpolation parameters (per-attribute spectrum modulation) and À (attribute mixture weights), which are used to generate model with weights oa, tailored to that specific paramcter choice. We use this framework to analyze the behavior of interpolated models within the entire continuous region between the anchor models.
distributions (Pascual et al., 2021; Yang & Klein, 2021; Dekoninck et al., 2024), changing model activations at inference time (Li et al., 2023), learning modifications to the embeddings (Li & Liang, 2021; Han et al., 2023), or training (Keskar et al., 2019; Krause et al., 2021). These methods, however, are not parametrized continuously and instcad roquire a fixed set of controls criteria. Thus, to achieve fine-grained control in the range between different altribute classes, they would have to be individually of intermediate attribute values, which is prohibitively expensive over continuous range. Similarly, fine- tuning models with data that contains a proportionate amount of documents from each desired objective (ie 0.5 positive and 0.5 negative sentiment documents for a neutral model) would allow for the most precise optimization. However, this is computationally infeasible to do for each combination of control variables and strengths of control in the entire (infinite) set possible combinations.
for each specific set
With these challenges in mind, linear weight interpolation has been proposed to adapt LLMs in a manner that takes ad advantage of the strengths of fine-tuning while making it computationally feasible to dynamically adapt the model for specific tasks user preferences. Recent work has demonstrated that multiple pre-trained fine-tuned models can be effectively composed through linear weight interpolation (Wortsman et al., 2022; Ilharco et al., 2023). This has also been shown to extend to models trained with earameter-efficient fine-tuning (PEFT) methods (Zhang et al., 2023; Huang et al., 2024) such as low-rank adaptation (Hu et al., 2021). As a result, linear interpolation has been used to adapt models through improving multitask performance (Matena Raffel, 2021; Yaday et al., 2023; Ortiz-Jimenez al., 2023) or aligning models to user preferences in the reinforcement learning setting (Ramé al., 2023; Jang et al., 2023; Wang et al., 2024). However, these approaches largely center on optimizing the interpolation for task performance or user reward, the behavior of fine-grained interpolation in the full continuous region between models remains poorly understood.
In this work, we show that linear weight interpolation effectively provides continuous parametrization of the (infinite) 'convex hull' of a set of fine-tuned models. To do so, we fine-tune two endpoint anchor models for each control attribute, one at each extreme of attribute strength. We then interpolate along the vector between the weights of these two models for cach attribute before computing weighted average across all of the single-attribute interpolated models (Figure 1). Thus, varying the interpolation and averaging weights gives dense coverage of the parameter space between endpoint models. We evaluate linear weight interpolation for multiple style attributes and demonstrate empirically that changes in the interpolation and averaging weights yield predictable and consistent responses in each control attribute in the generations.
A potential pitfall of linear interpolation is that, as seen in prior work in the vision domain (Ortiz-Jimenee et al., 2023), the weights for different single-attribute interpolated models may be entangled. This could lead to unexpected correlations between attributes in the averaged models. These correlations are detrimental controllability, changing the interpolation weights for attribute could have unexpected effect
2

---

Published in Transactions on Machine Learning Research (08/2025)
the correlated attributes in the output text. However, we find that there is surprisingly little entanglement between the vast majority of control attributes and analyze the pairs of controls where this is not the case.
In summary, our key contributions are: (1) we provide a framework for analyzing parameter-effcient adaptation in the continuous interpolation region between models fine-tuned with various distinct generation objectives; and (2) we demonstrate that changes in the interpolation yield smooth and predictable changes the properties of the generated text across multiple sets of controls with limited entanglement.
2 Fine-tuning and Weight Interpolation
We evaluate the ability of weight interpolation to control the outputs of LLMs on five commonly used style attributes defined in prior style transfer literature (Jin et al., 2022): simplicity, formality, politeness, sentiment, and humor. For every style characteristic, we first fine-tune two endpoint 'anchor' models, each of which optimizes for one extreme of the style attribute. We then use these models as the basis of the interpolation scheme.
2.1 Datasets
For each style attribute, we fine-tune a separale anchor Llama2-7b model (Touvron et al., 2023) on two English datasets representing the extremes of the attribute level. For simplicity, use the TinyStories dataset (Eldan & Li, 2023) to fine-tune a simple model and novel chapters from the BookSum dataset (Kryscinski et al., 2021) to fine-tune a complex model. We use the documents classified as formal and informal in Grammarly's Yahoo Answers Formality Corpus (GYAFC) dataset (Rao & Tetreault, 2018) to fine-tune formal and informal models, For the politeness attribute, we use the documents in the highest and lowest politeness class in the work by Madaan et al. (2020) for fine-tuning polite and impolite models, respectively. We fine-tune positive and negative sentiment models using the Stanford Sentiment Treebank (SST-2) dataset (Socher et al., 2013). For humor, we use the FlickrStyle dataset (Gan et al., 2017) to fine-tune humorous and non-humorous models.
2.2
tuning
We fine-tune our models in parameter-efficient manner using Low-Rank Adaptation (LoRA, Hu et al., 2021), which keeps pretrained model weights frozen but learns an additive low-rank matrix update for each layer during fine-tuning. Denoting the pretrained language model weights as ORRE Rdixds LoRA computes the updated weights as:
0= OPRE + BA
(1)
Here, A exds and B E Rdixk (with & & di, d2) are trainable parameters learned during fine-tuning. We use LoRA as an adaptation method because it requires significantly fewer parameters than traditional fine-tuning while maintaining similar performance, so LoRA weights can be quickly modified and applied to large pretrained language models. We use the parameters in Appendix A.1 for fine-tuning the models and finc-tune two LoRA models per style charactoristic, one on each of the extreme classes outlined in 2.1. We denote the two LoRA fine-tuned endpoint anchor models for attribute i by 0+ = OPRE + B4iA45 and 0_, 0rne + B_
2.3 Linear weight interpolation
Given collection of fine-tuned model weights obtained by LoRA described above, generate interpolated models by linearly interpolating between their weights. We formulate linear weight interpolation between the LoRA fine-tuned models in terms of interpolation weights a; and attribute mixing weights A as shown in Figure 1. For a single attribute, we interpolate along the vector between the two fine-tuned endpoint models by computing
0a. aa,4 - a=)=
(2)
= Arne + a, arBtiAti + (1 B_;
3

---

Published in Transactions on Machine Learning Research (08/2025)
We call a, the interpolation weight for the ith attribute dimension. We note that a; 0 and a, correspond to letting the interpolated model equal the fine-tuned models Oa, = 0- and 0z; 0+i, respectively. Using Equation 2, we then combine multiple interpolated models 9s, by taking their weighted sum:
0 Ex.0..
We denote X to be the mixing weight for the ith attribute and constrain =1 We note that the case with one attribute dimension corresponds to the sum having a single term with 2 =. With this formulation, we can construct any model in the convex hull of the fine-tuned models by choosing appropriate interpolation weights and mixing weights A. While the raw interpolation parameters do not have clear meaning, we seek to show that a user can controllably increase or decrease the level of each attribute by changing and X.
2.4 Evaluation
To evaluate the interpolated models, we subset of 1K randomly sampled prompts from the WritingPromptss dataset (Fan et al., 2018) and generate 3 continuations for each prompt. We compute scores for each attribute to evaluate the level of the control criterion. Similarly to prior work on text style transfer (Xu et al., 2018), we fine-tune a RoBERTa (Liu et al., 2019) classification head on each attribute using a held out split of the datasets in 2.1 and compute a sigmoid over the output logits to obtain the probability of class 1, which we report as the attribute score. We label the documents such that an attribute score closer to 1 corresponds to a document that is more simple, formal, polite, positive in sentiment, or humorous. We also compute perplexity the test split of the WikiText dataset (Merity et al., 2016) and n-gram diversity of the WritingPromphe generations to evaluate text quality.
We provide a comparison of weight interpolation to two baselines: DExperts (Liu et al., 2021) and model arithmetic (Dekoninck et al., 2024), as these are the main prior approaches that do not use lincar weight interpolation but allow for continuous values of altribute strength parameter. DExperts is formulated using a base model and two fine-tuned endpoint models. Then, given a prompt z<t, if we denote the output logits of the base model at time t as 2 and the logits of the two endpoint models ,, and 2, respectively, then the DExperts oulput probability distribution is defined by P(X,|=<))) softmax(z + a(=; - =)) The model arithmetic baseline uses the same formula for computing the output probability distribution, but instead of using two fine-tuned endpoint models, it uses two prompt-conditioned endpoint models to produce a, and 2 As a result, both DExperts and weight interpolation require 2 * mumber dimensions fine-tuned endpoint models, while model arithmetic does not require any. However, weight. interpolation uses a single inference pass, while both of these approaches require 2 number dimensions + 1 inference passes (Table 4).
Baselines:
We use the fine-tuned anchor models for DExperts and prompt-condition Llama-2-71 (see A.3 for details) for model arithmetic. For both comparisons, we compute the attribute scores for a E [-2, 2). To provide a direct comparison to weight interpolation, we scale the a parameter such that the scaled a = 0 and a 1 correspond to the models with altribute score equal to that of the fine-tuned endpoint models. We evaluate these interpolation methods by their proximity to the ground truth interpolated models, which are Llama-2-7b models fine-tuned with a fraction data from class and a fraction data from class 0.
3 Continuous Language Model Interpolation
We begin by investigating the linear interpolations between each pair of fine-tuned anchor models (3.1). We then extend this analysis to the convex hull of anchor models for multiple attributes (3.2).
3.1 Linear interpolation for a single attribute dimension
We first explore the effect of moving along the vector between a single pair of fine-tuned anchor models. We note that a = 0 and a = 1 correspond to the two fine-tuned anchor models, while a E (0.0, 1.0) is an interpolation and a E (-o0, 0.0) U (1.0, oo) is an extrapolation along the vector between the models.
Linear interpolation: Figure 2 shows the effect of a on attribute score when interpolating between each pair of fine-tuned anchor models. As a increases, there is smooth and predictable increase in the attribute
4

---

Published in Transactions on Machine Learning Research (08/2025)
Attribute Score WritingPrempts
Sentiment
Politeness
Formallty
Simplicity
Humor
Attribute Score
Fine-tuned (ground truth)
Weight interpolstion
DExperts
Arithmetic
0.0
0.
0.0 0.5 1.0
0.0
1.0 0.0
1.0 0.0 0.5 1.0
Scaled Attribute Parameter
Figure 2: Interpolated models recover custom fine-tuned models across the interpolation range. We show the attribute scores for our interpolation framework with weight a compared to DExperts (Liu et al., 2021) and model arithmetic ( (Dekoninck et al., 2024) with scaled such that the scaled a = 0 and a = models have the same score as the fine-tuned endpoint models. Weight interpolation most closely follows the trend the ground truth fine-tuned models. We find similar results for other language models in Appendix A.7.
Table 1: Weight interpolation best approximates custom fine-tuned models while producing high-quality text. For every attribute, we report the mean absolute error (MAE) between the attribute scores of the custom fine-tuned models and the models for each approach with corresponding a attribute parameters. We evaluate the text quality using WikiText perplexity and n-gram diversity scores. Weight interpolation produces text significantly closer in altribute score to the fine-tuned models with similar perplexity and better diversity than previous approaches. Appendix A.7 shows results for other language models and and we report additional diversity metrics Table 6.
Attribute score mean absolute error (MAE)
WikITest Perplexity
Diversity
Sentiment
Politeness
Formality
Simplicity
Humor
Averago
Dist-1
Dist-2
Dist-3
Fiue-tumed (ground truth)
5,040
0.930
0.911
0.896
DEsperte (Läu sl., 2021)
0.066
0.132
0.105
0.080
0.109
5.031
0.892
0.835
Model arithmetic (Dekoninek et al., 2024)
0.075
0.177
0.088
0.143
0.276
0.152
4.900
0.923
0.986
0.889
Weight interpolation
0.036
0.041
0.006
0.066
0.105
0.051
4.947
0.934
0.939
0.891
score for all of the control dimensions. Furthermore, linear interpolation follows the trend of the ground truth fine-tuned models more closely than either of the basclines while achieving similar perplexity and better diversity (Table 1). This demonstrates that interpolation can be used to approximate this continuous class of intermediate fine-tuned models by fine-tuning only 2 endpoint models. In contrast, for the model arithmetic baseline, the unprodictability of the attribute score the politencss and humor dimensions suggests that prompl-conditioned models provide somewhat inconsistent control in the continuous interpolation setting in comparison to fine-tuned models. We find similar results for other language models (Appendix A.7) and diversity metrics (Table 6).
These results indicate that for control attribute, weight interpolation between two endpoint models yields fine-grained control over the model outputs. Furthermore, the trend of increase with appears linear in some cases. For the majority of the attribute dimensions (politeness, formality, and simplicity) we observe a linear increase in the score as a increases the interpolation region. On the other hand, the other control dimensions (sentiment and humor) have a nonlinear increase in attribute score with a due to plateaus at the extremes.
Linear extrapolation: Figure 3 shows the attribute scores when extrapolating linearly beyond the two fine-tuned models along the vector between them. We find that even beyond the region of interpolation between the two finc-tuned models, there is small stable extrapolation regime to a values of around -1 and 2 (Figure 3). In this region, for many of the attribules, the altribute score continues to behave predictably as a is increased. However, beyond the stable extrapolation values, there is an unstable extrapolation regime where the attribute score changes unpredictably as a is variod. This is likely due to the model output quality degrading, since as shown in Figure 4 and Figure 11, the model perplexity increases sharply and the diversity decreases starting near the edges of the stable extrapolation regime. While prior work has shown that linear
5

---

Published in Transactions on Machine Learning Research (08/2025)
Attribute Score WittingPrompss
WikiText Perplexity
7.0
Sentiment
1.0
Formality
Simplictt
Humor
0.0
Attribke Score
Perplestt
5.5
Sentiment
0.2
Politeness Formality
5.0
0.0
Unstable estrap.
Stable extrap
Interp
Stable oxtrap.
Unstable extrap.
Simplicity
Humor
-2
Figure 3: Effect of linear weight extrapolation for single attribute dimension. For each style attribute, report the attribute score when linearly extrapolating beyond the fine-tuned models (a < O and a > 1). There is stable region where the score changes smoothly until a certain point (around a equal to 1 and 2), where performance degrades and the extrapolation is unstable.
Figure 4: Wikitext perplexity of linearly inter- polated and extrapolated models. We report the average perplexity (lower is better) of each model from Figure 3 on the Wikitext test set. For the extrapolated models (a < 0.0 and a > 1.0), the per- plexity increases rapidly beyond a values of around -1 and 2. We clip the y-axis at 7.0 for readability (full plot in Figure 10).
Effect of Interpolation Parameters on WritingPrompts Attribute Score Politeness
Sentiment
Formality
Simplicity
Humor
Attribute Scote
0.6
0.6
0.5
0.2
0.4
0.4
0.6
0.6
0.8
0.3
0.0
0.5
0.0
0.5
1.0
0.5
1.0
0.0
0.5
1.0 0.0
0.5
1.0
Figure 5: Effect a; and X on 5-dimensional interpolation. For each attribute, we show the attribute scores for models with the given a and A parameters, with all four other aj and X, = (1 1.)44//4. We find that increasing a; consistently increases the attribute score and increasing A consistently increases the effect of ag
weight extrapolation can be used for tasks such as model unlearning (Ilharco et al., 2023; Zhang et al., 2023), these results provide a cautionary tale against extrapolating too far, as they suggest that this ability only extends to a certain threshold before the attribute score and model outputs become unpredictable due to poor quality outputs. For the remainder of our experiments, we thus focus on the interpolation regime.
3.2 Multi-dimensional interpolation
In real-world LLM applications, users often have diverse output preferences across multiple control dimensions at once, and these preferences may change dynamically for different inputs to the LLM. In this section, we show that linear interpolation between fine-tuned parameter-efficient adapters can be used to parametrize a whole convex hull of models, which can be used dynamically generate text with attribute levels specified on-the-fly.
3.2.1 Parametrization of the convex hull
Analysis of interpolation parameter a and A: We find that when interpolating across up to five attribute dimensions, modifying the weight parameters X; and as results in predictable, fine-grained control over

---

Published in Transactions on Machine Learning Research (08/2025)
Politeness Score
Sentiment Score
Simplicity Score
Sentiment À
Sentiment X
Sentiment
0.7
0.65
0.8
0.2
0.8
0.2
0.75
0.4
0.4
0.7
0.6
0.4
0.6
0.55
0.4-
0.6
0.5
0.5
0.6
0.2
0.8
0.2
.0.8
0.55
0.2-
0.4
0.4
0.5
0.8
0.6
0.4
0.2
0.8
0.6
0.4
0.8
0.4
0.2
Politeness
X
Simplicity À Politeness
Simplicity À Politeness
Simplicity X
Figure 6: Effect. of A on interpolation between the sentiment, politeness, and simplicity dimen- sions for a, 1. The vertices of the triangle represent the models with a, for each of the three attribute dimensions. The scores in the simplex of weights between the three control dimensions smoothly interpolate between the extreme models.
Humor Score
Formality Score
Simplicity Score
Humor À
Humor À
Humor À
0.92
0.62
0.7
0.8
0.2
0.8
0.2
0.6
0.8
0.2
0.65
0.9
0.58
0.4
.0.4
0.6
0.4
0.6
0.56
0.4
0.54
0.5
0.2
0.8
0.86
0.2
0.52
0.2
0.5
0.4
Formality A
0.6 0.4
0.2
Simplicity
Furmality A
0.8
0.6
0.4
0.2
Simplicity À Formality
0.8
0.6
0.4
0.2
Simplicity À
Figure 7: Effect of A on interpolation between the humor, formality, and simplicity dimensions for a; 1. The vertices of the triangle represent the models with a; for each of the three attribute dimensions. The scores in the simplex of À weights smoothly interpolate between the three endpoints in the simplicity case, but the averaged models are the most neutral in the other cases due to correlations between the control dimensions.
the attribute scores for the desired attributes while having a comparatively small effect on the remaining altributes. Each attribute plot in Figure shows that increasing the as parameter for interpolating between the fine-tuned models increases the attribute score for the ith attribute predictable manner. Similarly, as the model mixture parameter X increases, the effect on the attribute score of changing a increases.
We also analyze the relationship throughout the whole simplex of X weights for sets of three control dimensions in Figures 6 and 7 (as well as Appendix Figures 29-42). For each set of three attributes listed, these plots show the scores in the three dimensional simplex of mixing weights À for which Σ, X = 1. The value of the interpolation weight a for each of the attributes is equal to in Figures and 7, so increasing the À weight of each attribute should increase the attribute score. We find that surprisingly, there very limited entanglement between the majority of the combinations of attributes (such as in Figure 6), likely because their LoRA weights are relatively orthogonal to each other (Appendix Figure 44). In these cases, we observe approximately even increase in score as X; for given attribute dimension increases, regardless of the other X parameters.
Changing mixing parameters À for multiple attributes at once:
However, in some cases, such as humor in the humoo-formalty--iinpiiiit simplex and formality in the formality
simplex with a, = (Figure 7), we observe regions at the corners of the simplex that are close the other finc-tuned models and have high attribute score. This is because these other models are correlated with a positive attribute score, so the mixture of models is the most neutral model. Neverth this still has limited effect on the attribute score, since even in these cases with correlations,
7

---

Published in Transactions on Machine Learning Research (08/2025)
DExperts
Model Arithmetic
Weight Interpolation
0.06
Sentiment
0. 0.055
0.034
0.032
0.007
0.032
0.006
0.011
0.006
0.034
0.004
0.002
0.038
Politeness
0.033
0.031
0.058
0.036
0.003
0.004
0.009
0.002
0.011
0
0.002
0.015
0.006
0.04
Formality
0.023
0.036
0.006
0.004
0.012
0.006
Simplicity
0.008
0.033
0.019
0.038
0.009
0.009
0.005
0.006
0.006
0.006
0.002
0.004
0.02
Humor
0.027 Politeness
0.031
0.063
0.04
0.007
0.094
0.026
0.003 Pollteness
0.015
0.017
0.011
0.00
Sentiment
Formality
Simplicity
Humor
Sentiment
Pollteness
Formality
Simplicity
Humor
Sentiment
Formality
Simplicity
Humor
Figure 8: Weight interpolation has entanglement lower than comparable to prior approaches. For each pair of dimensions, we fix (scaled) a; = 1 for the dimension in each row and vary (scaled) ag between O and for the dimension in each column. We set X; = >3 0.5. Then, report entanglement as the area under the curve of absolute value of change in attribute score as a; increases. Weight interpolation is less entangled than DExperts (Liu et al., 2021) and has similar entanglement to model arithmetic (Dekoninck et al., 2024). These results hold across different language model sizes and families (Appendix A.9).
the score still has the expected behavior unless the mixing weight ry is greater than around 0.4 to 0.6 for the correlated control dimensions. This indicates that in practice, the model has smoothly increasing attribute scores with A, for all pairs of attributes when > for the other attribute dimensions remains sufficiently low.
These results demonstrate that as the parameters X and a, increased for the ith attribute, there is significant effect on the attribute score for the ith control dimension and a limited effect on the scores for the remaining attributes. Therefore, A and a; parametrize the convex hull of models between all of the attribute dimensions and yield fine-grained control over the model outputs with respect to all of the altribules being considered.
3.2.2 Entanglement analysis
Given the results from the simplex plots, we analyze the entanglement between each pair of dimensions better understand the attribute score correlations. For every pair of dimensions (i,j), the two attributes are not entangled, it should be the case that if a; is held constant, then changing aj does not affect the altribule score. We can thus evaluate the amount of entanglement for given ag value by computing the absolute value of the difference between the attribute score for that aj value and the attribute score when = 0. We do so for scaled a; = (for a; 0 see Appendix Figure 20 and for additional language models see Appendix A.9) and aj E {0, 0.25, 0.5, 0.75, 1.0} with X; = Xy = 0.5. We then compute the area under the curve (AUC) of the entanglements for these aj values in Figure 8. We find that the entanglement for weight interpolation comparable to that of model arithmetic (Dekoninck al., 2024), but less than that of DExperts (Liu al., 2021).
4 Related Work
4.1 Controllable text generation (CTG)
As it is crucial to constrain generated text in many downstream applications, CTG has been recent focus of NLP research. Methods such as CTRL (Keskar et al., 2019) and GeDI (Krause et. al., 2021) pretrain language models on text prepended with control codes and generate text conditioned on the desired control. However, these mothods require pretraining a new model if new controls are addod, which computationally expensive. To mitigate these issues, variety of methods have been proposed to perform CTG without additional language model training. For example, Liu et al. (2021); Khalifa et al. (2021): Pascual et al.
8

---

Published in Transactions on Machine Learning Research (08/2025)
(2021); Yang & Klein (2021); Dekoninck et al. (2024) constrain language model outputs by modifying their output probability distributions. L, & Tiang (2021); Qian et al. (2022) learn prefixes and Dathathri et al. (2019); Han et al. (2023) train additional classifiers to guide generation. Subramani et al. (2022); Hernandez et al. (2023); Li et al. (2023); Turner et al. (2023) control model outputs by changing activations at inference time. Kumar et al. (2021) optimize the inference decoding. Mireshghallali el al. (2022); Qin et al. (2022) use energy-based constrained generation and Zhou et al. (2023) use instruction tuning for CTG.
However, we emphasize that the goal of our work is not to output text with the greatest least possible amount of given style attribute, but instead to show that interpolation can be used to dynamically and controllably increase or decrease the amount of a given style attribute. While most of these existing methods can be used to control text for multiple style attributes, the majority of them focus on binary or multi-class control (for example controlling text so that it has positive or negative sentiment) and would need to be re-run for each combination of attribute levels, which is not feasible for 'continuous' adaptive control.
As result, among existing methods only DExperts (Liu et al., 2021) and model arithmetic (Dekoninck et al., 2024) are composable and achieve fine-grained control over multiple attributes at once. Both weight interpolation and DExperts require fine-tuning two endpoint models for each attribute, while model arithmetic uses crompt--onditioned models. However, both DExperts and model arithmetic compose multiple models at inference time, so the inference cost. is significantly higher than weight interpolation, especially as the model size and number of controlled attributes increases, In addition, our experiments show that weight interpolation provides more precise fine-grained control and less entanglement than these approaches over the range of intermediate models.
4.2 Weight interpolation
Our work builds on prior work on linear weight interpolation, such as task vectors (Ilharco et al., 2023), parameter-offiient task vectors (Zhang et al., 2023), and model souping (Wortsman et al., 2022), as we use linear interpolation and weighted model averaging as the basis for our analysis. Prior work in this domain has focused mainly improving multitask performance when composing fully fine-tuned models (Matena & Raffel, 2021; Yadav et al., 2023; Ortiz-Jimenez et al., 2023) or parameter-efficien models (Huang et al., 2024; Jiang et al., 2024). However, these methods all differ from our work, since they focus on combining model weights lo improve a single multitask objective rather than analyzing performance across a wide range of flexible, diverse objectives. These approaches are orthogonal to our work and could be used in conjunction with it to better combine the a-intcrpolated models.
fine-tuned
Beyond multitask performance, a variely of reinforcement. learning approaches have been proposed for language model alignment to user preferences. Specifically, rewarded soups first fine-tunes model for each preference and computes the combination of interpolation parameters that maximizes the user's reward on a validation set (Ramé et al., 2023), personalized soups uses a multi oulti-objective reinforcement learning objective combined with parameter merging (Jang et al., 2023), and conditional language policy performs multi-objective fine-tuning to learn a set of parameters that is steerable by conditioning on the user reward at inference time (Wang et al., 2024). These methods show that reinforcement learning objectives can be combined with weight interpolation in order to optimize over a wide range of user preferences. However, they do not analyze the behavior across the full interpolation region and instead center around developing methods that obtain close to parelo-optimal reward or are more preferential to users with specified reward functions.
Perhaps most similar to our work are methods that analyze the interpolation regime between the weights of fine-tuned models over a range of outputs (Gandikota et al., 2023; Nylund ct al., 2023). However, Gandikota et al. (2023) focus on the vision domain and use a fine-tuning objective specific to diffusion models, and Nylund al. (2023) only analyze control over the time dimension.
5 Conclusion
In this work, we show that continuous linear interpolation between low-rank fine-tuned models can be used to parametrize the models in their convex hull. We achieve fine-grained, predictable control over multiple attributes of style at once by changing the interpolation weights between two anchor fine-tuned models and
9

---

Published in Transactions on Machine Learning Research (08/2025)
the mixing weights between different interpolated attribute models. We find that the interpolation profiles between models are smooth and there is surprisingly little entanglement between the models for different. control dimensions. In other words, changing the weight for one attribute has a very small effect on the scores for other attributes, especially for sufficiently small mixing weights. As a result, linear weight interpolation produces predictable behavior the entire continuous space of models between the fine-tuned anchors.
Future work
The main limitation of our work is that some pairs of attributes are correlated, when a correlated model has large mixing weight, it unpredictably affect other control attributes. While there is lower comparable correlation in weight interpolation compared to other approaches, in the future it would be valuable to investigate whether this correlation is inherent to the pair of tasks or if it can be eliminated. For example, text that is more polite might always be more formal. However, it may be the case that some correlations can reduced by regularizing the LoRA updates to be more orthogonal to each other or by merging the a-interpolated using more sophisticated methods that have recently shown improvement over naive weight, averaging in the multitask setting (Matena & Raffel, 2021; Yadav et al., 2023; Ortiz-limenez et al., 2023). Similarly to prior work task vectors (Ilharco al., 2023), correlations could also potentially be used to combine existing control altributes to make new ones (such as by combining irony and humor to control for sarcasm).
Another limitation is that the average generation attribute scorcs are limited to the range between the altribule scores the fine-tuned anchor models. The single altribute extrapolation results could expanded upon better understand when extrapolation can be used extend the range of the control attribute style. We also only consider controlling for text style in our experiments, but analyzing linear interpolation when controlling for other attributes such topics is possible direction for future work.
Acknowledgments
SK is supported by the National Science Foundation Graduate Research Fellowship under Grant No. DGE 2140743. DAM and SK acknowledge support from the Kempner Institute, the Aramont Fellowship Fund, and the FAS Dean's Competitive Fund for Promising Scholarship.
Broader Impacts
Continuous weight interpolation may output text that contains existing biases from the pre-trained models and fine-tuning datasets. It could also be used to control the level of undesirable attributes such as toxicity. However, we believe that this work still beneficial overall, since we can better understand the behavior of linearly interpolation for adapting LLMs, and these issues are faced by all pre-trained and fine-tuned language models.
References
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Loc, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. Sparks of artificial general intelligence: Early experiments with gpt-4, 2023.
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. Plug and play language models: A simple approach to controlled text generation. CoRR, abs/1912.02166, 2019. URL http://arxiv.. wwtp:///..gra/b///////.aass 1w1.9991111111
Jasper Dekoninck, Mare Fischer, Luca Keurer-Kellner, and Martin Vechev. Controlled text generation via language model arithmetic.. The Twelfth International Conference on Learning Representations, 2024. URL https:://ppnnrevier http://paarrrriiie..eeerrrrrtaa/ het/forumiid-SLw#fffe
Ronen Eldan and Yuanzhi Li. Tinystories: How small can language models be and still speak coherent english?, 2023.
10

---

Published in Transactions on Machine Learning Research (08/2025)
Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In Iryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Annual Meeling of the Association for Compulational Linguistics (Volume 1: Long Papers), pp. 889 898, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10. 111111/////1111/111 1082. URL
org/P18-1022.
Chuang Gan, Zhe Gan, Xiaodong He, Jianfeng Gao, and Li Deng. Stylenet: Generating attractive visual captions with styles. In 2017 IEFF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 955 964, 2017. doi: 11109/CVVR.201..100
Rohit Gandikota, Joanna Materzynska, Tingrui Zhou, Antonio Torralba, and David Bau. Concept sliders: Lora adaptors for precise control in diffusion models, 2023.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Amirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Cauchoteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia- Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Ilian Zarov, Imanol Arricta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadcokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Tyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala- Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landkaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengehuan Zhang, pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennahasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgion, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujiwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do Vish Vogeti, Vitor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasminne Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexci Bacvski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ranchandani, Annie Dong, Annic Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti,
Diego Perino, Dieuwke Hupkes, Egor
11

---

Published in Transactions on Machine Learning Research (08/2025)
Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Due Le, Dustin Holland, Edwaar Dowling, Eissa Jamil, Flaine Montgomery, Elconora Presani, Emily Hahn, Emily Wood Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Takshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Miin-Elenaa Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jcan-Baptisn Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstcin, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Picrre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayosh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterficld, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Kochler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiaojian Wu, Xiaolan Wang, Xilun Wa, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei 21783.
Zhao, and Zhiyu Ma. The Ilaaa herd of models, 2024. URL https: //ta///vvvvv//aaaae
Chi Han, Jialiang Xu, Manling Li, Yi Fung, Chenkai Sun, Nan Jiang, Tarek Abdekaher, and Heng Ji. Tan-switch: wight language model conditioning in word embedding space, 2023.
Evan Hernandez, Belinda Z. Li, and Jacob Andreas. Inspecting and editing knowledge representations in language models, 2023.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. CoRR, abs abs/2106.09685, 2021. URL https://arxiv: wwweriii.rr/bbb/////ee 09685.
Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, and Min Lin. Lorahub: Efficient cross-task generalization via dynamic lora composition, 2024.
12

---

Published in Transactions on Machine Learning Research (08/2025)
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannanch Hajishirzi, and Ali Farhadi. Editing models with task arithmetic, 2023.
Joel Jang, Seungone Kim, Bill Yuchen Lin, Yizhong Wang, Jack Hessel, Luke Zeuueeooyer, Hannaneh Hajishirzi, Yejin Choi, and Prithviraj Ammanabrolu. Personalized soups: Personalized large language model alignment via post-hoc parameter merging, 2023. URL https: //axiiv../xivvaaaaaa www.i..oggaab........oopt....... org/abs/23In 11564.
Weisen Jiang, Baijiong Lin, Han Shi, Yu Zhang, Zhenguo Li, and James T. Kwok. Byom: Building your own multi-task model for free, 2024.
Di Jin, Zhijing Jin, Zhiting Hu, Olga Vechtomova, and Rada Mihalcea. Deep Learning for Text Style Transfer: A Survey. Computational Linguistics, 48(1):155-200 04 2022. ISSN 0891-2017. doi: 10.1162/coli 00426. URL https://doi, org/10. 1ww1/co11lllloo 00426.
Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. CTRL: A 2019. URL
conditional transformer language model for controllable generation. CoRR, http://arxiv.. www. htt///9b/1999......./ 05858..
Muhammad Khalifa, Hady Elsahar, and Mare Dymetman. A distributional approach to controlled text generation. In International Conference on Learning Representations, 2021. URL https: http:://oopnrevierr
Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. GeDi: Generative discriminator guided sequence generation. In Mlarie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 1929-4952, Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 10.18653/v/ /00055//v/// 12021 (0.18653/11/222..... www.iliininngenl....... URL
emmlp..mm.p........ 424.
Wojeiech Kryscinski, Nazneen Fatema Rajani, Divyansh Agarwal, Caiming Xiong, and Dragomir R. Radev. Booksum: A collection of datasets for long-form narrative summarization. CoRR, abs/2105.08209, 2021.
URL http:///aariiv.ooraa/////// 08209.
Sachin Kumar, Fric Malmi, Aliaksei Severyn, and Yulia Tsvetkov. Controlled text generation continuous optimization with multiple constraints. CoRR, abs/2108.01850, 2021. URL 2108.01850.
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from language model, 2023.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. CoRR, abs/2101.00190, 2021. URL https: http://xxiv..../ http::///arxv...oraa//////...
Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. On-the-fly controlled text generation with experts and anti-experts. CoRR, abs/2105.03023, 2021.
URL https://arxiv. wwwpersxvvvvrrrarrreeeeee otggabs//105..oom 03023.
Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692,, 2019. URL http://arxiv wwtp:oo///////////o/ ttabb///997.1100 11692.
Aman Madaan, Amrith Setlur, Tanmay Parekh, Barnabás Póczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov, Alan W. Black, and Shrimai Prabhumoye. Politeness transfer: A tag and generate approach. CoRR, abs/2004..1225 142577, 2020. URL
http///aaiwiarrra////aa///////// 14257.
Michael Matena and Colin Raffel. Merging models with fisher-weighted averaging. CoRR, abs/2111 2111.09832, 2021. URL https://arxiv. hwt.aaawwwwwo.rrb////aaaaaaaae/e http//ab/2211..99...
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. CoRR, /t///1699008433 2016. URL http://arxivg http::////aaii.oorr/a///////////
0 07843.
13

---

Published in Transactions on Machine Learning Research (08/2025)
Fatemehsadat Mireshghallah, Kartik Goyal, and Taylor BrrgKKirppatrick. Mix and match: Learning free controllable text generation using energy language models, 2022.
Kai Nylund, Suchin Gururangan, and Noah A. Smith. Time is encoded in the weights of finetuned language models, 2023.
Guillermo Ortiz-Jimenez, Alessandro Favero, and Pascal Frossard. Task arithmetic in the tangent space: Improved editing of pre-trained models, 2023.
Damian Pascual, Beni Egressy, Clara Meister, Ryan Cotterell, and Roger Wattenhofer. A plug-and-play method for controlled text generation. CoRR, abs/2109.09707,, 2021. URL https://arxiv. http:///axxiv..ogabbbbaaa/ wwt.oorg///////oooo/o 09707.
Jing Qian, Li Dong, Yelong Shen, Furu Wei, and Weizhu Chen. Controllable natural language generation with contrastive prefixes, 2022.
Lianhui Qin, Sean Welleck, Daniel Khashabi, and Yejin Choi. Cold decoding: Energy-based constrained text generation with langevin dynamics, 2022.
Alexandre Ramé, Guillaume Couairon, Mustafa Shukor, Corentin Dancette, Jcan-Baptiste Gaya, Laure Soulier, and Matthieu Cord. Rewarded soups: towards pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards, 2023.
Sudha Rao and Joel Tetreault. Dear sir or madam, may I introduce the GYAFC datasct: Corpus, benchmarks and metrics for formality style transfer. In Marilyn Walker, Heng Ji, and Amanda Stent (eds.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 129 140, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: (0.18653/v1/N18- 1012. URL https: hatps////caaaahooog/o org/N18-1012 10122.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treebank. In David Yarowsky, Timothy Baldwin, Anna Korhonen, Karen Livescu, and Steven Bethard (eds.), Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631 1642, Seattle, Washington, USA, October 2013. Association for Computational Linguistics. URL https: //aclanthology. wwwphhhology.ooo//1-11111 org/D13-1170.
Nishant Subramani, Nivedita Suresh, and Matthew E. Peters. Extracting latent steering vectors from pretrained language models, 2022.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Fric Hambro, Faisal Azhar, Aurelien Rodrignez, Armand Joulin, Edouard Grave, and Guillaune Lample. Llama: Open and efficient foundation language models, 2023.
Alexander Matt Turner, Lisa Thiergart, David Udell, Gavin Leech, Ulisse Mini, and Monte MacDiarmid. Activation addition: Steering language models without optimization, 2023.
Kaiwen Wang, Rahul Kidambi, Ryan Sullivan, Alckh Agarwal, Christoph Dann, Andrea Michi, Marco Gelmi, Yunxuan Li, Raghav Gupla, Avinava Dubey, Alexandre Ramé, Johan Ferret, GeofTrey Cideron, Le Hou, Hongkun Yu, Amr Ahmed, Aranyak Mehta, Léonard Hussenot, Olivier Bachem, and Edouard Leurent. Conditional language policy: A general framework for steerable multi-objective finetuning, 2024. URL 15762. 15762
https://arxiv orw/abs/////ee /200s//447...
Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S. Morcos, Hongsook Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time, 2022.
14

---

Published in Transactions on Machine Learning Research (08/2025)
Jingjing Xu, Xu Sun, Qi Zeng, Xiaodong Zhang, Xuancheng Ren, Houfeng Wang, and Wenjie Li. Unpaired sentiment--ossentmmenn translation: A cycled reinforcement learning approach. In Tryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 979-988, Melbourne, Australia, July 2018. Association for CRL https::/aclanthhllogg ort.:a/tthhooooooo//-------00000 org/P18-1090.
Computational Linguistics. doi: 10. 10. 1t.11855/v1 1
Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. Ties-merging: Resolving interference when merging models, 2023.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yugiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. URL https://arxiy: org/ htt.//5555099990
Kevin Yang and Dan Klein. FUDGE: Controlled text generation with future discriminators. In Kristina Toutanova, Anna Rumshisky, Luke ZZetlemoyer, Dilek Taakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 3511-3535, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/2022 naacl-main.276. URL https: https://allatthooog org/2021 naacl-main.276.
Jinghan Zhang, Shiqi Chen, Junteng Liu, and Junxian He. Composing parameter-efficient modules with arithmetic operations, 2023.
Wangchunshu Zhou, Yuchen Eleanor Jiang, Ethan Wilcox, Ryan Cotterell, and Mrinmaya Sachan. Controlled text generation with natural language instructions, 2023.
15

---

Published in Transactions on Machine Learning Research (08/2025)
A Appendix
A.1 Hyperparameters for fine-tuning
Table 2: Parameters for LoRA fine-tuning. We use 20 epochs for fine-tuning the sentiment attribute
models and epoch for the remaining finc-tuned models. All experiments were run single NVIDIA A100 80GB SXM GPU nodes.
LoRA hyperparameter
Value
Batch size
64
Learning rate
56-5
LoRA
32
LoRA a
16
LoRA dropout
0.1
Max sequence length
128
Quantization
4 bit
Table 3: Fine-tuning splits. We report the number of examples from each attribute dataset used to fine-tune Llama2-7b generation and RoBERTa attribute scoring models. Each split is sampled from the combined train, lest, and validation set.
Llama2 split size
ROBERTS *plit wine
Domsin
Class
Class
Sentimrrr Sochre (2013)
25k
30%
Likk
Politenes Madian al. (2H20)
75k
21kk
Forualitt Rao & Totrosalt (2018)
10tk
104k:
1ilk
Simplictt (Keyscinsli et al,, 2021: Eldon & Lá, 2020)
8k
100k
Lik
Humor Gan al. (2017)
20k
A.2 Complexity comparisons to previous approaches
Table Comparing weight interpolation to previous CTG approaches. N the number of controlled attributes. Similarly to DExperts (Liu al., 2021), weight interpolation requires fine-tuned anchor models. However, in contrast to prior work where the number of inference passes scales with the number of attributes, weight interpolation uses only single inference pass.
Approuch
Anchor models
Infermace passes
DEsperts (Láu et al., 2021)
Fine-tuned
2N
Model arithmetic
(Deloninck et al., 2024)
Prampl-conditioned
++
Weight interpolation
Fine-tuned
A.3 Model Arithmetic Formulation
For the model arithmetic (Dekoninck et al., 2024) comparison, we use the following formula, inspired by DExperts (Liu al., 2021):
M + (AMps Mneg)
Here, M is the base model, Mpus is the model conditioned for class 1, and Maeg is the model conditioncd for class O. The system prompts used for conditioning are listed in Table 5.
16

---

Published in Transactions on Machine Learning Research (08/2025)
Table 5: System prompts used for conditioning model arithmetic.
Conditioned
Llama2-7b System Prompt
model name
sentiment pos
'The following is a positive story, with very positive sentiment and very positive tone.'
sentiment neg
"The following is negative story, with very negative sentiment and very negative tone.'
formality pos formality neg
'The following is a formal story, with very formal language and very formal tone.'
"The following is an informal story, with very informal language and a very informal tone.'
simplicity_ simplicity_nu neg
'The following is a simple story, with very simple language." 'The following is complex story, with very complex language." !
humor pos
"The following is a humorous story, with very humorous language and very humorous tone.'
humoor neg
"The following is nonhumorous story, with factual language and a serious tone.'
politeness
"The following is polite story, with very polite language and very polite tone.'
politeness neg
'The following an impolite story, with very impolite language and a very impolite tone.'
A.4 Attribute comparison to prompting
In this section (Figure 9), provide comparisons for each attribute between prompting Llama2-13b-cha instruction-tunct models and the attribute score of models fine-tuned with fraction of data from class and (1 - a)) fraction of data from class 0 for each altribule. We use the following prompting set-up inspired by Han et al. (2023):
CCmplette this story that it embodies sentiment score of 0.5, where 0 is negative and 1 is positive:
For each style attribute, we replace the words "sentiment", "negative", and "positive" with the corresponding attribute and class names, and 0.5 with the corresponding score.
Attribute on WritingPrempts Formality
Sentiment
Politeness
Simplicity
Humor
Sentiment Prompted
Attribute Score
1.00
Politaness Prompted
Sentiment
0.75
Politeness Fine-tuned
0.50
Formality Prompted
Formality Fine-tuned
Simplicity Prompted
0.00
Simplkity Fine-tuned
Humor Prompted
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0 0.0
0.5 1.0 0.0
0.5
1.0
Humor Fine-tuned
Figure 9: Attribute scores for prompting Llama2-13b-chan versus finc-tuned models. We report the altribute score when prompting Llama2-13b-chat models to produce output with score as compared to the attribute score for models trained with fraction class 1 and 1 fraction class 0 data. The attribute scores of the prompted model only slightly increase with a and do not closely follow the trend of the finc-tuned models.
17

---

Published in Transactions on Machine Learning Research (08/2025)
A.5 Extrapolation text quality analysis
WikiText Perplexity
10
Sentiment
Politnnes
10
Formality
Simpircity
10
Humor
1012
102
10a-
10a
-3
Figure 10: Wikitext perplexity of linearly interpolated and extrapolated models. We report the average perplexity of each model from Figure on the Wikitext test set. For the extrapolated models not shown in Figure 4, the perplexity increases rapidly.
A.6 Diversity analysis
Table 6: Diversity comparison. For each model, report the mean compression ratio, BERT and ROUGE- L homogenization scores, and self-repetition score (lower is better for all scores) for the single-attribute linear interpolation from Table 1. Weight interpolation produces text with similar or better diversity than previous approaches for all of the models tested.
Compression
Homogenization score (BERT)
Homogenization score (ROUGE-L)
Self-repetition
Model
Method
ratio
score
Fine-tuned (ground truth)
2.174
0.692
0.077
0.063
DExperts (Liu et al., 2021)
1.459
0.699
0.086
0.207
Llama 2-7b
Model Arithmetic (Dekouinck et al., 2021)
2.512
0.692
0.100
0.627
Weight Interollation
2.140
Fine-tuned (ground truth)
2.261
0.702
0.080
DExperts (Liu 2021)
2.209
0,070
0.051
Llama-2-13h
Model Arithmetic (Dekoniuck et al., 2024)
2.598
0.694
0.101
0.728
Weight Interplation
2.238
0.698
0.080
0.065
Fine-tuned (ground truth)
2.225
0.697
0.082
0. 00106
DExperts (Liu et al., 2021)
2. 402
0.689
0.097
0.228
Alama-3.1-8B
Model Arithmetic (Dekominkk et al., 2024)
2.170
0.689
0.081
0.048
Weight Interpolation
2.208
0.695
0.080
Fine-tuned (ground truth)
2.269
0.706
0.083
DExperts (Liu 2021)
2.301
Qwen3-81B-Base
Model Arithmetic (Dekoniuck et al., 2024)
2.405
0.687
0.109
0.667
Weight Interpolation
2.2422
0.702
0.083
0.059
Fine-tuned (ground truth)
2.265
0.705
0.084
0.131
DExperts (Liu et al,, 2021)
2.317
0.683
0.089
0.556
Qwen3-14BB 1-B---
Model Arithmetic (Dekominck rt al., 2024)
2.417
0.688
0.103
0.944
Weight Interpolation
0.702
18

---

Published in Transactions on Machine Learning Research (08/2025)
Crentry
Figure 11: N-gram diversity scores for the extrapolated models. We report the 1-, 2-, and 3-gram diversity scores for the singlo-attribute interpolated and extrapolated models. The diversity scores remain similar to or wetneen those of the endpoint fine-tuned models within the interpolation region (a E [0, 1J) and in the stable extrapolation region (a () (1, but become unstable beyond the stable extrapolation region.
Dist1 Diversity
Sentiment Prompted
0.925
0.935
0.92
0.96
Sentiment fine-tunes
Politeness Prompted
dist] Score
0.930
0.91
0.95
Politeness Rinestunad
0.920
0.90
Formality Prompted
0.925
0.94
Formaltty Fina-tuned
0.915
0.89
0.93
Simplicity Prompted
0.88
Simplicity fine-tuned
0.915
Humor Prompted
0.0
0.5
1.0
1.0
1.0
Humor Fine-tuned
Figure 12: 1-gram diversity comparison of prompted versus fine-tuned models. We report the dist diversity scores for the Llama2-13b-chat prompted models with weight as compared to the perplexity for modols trained with a fraction class and a fraction class 0 data. The diversity scores remain similar to the endpoint fine-tuned models within the interpolation region.
Dist2 Diversity
0.950
0.945
Sentiment Prompted
0.945
0.945
Sentiment Fine-tuned
dist Score
0.945
0.94
Politeress Prompted
Politeness Fine-tuned
0.935
Formality Promptad
0.935
0.93
0.935
Formaiity Fine-tuned
Simpliitty Prompted
0.930
0.930
0.92
0.925
Simplicity Rin-ttuned
Prompted Humor iine-tund
0.0
0.5
1.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
Figure 13: 2-gram diversity comparison of prompted versus fine-tuned models. We report the dist.2 diversity scores for the Llama2-13b-chat prompted models with weight as compared to the perplexity for models trained with fraction class and - a fraction class 0 data. The diversity scores remain similar to the endpoint fine-tuned models within the interpolation region.
Dist3 Diversity
Sentimant Prompted
0.900
0.90
0.905
Sentiment Fine-tuned
diat3 Score
0.905
0.900
0.89
0.900
0.895
Politeness Rine-tuned
Politaness Prompted
0.895
0.88
0.895
0.890
Formality Fine-tured
Formality Prompted
0.890
Simplicity Prompted
0.885
0.87
0.890
0.880
Humar Prompted
Simplkily fire-tuned
0.0
0.5
1.0
0.0
0.5
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
Humor Fine-tuned
Figure 14: 3-gram diversity comparison of prompted versus fine-tuned models. We report the dist. diversity scores for the Llama2-11b-chat prompted models with weight compared to the perplexity for models trained with fraction class and - fraction class 0 data. The diversity scores remain similar to the endpoint fine-tuned models within the interpolation region.
19

---

Published in Transactions on Machine Learning Research (08/2025)
A.7 Single attribute dimension interpolation results for additional models
We show that our single attribute dimension results generalize across language model sizes and families by reporting the results from Figure 2 and Table 1 for Llama-2-Iat 2-13b (Touvron et al., 2023), Llama-3.1-80 (Grattafiori et al., 2024), Qwen3-8B-Ban Basee (Yang et al., 2025), and 14B-Base (Yang et al., 2025).
Attribute Score on WritingPrompts Simplicity
Sentiment
Politeness
Formality
Humor
Attribute Score
1.00
fine-tuned (ground truth)
0.50
Wright Interpolation
DExports
0.25
Model Arithmetic
0.00
0.0
0.5
0.5 Scaled Attribute Parameter
0.5
0.5
Figure 15: Llama-2- 13b interpolated models recover custom fine-tuned models across the interpolation range. We show the attribute scores for our interpolation framework with weight a compared to DExperts (Liu et al., 2021) and model arithmetic (Dekoninck et al., 2024) with a scaled such that the scaled 0 and models have the same score as the fine-tuned endpoint models. Weight interpolation most closely follows the trend of the ground truth fine-tuned models.
Table 7: Llama-2-13h weight interpolation best approximates custom fine-tuned models while producing high-quality text. For every attribute, we report the mean absolute error (MAE) between the attribute scores of the custom finc-tuned models and the models for each approach with corresponding a altribute parameters. We evaluate the text quality using WikiTeex, perplexity and n-gram diversily scores. Weight interpolation produces text significantly closer in attribute score to the fine-tuned models with similar perplexity and better diversity than previous approaches. We report additional diversity metrics in Table 6.
Attribute
absolute error (MAE)
WikiText Perplexity
Diversity
Sontiment
Politeness
Formality
Simplicity
Humor
Averago
Dist-l
Dist-2
Dist-3
Fine-tumed (ground truth)
1.195
0.925
0.915
0.901
DExperts (Liu et al., 2021)
0.017
0.040
0.015
4.577
0.916
Model urithmetic (Dekoniuck al., 2024)
0.029
0.141
0.079
0.061
0.325
0.127
4.364
0.931
0.887
Weight interpolation
0.006
0.038
0.012
0.050
0.157
0.053
4.436
0.919
0.944
0.904
Attribute Score WritingPrompts Formality
Sentiment
Paliteness
Simplicity
Humar
Attribute Score
1.00
0.75
-
Ane-tuned (ground truth)
Weight interpolation
0.50
DExperts
0.25
Arithmetic
0.00
0.5
0.5
0.5
Scaled Attribute Parameter
Figure 16: Llama-3.1-8B interpolated models recover custom fine-tuned models across the interpolation range. We show the attribute scores for our interpolation framework with weight a compared to DExperts (Liu et al., 2021) and model arithmotic (Dekoninck et al., 2024) with a scaled such that the scaled a and a models have the same score as the fine-tuned endpoint models. Weight interpolation most closely follows the trend of the ground truth fine-tuned models.
20

---

Published in Transactions on Machine Learning Research (08/2025)
Table 8: Llama-3.1-8B weight interpolation best approximates custom fine-tuned models while producing high-quality text. For every attribute, report the mean absolute error (MAE) between the attribute scores of the custom fine-tuned models and the models for each approach with corresponding a attribute parameters. We evaluate the text quality using Wiki'Text perplexity and n-gram diversity scores. Weight interpolation produces text significantly closer in attribute score to the fine-tuned models with similar perplexity and diversity to previous approaches. We report additional diversity metrics in Table
Attribute score absolute (MAE)
WikfText Perplexity
Diversity
Sentiment
Politeness
Formality
Simplicity
Humor
Average
Dist-1
Dist-2
Dist-3
Fine-tumed (ground truth)
4.495
0.917
0.946
0.917
DEsperte 2021)
0.025
0.031
0.076
1.777
0.923
0.876
Model arithmetic (Dekoninck et al., 2024)
0.037
0.127
0.182
0.058
0.281
0.137
4.364
0.917
0.952
0.917
Weight interpolation
0.018
0.040
0.003
0.078
0.171
0.062
4.436
0,910
0.942
0.910
Attribute on WritingPrompts Formality
Sentiment
Simplicity
Altribute Score
0.75
(ground truth)
0.50
Wright interpolation
DExperts
0.25
Modal Arttmmatic
0.00
0.0
0.5
1.0
0.0
0.5
1.0 0.0
0.5
10
0.0
0.
1.0 0.0
0.5
1.0
Scaled Attribute Parameter
Figure 17: Qwen3-8B-Base interpolated models recover custom fine-tuned models across the interpolation range. We show the attribute scorcs for interpolation framework with weight compared to DExperts (Liu et al., 2021) and model arithmetic (Dekoninck et al., 2024) with scaled such that the scaled a 0 and models have the same score as the fine-tuned endpoint models. Weight interpolation most closely follows the trend of the ground truth fine-tuned models.
Table 9: Qwen3-8B-Base weight: interpolation best approximates custom fine-tuned models while producing high-quality text. For every attribute, report the mean absolute error (MAE) between the attribule scores of the custom fine-tuned models and the models for each approach with corresponding a attribute parameters. We evaluate the text quality using WikiText perplexity and n-gram diversity scores. Weight interpolation produces text significantly closer attribute score the finc-tuned models with similar perplexity and belter diversity than previous approaches. We report additional diversity metries in Table 6.
Attribute score mean absolute error (MAE)
WikfText Perplexity
Diversity
Sentiment
Politeness
Formelity
Simplicity
Humor
Average
Dist-2
Dist-3
Fine-tuned (ground teuth)
6.857
0.916
0.949
0.914
DEsperta (Liu et al., 2021)
0.078
0.078
0.010
0.096
0.155
0.089
6,711
0.856
0.931
0.899
Model arithmetic (Dekoninck al., 2024)
0.072
0.124
0.083
0.194
0.367
0.168
6.340
0.936
0.908
Weight interpolation
0.032
0.035
0.017
0.129
0.078
0.058
6.464
0.911
0.948
0.914
21

---

Published in Transactions on Machine Learning Research (08/2025)
Attribute Score WritingPrompts
Sentiment
Politeness
Formality
Simplicity
Humor
Attribute Score
1.00
0.75
Fine-tuned Iground truth)
Weight interpolstion
DExperts
0.25
Arithmetic
0.00
0.0
0.5
1.0
0.0
0.0
0.5
0.5
0.5
1.0
Scaled Attribute Parameter
Figure 18: Qwen3-14B-Bal interpolated models recover custom fine-tuned models across the interpolation range. We show the attribute scores for our interpolation framework with weight compared to DExperts (Liu et al., 2021) ) and model arithmetic (Dekoninck et al., 2024) with a scaled such that the scaled a and a = models have the same score as the fine-tuned endpoint models. Weight interpolation most closely follows the trend of the ground truth fine-tuned models.
Table 10: 1reo3-144-BBsee weight interpolation best approximates custom fine-tuned models while producing high-quality text. For every altribule, we report the mean absolute error (MAE) between the attribute scores of the custom fine-tuned models and the models for each approach with corresponding a attribute parameters. We evaluate the text quality using Wexi perplexity and n-gram diversity scores. Weight interpolation produces text significantly closer in attribute score to the fine-tuned models with similar perplexity and better diversity than previous approaches. We report additional diversity metries in Table
Attribute some mean absolute error (MAF)
WikiTest Perplexity
Diversty
Sentiment
Politeness
Formality
Simplicity
Humor
Average
Dist-1
Dist-2
Dist-3
Fine-tuned (ground teuth)
6.282
0.916
0.960
0.914
DEspports (Liu et al., 2021)
0.057
0.067
0.021
0.070
0.162
0.075
6.062
0.879
0.931
0.899
Model arithmettc (Dekoniack al., 2024)
0,083
0.106
0.049
0.171
0.363
0.154
5.782
0.871
0.931
0.905
Welght Interpolation
0.041
0.022
5.906
0.949
0.915
A.8 Generation example
We provide an example generation to compare between weight interpolated models for a single attribute and prompting an instruction-tuned model 1ab-caa).b We provide the model generations for the following prompt set-up inspired by Han al. (2023):
"Complete this story so that it embodies a sentiment score of 0.5, where 0 is negative and 1 is positive: You find a rip in time walking through the alleys You enter to find yourself
For each style attribute, we replace the words "scntiment", "negative", and "positive" with the corresponding attribute and class names, and 0.5 with the corresponding a score.
We report the output until the first occurrence of newline character the amount of output that fits in 2-3 lines of the table.
In general, is challenging to achieve fine-grained control over the oulput altributes with prompting as compared interpolation. The prompted model often does not properly account for the a value and produces outputs at attribute extreme the other rogardless of a. Furthermore, for dimensions that are less commonly used in CTG (ie formality), the prompted model often produces very similar outputs for each value of a, as reflected by many of the scores in Figure 9.
22

---

0.3 you slowly move forward looking around your surroundings. Seeing man sitting under a [..] 300 years before your time. Some kids around Published in Transactions on Machine Learning Research (08/2025)
Table 11: Generation comparison: we present a comparison of generations for single attribute interpolation versus prompted Llama2-13b-char for various a values with the prompt 'You find a rip in time walking through the alleys. You enter it to find yourself*
Dimension
Single attribute interpolation
Prompted Llama2-13b-chat
40 minutes later still wondering what the h+1l you did wrong. [..]
months in the future. Everything has changed but... [..]
0.0
Sentiment
months in the future. Everything has
12 hours earlier with your hopes and sanity battered only to discover the time rip still [--]
0.3
changed but it seems the world has gotten better. [..]
10 minutes in the past, before you
40 years earlier, passing through an archway into deeply familiar but different world. [..]]
0.7
were born. You decide to go back in time and give your younger self [..] 20 years earlier, in a world before the wars, global warming and the division of society. [--]
10 years older and wondering how it happened. [..]]
1.0
100 miles away from the starting line, but who told you quit. [...]
30 years earlier. You ask the current you what to do next to maximize [--] 10 minutes in the past, before you were scheduled to meet a friend for coffee.
0.0
Politeness
100 miles away from nobody jerk. you find a rip in space boarding a bus on west 96th [..]]
0.3
You realize that by altering the past, [..]] 7 years ago in Cafe you have been to before. [..]
10 years in the future. you are discovering all sorts of things. it comes to you [...]
0.7
years ago Cafe you have been to before, supposed to meet with friend
10 years ago, trying figure where the next stride will take you or perhaps where [..]]
1.0
that never showed up. [...]
0.0
20 LOL that doesn't seem right fair. [--] years older. END OF STORY!! Things were going well until you started to take [...] 40 or older, happily married, and with 3 kids. [...
in the past... [-..]
in the past, an absolute fantasy. You see young version of yourself there, who [...] 10 minutes in the past, before the recent break-in at your apartment. [...]
Formality
0.3
0.7
10 minutes in the past, before the recent break-in at your office. [...]
1.0 21 years older from your prior adventures. [...]
black public hillside, and yellow sun butchered and bleeding in an ugly sky, and
20 years earlier, in a world before the great collapse. Children are playing, birds are
0.0
you know the cut of sandstone in the [--] kilometres outside of town at a main road.
chirping, and people are smiling, [..]]
Simplicity
7 years ago in crucial moment of your past. [...]
10 minutes in the past, before the recent downpour. How do you handle it? [..]
0.7 you are rumning off to play in the forest. You stand there trying figure out what to do [...] 300 years back! It is 1828 in London. You stay 1.0 in the alley until it becomes fully sunny. [..]
10 minutes in the past, before the recent downpour of rains and flooding, [-..]
30 years in the past under another name. You're married an old fling [--]]
7 years ago in parking lot looking 7 years younger. [...]
0.0
Humor
30 years back, walking through the alleys.
7 years ago in a parking lot looking 7 years younger. You see car you can't remember [..]] 20 years earlier, in high school. Your younger self is looking at you, confused. You then see yourself in high school and [-.]]
0.3
So much for not being surprised. [...]
75 years in the future, Washington DC's
0.7
Newbridge Apartments has become an urban theme park [..]]
255 years into the future, the day at the
10 minutes in the past, but you bring hand-held portal weapon with you. [...]
1.0 'harmless' age of sixty million, a desperate, crazed- looking [...]
23

---

Published in Transactions on Machine Learning Research (08/2025)
A.9
Additional multi-dimensional plots
Effect of Interpolation Parameters on WritingPrompts Attribute Score
Sentiment
Politeness
Formality
Simplicity
Humar
Attribute Score
0.6
0.6
0.5
0.5
0.2
0.5
0.5
0.4
0.4
0.6
0.6
0.3
Figure 19: Effect of a; and X; on 5-dimensional interpolation. For each attribute, we show the attribute
scores for models with the given og and X; parameters, with all four other aj = 1 and X = (1 - A:)/4. We find that increasing a; consistently increases the attribute score and increasing X consistently increases the effect of az
DExperts
Model Arithmetic
Weight Interpolation
0.05
Sentiment
0.013
0.006
0.008
0.015
0.006
0.011
0.04
Politeness
0.023
0.021
0.013
0.018
0.00
0.003
0.006
0.007
0.008
0.01
0
0.006
0.012
0.016
0.03
Formality
0.01
0.005
0.004
0.005
0.002
0.008
0.007
0.018
0.002
0.014
0.003
0.009
0.02
Simplicity
0.026
0.033
0.009
0.009
0.01
Humor
0.051 Pollteness
0.038
0.048
0.04
0.008 Pollteness
0.005 Formality
0.002 Simpiicity
0.011
0.013 Pollteness
0.027
0.024 Simplicity
0.011
0
0.00
Sentiment
Formality
Simplicity
Humor
Sentiment
Humor
Sentiment
Formality
Humor
Figure 20: Weight interpolation has entanglement lower than or comparable to prior approaches. For each pair of dimensions, we (scaled) a; = for the dimension each row and vary (scaled) ag between and for the dimension each column. We set X, = >; 0.5. Then, we report entanglement as the area under the curve of absolute value of change in attribute score as aj increases. Weight interpolation is less entangled than DExperts (Liu et al., 2021) and has comparable entanglement to model arithmetic (Dekoninck et al., 2024).
24

---

Published in Transactions on Machine Learning Research (08/2025)
DExperts
Model Arithmetic
Weight Interpolation
0.06
Sentiment
0.057
0.015
0.003
0.007
0.06
0.013
0.025
0.013
0.025
0.017
0.003
0.022
Politeness
0.018
0.005
0.006
0.04
Formality
0.021
0.01.
0
0.003
0.009
0.002
0.045
0
0.005
0.046
0.006
0.006
0.009
0.015
Simplicity
0.007
0.032
0.039
0.024
0.025
0.009
0.062
0.011
0.001
0.014
0.002
0.023
0.02
Humor
0.008
0.00
Sentiment
Pollteness
Formaliyy
Simplicity
Humor
Sentiment
Pollteness
Formality
Simplicity
Humar
Sentiment
Pollteness
Fomality
Simplicity
Humor
Figure Llama-2-13b weight interpolation has entanglement lower than or comparable prior approaches. For each pair of dimensions, we fix (scaled) a; for the dimension in each row and vary (scaled) aj between and for the dimension in each column. We set = A3 0.5. Then, report entanglement the area under the curve of absolute value of change in attribute score aj increases. Weight interpolation is less entangled than DExperts (Liu et al., 2021) and has comparable entanglement to model arithmetic (Dekoninck et al., 2024).
DExperts
Model Arithmetic
Weight Interpolation
Sentiment
0.055
0.008
0.006
0.011
0.046
0.018
0.02
0.004
0.025
0.01
0.082
0.031
0.05
Politeness
0.027
0.01
0.04
Formality
0.014
0.019
0
0.007
0.015
0.004
0.027
0
0.014
0.018
0.01
0.023
0.002
0.009
0.03
0.02
Simplicity
0 004
0.034
0.016
0.015
01.14
0.014
0.056
0.018
0.005
0.019
0.011
0.025
0.01
Humor
0.00
Sentiment
Pollteness
Formality
Simplicity
Humor
Sentiment
Pollteness
Formality
Simplicity
Humor
Sentiment
Pollteness
Formality
Simplicity
Humor
Figure Llama-2-13b weight interpolation has entanglement lower than or comparable to prior approaches. For each pair of dimensions, we fix (scaled) a; for the dimension in each row and vary (scaled) ay between 0 and for the dimension in each column. We set X; = X3 0.5. Then, report entanglement the area under the curve of absolute value of change in attribute score aj increases. Weight interpolation is less entangled than DExperts (Liu et al., 2021) and has comparable entanglement to model arithmetic (Dekoninck et al., 2024).
25

---

Published in Transactions on Machine Learning Research (08/2025)
DExperts
Model Arithmetic
Weight Interpolation
Sentiment
0.057
0.018
0.015
0.023
0.018
0.016
0.01
0.003
0.039
0.014
0.003
0.011
0.05
Politeness
0.003
0.003
0.04
Formality
0.02
0.02
0
0.002
0.006
0.005
0.01
0
0.008
0.006
0.005
0.002
0.005
0.033
0.03
0.02
Simplicity
0.003
0.018
0.02
0.016
0.000
0.011
0.007
0.006
0.001
0.021
0.021
0.03
0.01
Humor
0.002 Simplicity
0.00
Sentiment
Pollteness
Formaliyy
Simplicity
Humor
Sentiment
Pollteness
Formality
Humor
Sentiment
Pollteness
Formality
Simplicity
Humor
Figure 23: Llama-3.1-8P weight interpolation has entanglement lower than or comparable prior approaches. For each pair of dimensions, we fix (scaled) a; for the dimension in each row and vary (scaled) aj between and for the dimension in each column. We set = A3 0.5. Then, report entanglement the area under the curve of absolute value of change in attribute score aj increases. Weight interpolation has comparable entanglement to DExperts (Liu et al., 2021) and model arithmetic (Dekoninck et al., 2024).
DExperts
Model Arithmetic
Weight Interpolation
Sentiment
0.086
0.018
0.009
0.019
0.026
0.013
0.007
0.004
0.046
0.018
0.003
0.023
0.08
Politeness
0.009
0.004
0.06
Formality
0.015
0.035
0
0.003
0.008
0.009
0.008
0
0.008
0.008
0.00s
0.006
0.096
0.025
0.04
Simplicity
0 003
0.021
0.016
0.012
0.002
0.006
0.014
0.005
0.002
0.026
0.013
0.029
0.02
Humor
0.001
0.00
Sentiment
Pollteness
Formality
Simplicity
Humor
Sentiment
Pollteness
Formality
Simplicity
Humor
Sentiment
Pollteness
Formality
Simplicity
Humor
Figure 24: Llama-3.1-8B weight interpolation has entanglement lower than or comparable to prior approaches. For each pair of dimensions, we fix (scaled) a; for the dimension in each row and vary (scaled) ay between 0 and for the dimension in each column. We set X A3 0.5. Then, report entanglement the area under the curve of absolute value of change in attribute score aj increases. Weight interpolation has comparable entanglement to DExperts (Liu et al., 2021) and model arithmetic (Dekoninck et al., 2024).
26

---

Published in Transactions on Machine Learning Research (08/2025)
DExperts
Model Arithmetic
Weight Interpolation
0.05
Sentiment
0.051
0.014
0.004
0.003
0.048
0.022
0.017
0.011
0.038
0.006
0.006
0.021
0.04
Politeness
0.015
0.007
0.004
0.03
Formality
0.021
0.018
0
0.01
0.006
0.01.5
0.016
0
0.013
0.016
0.002
0.007
0.006
0.006
0.02
Simplicity
0.01
0.014
0.005
0.009
0.024
0.01
0.039
0.021
0.009
0.01
0.006
0.032
0.01
Humor
0.033
0.009
0.00
Sentiment
Politeness
Formality
Simplicity
Humor
Sentiment
Pollteness
Formality
Simplicity
Humor
Sentiment
Pollteness
Formality
Simplicity
Humor
Figure 25: Qwen3-8B-Ban weight interpolation has entanglement lower than or comparable prior approaches. For each pair of dimensions, we fix (scaled) a; for the dimension in each row and vary (scaled) aj between and for the dimension in each column. We set A3 0.5. Then, report entanglement the area under the curve of absolute value of change in attribute score aj increases. Weight interpolation has lower entanglement than DExperts (Liu et al., 2021) and model arithmetic (Dekoninck et al., 2024).
DExperts
Model Arithmetic
Weight Interpolation
0.06
Sentiment
0.061
0.004
0.04
0.005
0.05
0.028
0.011
0.012
0.055
0.015
0.011
0.031
Politeness
0.019
0.04
Formality
0.014
0.024
0
0.009
0.014
0.011
0.013
0
0.003
0.009
0.006
0.018
0.004
0.012
Simplicity
0.025
0.011
0.016
0.013
01012
0.005
0.031
0.018
0.003
0.013
0.002
0.019
0.02
Humor
0.007
0.00
Sentiment
Pollteness
Formality
Simplicity
Humor
Sentiment
Pollteness
Formality
Simplicity
Humor
Sentiment
Pollteness
Formality
Simplicity
Humor
Figure 26: Qweni-8B-Ba weight interpolation has entanglement lower than or comparable prior approaches. For each pair of dimensions, we fix (scaled) a; for the dimension in each row and vary (scaled) a, between 0 and 1 for the dimension in each column. We set X; X3 0.5. Then, we report entanglement as the area under the curve of absolute value of change in altribute score aş increases. Weight interpolation is less entangled than DExperts (Liu et al., 2021) and has comparable entanglement to model arithmetic (Dekoninck et al., 2024).
27

---

Published in Transactions on Machine Learning Research (08/2025)
DExperts
Model Arithmetic
Weight Interpolation
Sentiment
0. 0.049
0.02
0.006
0.017
0.045
0.017
0.008
0.002
0.044
0.006
0.002
0.029
0.08
Politeness
0.021
0.005
0.06
Formality
0.025
0.004
0
0.002
0.004
0.005
0.033
0
0.002
0.0227
0.01
0.008
0.009
0.006
0.04
Simplicity
0.008
0.088
0.022
0.003
0. 0.0012
0.002
0.066
0.024
0.006
0.007
0.008
0.017
0.02
Humor
0.023
0.014
0.00
Sentiment
Pollteness
Formaliyy
Simplicity
Humor
Sentiment
Pollteness
Formality
Simplicity
Humar
Sentiment
Pollteness
Fomality
Simplicity
Humor
Figure 27: Swen3-14B-BBae weight interpolation has entanglement lower than or comparable prior approaches. For each pair of dimensions, we fix (scaled) a; for the dimension in each row and vary (scaled) aj between and for the dimension in each column. We set A3 0.5. Then, report entanglement the area under the curve of absolute value of change in attribute score aj increases. Weight interpolation has lower entanglement than DExperts (Liu et al., 2021) and model arithmetic (Dekoninck et al., 2024).
DExperts
Model Arithmetic
Weight Interpolation
0.06
Sentiment
0.054
0.008
0.005
0.011
0.046
0.014
0.004
0.008
0.058
0.006
0.003
0.038
Politeness
0.009
0.04
Formality
0.016
0.018
0
0.00.
0.019
0.002
0.026
0
0.003
0.017
0.008
0.013
0.082
0.025
Simplicity
0.017
0.02
0.005
0.015
0. 0..15
0.006
0.053
0.018
0.005
0.013
0.012
0.011
0.02
Humor
0.00
Sentiment
Pollteness
Formality
Simplicity
Humor
Sentiment
Pollteness
Formality
Simplicity
Humar
Sentiment
Pollteness
Formality
Simplicity
Humor
Figure 28: Qwen3-14BBBaa weight interpolation has entanglement lower than or comparable prior approaches. For each pair of dimensions, we fix (scaled) a; for the dimension in each row and vary (scaled) a, between 0 and 1 for the dimension in each column. We set X; X 0.5. Then, we report entanglement as the area under the curve of absolute value of change in altribute score aş increases. Weight interpolation is less entangled than DExperts (Liu et al., 2021) and has comparable entanglement to model arithmetic (Dekoninck et al., 2024).
28

---

Published in Transactions on Machine Learning Research (08/2025)
A.10
Additional multi- multi-dimensional lambda simplex plots
Sentiment Score
Politeness Score
Humor Score
Sentiment À
Sentiment À
Sentiment
0.33
0.7
0.2
0.2
0.3
0.2
0.32
0.3
0.5
0.4
0.6
0.25
0.4
0.6
0.3
0.4
0.6
0.4
0.2
0.8
0.2
0.8
0.8
0.3
0.28
0.2
0.6
0.8
0.6
0.4
0.2
Politeness
Humor Politeness
Humor
Politeness
Humor
Figure 29: Effect of X on interpolation between the sentiment, politeness, and humor dimensions for a; 0. The vertices of the triangle represent the models with ay for each of the three dimensions.
Sentiment Score
Politeness Score
Humor Score
Sentiment À
Sentiment À
Sentiment À
0.52
0.81
0.5
6.8
0.9
0.2
0.8
0.2
0.8
0.2
0.6
0.4
0.46
0.6.
0.4
0.4
0.6
0.4
0.78
0.6
0.44
0.77
0.38
0.76
0.2
0.2
0.2
0.4
0.36
0.8
0.6
0.4
0.2
0.8
0.6
0.4
0.2
0.8
0.6
0.4
0.2
Politeness A
Humor
Politeness À
Humor
Politeness
Humor
Figure 30: Effect of X interpolation between the sentiment, politeness, and humor dimensions for a; 0.5. The vertices of the triangle represent the models with 0.5 for each of the three dimensions.
Sentiment Score
Politeness Score
Humor Score
Sentiment À
Sentiment À
Sentiment
0.65
0.9
0.8
0.8
0.7
0.6
0.6
.0.4
0.6
0.4
0.85
0.55
0.4
0.6
0.4
0.6
0.4
0.6
0.5
0.8
0.2
0.8
0.45
0.2
0.8
0.4
0.4
0.75
0.3
0.5
0.4
0.2
0.8
0.6
0.4
0.2
0.8
0.6
0.4
0.2
Politeness A
Humor À
Politeness
Hurno
Politeness
Humor
Figure 31: Effect of X on interpolation between the sentiment, politeness, and humor dimensions for a; = 1. The vertices of the triangle represent the models with a; = 1 for each the three dimensions.
29

---

Published in Transactions on Machine Learning Research (08/2025)
Sentiment Score
Politeness Score
Formality Score
Sentiment À
Sentiment À
Sentiment À
0.35
0.8
0.4
0.8
0.38
0.6
0.4
0.3
0.6-
0.4
0.36
0.6
0.4
0.4.
0.6
0.25
0.4
0.6
0.4.
0.6
0.35
0.2.
.08
0.2
0.88
0.2
0.8
0.3
0.3
0.28
0.8
0.6
0.2
0.8
0.6
0.4
0.2
0.8
0.6
0.4
0.2
Politeness
Formality
Politeness
Formality À Politeness
Formality À
Figure 32: Effect of X; on interpolation between the sentiment, politeness, and formality dimensions for a; = 0. The vertices the triangle represent the models with a; = for each of the three dimensions.
Sentiment Score
Politeness Score
Formality Score
Sentiment
0.5
Sentiment À
Sentiment À
0.8 0.48
0.8
0.44
0.8
0.6
0.4
0.46
0.6
0.4
0.42
0.6
.0.
0.4
0.6
0.4
0.6
0.4
0.4
0.6
0.2
0.8
0.2
0.8
0.2
.088
0.42
Politeness
0.8
0.6
0.4
0.2
Formality
Politeness
0.8
0.6
0.4
0.2
Formality
Politeness
0.8
0.6
0.4
0.2
Formality A
Figure 33: Effect of A; on interpolation between the sentiment, politemess, and formality dimensions for ag = 0.5. The vertices of the triangle represent the models with a, = 0.5 for each of the three dimensions.
Sentiment Score
Politeness Score
Formality Score
Sentiment
Seatiment À
Sentiment À
0.8
0.75
0.6
0.8
0.8
0.8
0.7
0.5
0.4
0.6
.0.
0.6
0.4
0.4
0.6
0.6
0.4
0.5
0.4
0.6
0.55
0.5
0.2
0.8
0.5
0.2
.088
0.2
0.8
0.45
0.45
0.8
0.6
0.4
0.2
0.8
0.6
0.4
0.2
0.4
0.8
0.6
0.4
0.2
Politeness
Formality
Politeness
Formality Politeness
Formality À
Figure 34: Effect of X; on interpolation between the sentiment, politeness, and formality dimensions for a; = 1. The vertices the triangle represent the models with a; = for each of the three dimensions.
30

---

Published in Transactions on Machine Learning Research (08/2025)
Sentiment Score
Politeness Score
Simplicity Score
Sentiment À
Sentiment À
Sentiment À
0.8
0.35
0.8
0.6
0.4
0.3
0.6-
0.32
0.6
0.36
0.31
0.34
0.4
0.6
0.4
0.6
0.4g
0.6
0.25
0.3
0.32
0.2
0.8
0.2
00.8
0.29
0.2
0088
0.3
0.2
0.28
Politeness
0.8
0.6
0.2
Simplicity Politeness
0.8
0.6
0.4
0.2
Simplicity
Politeness
0.8
0.6
0.4
0.2
Simplicity
Figure 35: Effect of X on interpolation between the sentiment, politeness, and simplicity dimensions for a; = 0. The vertices the triangle represent the models with a; = for each of the three dimensions.
Sentiment Score
Politeness Score
Simplicity Score
Sentiment
0.5
Sentiment À
0.44
Sentiment
0.49
0.43
0.8
0.8
0.8
0.48
0.42
0.6
0.4
0.47
0.6
0.4
0.41
6
0.4
0.4
0.4
0.4
0.6
0.4
0.6
0.4
0.6
0.8
.0.8
0.38
0.8
0.36
0.37
0.34
0.8
0.6
0.4
0.2
0.8
0.6
0.4
0.2
0.36
0.8
0.6
0.4
0.2
Politeness A
Simplicity
Politeness
Simplicity À Politeness
Simplicity
Figure 36: Effect of X; on interpolation between the sentiment, politeness, and simplicity dimensions for a; = 0.5. The vertices of the triangle represent the models with a; = 0.5 for each of the three dimensions.
Humor Score
Formality Score
Simplicity Score
Humor A
Humor
Humor À
0.5
0.45
0.7
0.8
0.2
0.45
0.8
0.2
0.6.
0.4
0.6
0.6
0.4
0.6
0.4
0.4
0.5
0.4
0.6
0.4
0.6
0.4
0.6
0.35
0.2-
0.3
0.3
0.2
Formality A
0.4
Simplicity À Formality
0.8
0.6
0.4
0.2
Simplicity
Formality 0.8
0.6
0.4
0.2
Simplicity À
Figure 37: Effect of X; on interpolation between the humor, formality, and simplicity dimensions for a₂ = 0. The vertices of the triangle represent the models with = 0 for each of the three dimensions.
31

---

Published in Transactions on Machine Learning Research (08/2025)
Humor Score
Formality Score
Simplicity Score
Humor
Humor À
0.52
Humor À
0.47
0.5
0.2
0.86
0.2
0.8
0.2
0.45
0.4
0.4
0.46
0.4.
0.4-
0.4-
0.43
0.82
0.44
0.2
0.8
0.2-
00.8
0.42
0.2
0.8
0.41
0.4
0.8 0.6
0.4
0.8
0.6
0.4
0.2
Formality A
Simplicity A Formality A
Simplicity À Formality
Simplicity À
Figure 38: Effect of X; on interpolation between the humor, formality, and simplicity dimensions for o2 0.5. The vertices of the triangle represent the models with oş 0.5 for each of the three dimensions.
WikiText Perplexity Score
WikiText Perplexity Score
WikiText Perplexity Score
Sentiment A
Sentiment
Sentiment
5.1
5.02
5.02
5.05
4.98
0.6.
0.6
0.4
4.96
0.5
0.4
4.96
0.4
0.6
0.4
0.6
4.94
0.4
0.6
4.92
0.2
0.2
4.9
0.2
4.88
4.00
Politeness
Humor
Politeness
Humor
Politeness
Humor
(a) 02 0.0
(b) a, 0.5
(c) 04 = 1.0
Figure 39: Effect of X; perplexity for interpolation between the sentiment, politemess, and humor dimensions for various a; values. The vertices of the triangle represent the models with the given a; value for cach of the three dimensions. The perplexity for cach model is bounded above by the perplexities of the fine-tuned anchor models.
WikiText Perplexity Score
WikiText Perplexity Score
WikiText Perplexity Score
Sentiment.J
Sentiment
Sentiment
5.1
5.02
5.02
0.8
0.2
5.05
0.8
02
0.8
0.2
5
4.98
0.6
0.6
0.6
4.96
4.94
4.95
4.92
0.0
4.94
4.92
4.9
4.9
0.8
0.6 0.4 0.2
0.8
0.6
0.4
0.2
1,80
0.8
0.6
0.4
0.2
Politeness
Formality
Politeness
Formality
Politeness À
Formality
(a) 0. = 0.0
(b) 0.5
(c) a, = 1.0
Figure 40: Effect of X; on perplexity for interpolation between the sentiment, politeness, and formality dimensions for various a; values. The vertices the triangle represent the models with the given a value for cach of the three dimensions. The perplexity for cach model bounded above by the perplexities of the fine-tuned anchor models.
32

---

Published in Transactions on Machine Learning Research (08/2025)
WikiText Perplexity Score
WikiText Perplexity Score
WikiText Perplexity Score
Sentiment À
Sentiment X
Sentiment À
0.2
5.05
0.8
0.2
0.8
0.2
4.98
0.4
0.4
4.96
0.4
0.4
4.94
0.4
4.94
4,95
0.2
0.8
0.2
0.8
4.92
0.2
0.8
4.92
4.9
4.9
4.9
0.6
0.2
0.8
0.6
0.4
0.2
4.88
0.8
0.6
0.4
0.2
Politeness A
Simplicity À Politeness
Simplicity À Politeness
Simplicity À
(a) 0.0
(b) 0.5
(c) 1.0
Figure 41: Effect of Å; on perplexity for interpolation between the sentiment, politeness, and simplicity dimensions for various a; values. The vertices of the triangle represent the models with the given 0; value for each the three dimensions. The perplexity for each model bounded above by the perplexities of the fine-tuned anchor models.
WikiText Perplexity Score
WikiText Perplexity Score
WikiText Perplexity Score
Humor À
Humor À
Humor À
4.98
5.02
0.9
0.2
0.8
0.2
0.8
0.2
0.4
4,94
.0.
4.94
0.4
4.96
0.4
4.92
0.4
4.92
0.4
4.94
0.2
0.8
0.2
.0.8
4.9
0.2
.08
4,9
0.8
0.6
0.4
0.2
0.8
0.6
0.4
0.2
0.8
0.6
0.4
0.2
4.86
Formality A
Simplicity À
Formality A
Simplicity À
Formality X
Simplicity X
(a) a2 = 0.0
(b) a, 0.5
(c) = 1.0
Figure 42: Effect of X; on perplexity for interpolation between the humor, formality, and simplicity dimensions for various a; values. The vertices of the triangle represent the models with the given a value for each of the three dimensions. The perplexity for each model is bounded above by the perplexities of the fine-tuned anchor models.
33

---

Published in Transactions on Machine Learning Research (08/2023)
A.11 Multi-dimensional scaling (MDS) analysis of fine-tuned models
We project the weights of the LoRA fine-tuned endpoint models, well
of the interpolated models,
into two dimensions using multi-dimensional scaling (MDS). As shown in Figure 43, we find that the interpolating between the endpoint fine-tuned models generally results in models that are closer to the base model. This is expected behavior since we would anticipate that the base model is fairly neutral with respect to all attributes.
Impolite
100
Polite
80
Humorbus
40
Nonhumorous
legative Sentiment entimmen
Positive
100
880
-60
-40
-20
20
60
sim Pis
Complex
Infermal
40
Formal
Figure 43: Multi-dimennional scaling (MDS) plot for the fine-tuned models and linear interpola- tions. This plot shows the 2-dimensional MDS projection of the fine-tuned anchor models and the models interpolated intervals of 0.1. This corresponds to the models in Figure 2.
34

---

Published in Transactions on Machine Learning Research (08/2025)
A.12 Similarity between pairs of fine-tuned models
We compute the cosine similarities between LoRA weights in Figure 44 and use the average squared L2 norm of the difference between the LoRA updates to analyze the distances between models in Figure 45. For the cosine similarity, the models appear relatively orthogonal, except for the opposing models from the same attribute dimension. In terms of L2 norm, the models fine-tuned on the classes with attribute score of (positive sentiment, polite, simple, formal, humorous) tend to be closer to the other models than the models fine-tuned on classes with attribute score 0. We also find that the polite and impolite LoRA fine-tuned endpoint models are the farthest from the other models average. This is consistent with the results from the MDS plot (Figure 43).
Average Cosine Similarity Between LoRA Layers
Pas. Sent.
Neg. Sent.
Polite-
Impolite
0.6
Simple
Complex
0.4
Formal
Informal
0.2
orous
0,025
Nonhumorous orous
0.042
0.027
6.1
0.000
s.cea
5.042
0.044
0.
Pos, Sent.
Neg. Sant
Polite
Impal liee
Simple
Compler
Formal
Informal
Humoruus
Nonhumorous murous
Figure 44: Cosine similarity of LoRA weights averaged across layers between each pair of fine-tuned anchor models. The LoRA weights all relatively orthogonal to each other, except some of the two endpoint models for the same altribute are less orthogonal to each other, well as the politeness and humorous models.
Average Squared L2 Norm Between LoRA Layers
Posilvve Sentiment
h.a66
9:22
6:71
116
Negative Sentiment-pne 5.000
9.52
1.5
0.32
2.24
0.32
0.48
0.28
0:35
1.4
Polite
0.01
0.71
28
1.2
Impolite
0.94
1.4
155
1.3
1.0
Simple
5008
0:
0.59
1.3
9.39
2.5
0.64
0:22
0.31
Complex
0.8
Formal-
0.21
5.8
2.5
2.42
0:48
0:22
0.6
Informal
0.44
R/2
0.4
0.2
Nonhumorous
0.32
0.38
0.86
1.3
0.31
9:42
0,32
041
0.0
Positive Sentiment
Negative Sentiment
Polite
Impolite
Simple
Complex
Famaal
Informal
Humarous
Monhumarous
Figure 45: Average pairwise squared L2 norms between LoRA layers. The fine-tuned anchor models trained on the class with attribute score of 1 tend be closer the other models than those trained on the class with attribute score of O. The polite and impolite models are the farthest from the other models.
35


Reviews:
[{"review_id": "WvzgKkmxum", "reviewer": "Reviewer_zLdD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strength\n\n-  Identifies a clear λ threshold for when attribute interference becomes problematic.\n-  Uses perplexity, attribute accuracy, and correlation analysis for a well-rounded view.\n-  Exhaustively evaluates all 26 subsets of 5 attributes.\n\n# Weaknesses\n- Results are only shown for Llama2-7B, not other models.\n- Relies entirely on RoBERTa classifiers for attribute control.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a method called continuous linear weight interpolation to blend LoRA fine-tuned models. It starts by training two \"anchor\" models for each style attribute—like simplicity, formality, or sentiment—representing opposite extremes.\n\nA two-parameter system controls the blending: αᵢ adjusts how much to interpolate between anchor models for each attribute, while λᵢ controls each attribute's overall influence on the final model.\n\nThis results in a smooth, continuous space of models spanning the convex hull of all anchor models, enabling fine-grained control over multiple stylistic features.\n\nThis paper addresses key gaps in controllable text generation by enabling continuous, real-time control over multiple text attributes through simple weight interpolation. Its core innovation is modeling the space of all attribute combinations as a navigable convex hull, allowing users to fine-tune outputs (e.g., \"70% formal, 30% humorous\") without retraining. Unlike prior work that relied on discrete categories or complex ensembles, this approach reframes CTG as a smooth mathematical navigation problem.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Acknowledge the limitation in the experimentation results, like only Llama2-7B is tested.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "2CSbqr9kvW", "reviewer": "Reviewer_YfNs", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths:\n\n- The idea of using LoRA-based weight interpolation to span a continuous control space is elegant and practical.\n\n- The method supports simultaneous control of several style attributes, outperforming prior methods in precision and controllability.\n\n- Experiments across five attributes show strong trends, low entanglement, and controlled behavior, supported by clear visualizations (e.g., simplex plots, entanglement metrics).\n\n- The paper is well-organized, with helpful figures (especially Figure 1) and detailed methodology.\n\n## Weaknesses:\n\n- Only one base model (LLaMA2-7B) with five style attributes is tested. It's unclear whether the approach generalizes to other models, e.g., newer model families like Qwen-2.5 and Llama-3.\n\n- Attribute strength is measured using RoBERTa-based classifiers. While standard, this risks overestimating control quality without human validation.\n\n- Outside the [0,1] interpolation range, generation quality degrades, limiting usable control space.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a framework for continuous and controllable text generation via linear interpolation of LoRA-adapted LLMs. For each stylistic attribute (e.g., simplicity, formality), the authors fine-tune two anchor models on opposite extremes, then interpolate between them using per-attribute weights. By mixing across attributes, the method parameterizes a convex hull of controllable models. Experiments on Llama2-7B across five style attributes show smooth, predictable control with minimal attribute entanglement, outperforming DExperts and model arithmetic in precision and efficiency.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Extending the analysis to more model families. \n\n- Even a small-scale human study or readability/diversity metrics would complement the classifier-based scores and confirm perceptual control.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZfrciJV8Dn", "reviewer": "Reviewer_JTFg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The method is simple and easy to understand and implement, since the main idea is just linear interpolation of the weights of fine-tuned model variants\n* The experimental evaluation and results are intuitive and effectively show that the method works as expected\n\nWeaknesses:\n* The authors could elaborate more on the advantages/disadvantages of the proposed method in comparison with related work\n* The experimental evaluation seems to rely on a single LLM", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a method for controlling certain attributes of LLM outputs (e.g., politeness, humor) by averaging the weights of model variants fine-tuned to increase or decrease each individual attribute. The method relies on two types of parameters: first, parameters that independently weigh each attribute's level, and second, parameters that weigh attributes relative to one another (i.e., forming a simplex). The authors conduct a series of experiments to evaluate their method against two baselines. They first focus on changes to individual attributes independently, demonstrating that linear interpolation successfully increases or decreases each target attribute. The authors also show that extrapolation beyond the weights of the two fine-tuned endpoint models is possible, though it quickly leads to instability and less predictable outputs. Finally, they demonstrate that adjusting the relative weights of attributes does increase the presence of the respective attribute in the model's responses, with some (but limited) interference between different attributes and their corresponding parameters.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Here are some more specific comments regarding the paper:\n\n* In terms of writing, I found the introduction of the paper somewhat hard to follow. To me, it only made sense after I had started reading section 2. The authors start talking about \"distinct and continuously evolving preferences of users\" without explaining via concrete examples what they mean by \"preferences\". Then they briefly argue that user preferences are continuous before jumping into a detailed discussion of prior work on controllable text generation, which makes the argument regarding continuous preferences (i.e., the main point of the paper) quite weak. Specifically, the phrase \"expressing fine-grained continuous preferences (e.g., simplifying a response by 25%) is often difficult in---inherently discrete---natural language\" is particularly confusing. It is unclear how the fact that natural language is discrete is relevant to the argument.\n\n* The experimental evaluation depends on a single model (Llama2-7b). It would be interesting to see whether the effectiveness of linear weight interpolation changes depending on (i) the model size and (ii) different families of models. This is missing from the current version of the paper.\n\n* In equation 2, I believe that the term $\\theta_\\text{PRE}$ is unnecessary in the first equation since its appearance in the second equation would result by the convex combination of $\\theta_{+i}$ and $\\theta_{-i}$.\n\n* In section 2.4, the authors mention that the two baselines require 2*number of dimensions + 1 inference passes, but it is not immediately clear why. More generally, since the authors compare with only 2 baselines, I believe the paper would benefit from a better description of those two baselines. What exactly do they do? I believe that the baseline \"model arithmetic\" is just a different way of prompting the model to generate responses with the required attributes, so it would be quite easy to describe the method in words to provide more context.\n\n* In figure 2, the baseline \"DExperts\" seems to behave similarly to the method that the authors propose. Again, some clarity regarding what this baseline does would help, but I think it would also be useful to emphasize more what are the benefits of the proposed method in comparison to this baseline.\n\n* It is a bit secondary to the main topic of the paper, but it would be useful if the authors could look into what attribute of the network itself leads to extrapolation not working, rather than just reporting properties of the output (i.e., perplexity). One idea could be to look into the norms of the network's weights (e.g., the spectral norm) and how these vary as a function of $\\alpha$.\n\n* In section 3.1, the authors write that interpolating the weights of fine-tuned versions of a model leads to controllable output when considering *up to five* attribute dimensions. Is there some problem with the method when considering more than five attributes? If so, could the authors add relevant experimental results?\n\n* In Figure 7 (left), the humor score seems to increase comparably when increasing the parameter $\\lambda$ for both humor and formality. Does that imply that increasing the parameter for formality also causes an increase in humor? That's a bit strange since one would expect outputs with humor to be less formal. Related to that, in Figure 8, shouldn't there be an entanglement between formality and humor? Can the authors clarify this?\n\n* The discussion of related work in section 4.1 is a bit shallow, with relatively uninformative descriptions of what other methods are doing. Can the authors explain what makes the adaptation of those methods to the multiple attribute setting difficult or not possible?\n\n* In terms of limitations, something that the authors do not mention is that in order for their method to work, one needs to perform fine-tuning twice for each of the attributes under consideration, which presents a significant overhead compared with approaches that control the outputs via prompting, for example. I think this needs to be highlighted.\n\n* In terms of future work, do the authors think that their method could be used to generalize to other attributes without requiring additional fine-tuning? For example, if you have attributes \"humor\" and \"irony\", I can imagine the interpolation of the fine-tuned models could lead to outputs that have a high score for an attribute \"sarcasm\".", "broader_impact_concerns": "The work already contains a broader impact statement that sufficiently discusses its ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)