# LLM-based Co-Evolution of Configurable Software Systems

Nada Zine, Clément Quinton, Romain Rouvoy

► To cite this version:

Nada Zine, Clément Quinton, Romain Rouvoy. LLM-based Co-Evolution of Configurable Software Systems. SPLC'25 - 29th ACM International Systems and Software Product Line Conference, Sep 2025, A Coruña, Spain. hal-05090995

HAL Id: hal-05090995

https://hal.science/hal-05090995v1

Submitted on 20 Jul 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# LLM-based Co-Evolution of Configurable Software Systems

Nada Zine

Univ. Lille, CNRS, Inria, Centrale Lille,
UMR 9189 CRISTAL, F-59000, Lille

Lille, France

nada.zine@inria.fr

Clément Quinton

Univ. Lille, CNRS, Inria, Centrale Lille,
UMR 9189 CRISTAL, F-59000, Lille

Lille, France

clement.quinton@univ-lille.fr

Romain Rouvoy

Univ. Lille, CNRS, Inria, Centrale Lille,
UMR 9189 CRISTAL, F-59000, Lille

Lille, France

romain.rouvoy@univ-lille.fr

## Abstract

Software Product Lines (SPLs) and Feature Models (FMs) are de facto standards for managing variability in software systems. However, maintaining an up-to-date FM during software evolution is particularly challenging. Ensuring its consistency with the artifacts of an SPL requires co-evolving them alongside the developed system. When performed manually, this co-evolution process is tedious and error-prone, highlighting the need for automated support. Yet, little attention has been given to the automation of co-evolution between FM and source code.

In this paper, we explore the potential of open-source Large Language Models (LLMs) to fill this gap. Specifically, we investigate the extent to which LLMs can support bidirectional co-evolution: from FM to source code—where modifications in the FM drive changes in the code—and from source code to FM—where updates in the code are reflected back into the FM. We evaluate our LLM-based approach on a real-world configurable system. Our results demonstrate that co-evolution from source code to FM achieves F1 scores ranging from 0.93 to 1.0, while co-evolution from FM to source code achieves F1 scores between 0.41 and 0.99. These findings highlight the potential of LLMs to support this co-evolution process, while also showing limitations and suggesting areas for improvement, particularly for the co-evolution from FM to code. Additionally, we conduct a comparative study across various LLMs, revealing how LLM choice affects co-evolution and, incidentally, how it affects model and code generation. Up to a certain size limit, larger LLMs tend to produce more accurate and stable outputs than smaller ones, however, this influence is less pronounced in the code generation task.

Overall, our work opens a new research avenue where LLMs are leveraged for automating the co-evolution between configurable software systems and variability models.

## CCS Concepts

* Software and its engineering → Software product lines; 
* Computing methodologies → Natural language processing.

## Keywords

Software Product Lines, Large Language Models, Feature Models, Co-evolution

<figure><img src="image_2.png" /></figure>

This work is licensed under a Creative Commons Attribution 4.0 International License.
SPLC-A '25, A Coruña, Spain

© 2025 Copyright held by the owner/author(s).

ACM ISBN 979-8-4007-2024-6/2025/09

https://doi.org/10.1145/3744915.3748460

## ACM Reference Format:

Nada Zine, Clément Quinton, and Romain Rouvoy. 2025. LLM-based Co-Evolution of Configurable Software Systems. In *29th ACM International Systems and Software Product Line Conference - Volume A (SPLC-A '25), September 1-5, 2025, A Coruña, Spain*. ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3744915.3748460

## 1 Introduction

With the increasing demand for customized, software-intensive systems, organizations must manage diverse feature sets meeting varying customer requirements. Traditionally, many companies have relied on the clone and own approach, where new software variants are created by copying and modifying existing features [43]. While straightforward and intuitive [27, 43], this method lacks a structured methodology for systematic reuse, making maintenance difficult as the number of variants grows [31]. This growing complexity has led to the emergence of Software Product Lines (SPLs), which offer a structured and scalable solution for variability management [7, 23, 56]. SPL engineering enables the systematic development of related software variants that share common features, while accommodating necessary differences [13]. Studies have demonstrated that SPLs are more effective than clone and own approaches in managing variability and reducing long-term costs [28].

Adoption of SPLs, nonetheless, requires a significant initial investment and demands continuous maintenance efforts. As a result, it is often seen as less attractive than the clone and own approach [9, 28], which leaves feature extraction and systematic variability management to later [54]. Yet, even delayed, the feature extraction process remains largely manual and costly, posing a significant barrier to SPL adoption [8]. To ease their engineering and variability management, various variability modeling techniques have been proposed [21, 24, 62], such as Feature Models (FMs) [10, 25]. To take advantage of these models, the synchronization of the artifacts involved in the evolution of the SPL is necessary. This requirement has motivated research on SPL evolution from multiple perspectives. Some studies have focused on identifying common evolution scenarios and patterns [55, 69], while others have addressed consistency management [37, 57] and change impact analysis [30, 47, 50, 53]. Some contributions, such as those by Seidl et al. and Kebaili et al., explored co-evolution between FMs and solution space models, and between metamodels and source code, respectively [42, 63]. However, to the best of our knowledge, no prior work has examined automated, bidirectional co-evolution directly between FMs and source code. Yet, managing FM-to-code co-evolution is particularly important. First, FM are the most widely adopted artifacts for representing SPLs [14, 24, 51]. Their central
role has led to the development of numerous automated FM analysis techniques, making it a key concern within the SPL community [12, 32]. Leveraging these analyses requires synchronized and consistent artifacts, as misaligned co-evolution between FM and code can introduce residual inconsistencies and undermine the applicability of these analyses [29, 40]. Second, among all artifacts in SPLs, source code is one of the most frequently co-evolving with FM [51], reinforcing the need for dedicated support. Recent advances in Large Language Models (LLMs) offer new opportunities to address this challenge [35]. Studies have demonstrated that LLMs can support various SPL-related tasks such as variability modeling and implementation [4, 33], reverse-engineering product variants into SPLs [5, 64], and generating reusable artifacts and feature mappings from a given FM [65]. LLMs have also been applied to evolution-related scenarios, such as maintaining consistency between metamodels and code [41]. These results suggest that LLMs could be effectively leveraged to support the co-evolution of variability models and source code.

In this paper, we address the gap in FM and code co-evolution by proposing an LLM-based approach for automating the co-evolution between FM and code—i.e., evolving the FM when the code is changed and conversely. To achieve this, we focus on prompt engineering and incorporate a response validation loop to verify the LLM’s output format and refine it when necessary. We evaluate our approach on a real-world configurable system, using various open-source LLMs to assess the impact of the LLM choice on this process. The key contributions of this paper can therefore be summarized as:

* We propose an approach to assess to what extent various open-source LLMs can support the co-evolution process from code to FM.

* Using the same approach, we further explore their abilities to handle the reverse direction, from FM to code.

* Incidentally, we evaluate these LLMs effectiveness in model and code generation tasks.

The remainder of the paper is structured as follows. In Section 2, we present our approach in detail. Section 3 shows the methodology used to evaluate the approach and the results of this evaluation are reported in Section 4. We address potential threats to validity in Section 5, followed by further discussions in Section 6. Section 7 reviews related work. Finally, we conclude the paper and outline our future work in Section 8.

## 2 LLM-based Automated Co-Evolution

Figure 1 provides an overview of our LLM-based approach to co-evolve an SPL’s FM and source code. It starts with two inputs: the previous version of the system and its evolved counterpart. Depending on the co-evolution direction, the evolved version may either include updated code (therefore requiring the FM to be adapted) or an updated FM (requiring corresponding code changes). Based on the chosen direction, the Data Extraction phase retrieves relevant data from both versions to support the specific transformation objective. This data is then passed to the Prompt Generation step, which constructs an input prompt using prompt engineering techniques. The generated prompt is submitted to the LLM, and the

resulting output is evaluated by the Validation phase, which verifies its correctness and conformance to the expected format. If the validation fails, the process loops back to the Prompt Generation for prompt refinement and re-submission to the LLM, forming a self-correcting feedback loop. Once validated, the output is passed to the Modification step, which applies the proposed modifications—either updating the FM or generating the necessary code—depending on the co-evolution direction.

<figure><img src="image_3.png" /><figcaption>Figure 1: Overview of the proposed approach for FM and code co-evolution.</figcaption></figure>

### 2.1 Data Extraction

The first phase of our approach focuses on extracting the most relevant project information. This is crucial as LLMs are constrained by limited context windows, which usually prevent them from processing long sequences in their entirety [70]. To address this constraint, we tailor the data extraction process to the chosen co-evolution direction, ensuring that only the most relevant information is included. Each direction serves a different purpose: source code to FM involves model generation, while FM to source code involves code generation. As such, the requirements differ for each direction. For the source code to FM direction, the following elements are extracted:

* Source code differences (*diff*) capture the code changes applied between version $v_n$ and the evolved version $v_{n+1}$ to highlight modifications made during the evolution process.

* Current FM refers to the FM prior to evolution.

* Metadata includes commits-related information to provide additional context.

In the opposite direction, from FM to code, the following elements are extracted:

* FM differences (*diff*) capture the model changes between version $v_n$ and the evolved version $v_{n+1}$ of the FM, to drive code modification.

* Project structure describes the organization of the existing codebase to ensure consistency and valid integration of newly generated code.

* Code examples include selected snippet from the original project. The purpose is to guide the generation process, as
the LLM lacks sufficient context for introducing new features
or adhering to existing coding conventions.

* **Metadata** includes commits-related information for contextualization, similar to the source code to FM direction.

## 2.2 Prompt Generation

For prompt generation, we employ prompt engineering techniques. We chose not to use fine-tuning, which involves retraining the model on domain-specific data to improve its performance [72], as it consumes a large amount of resources and requires a lot of data. While *Retrieval Augmented Generation* (RAG), used to enhance the model's responses with dynamically retrieved data [34], presents a more resource-efficient alternative to fine-tuning, it still requires a solid dataset that spans the project's evolution across multiple commits. Given these constraints, we designed a prompt structure independent of such techniques. Regardless of the co-evolution direction, the prompts adhere to a standardized format, illustrated in Figure 2a for FM to source code, and in Figure 2b for source code to FM.

Each prompt comprises the following key components:

* **Task** defines the objective the LLM must achieve.

* **Input** provides the direction-specific data extracted during the Data Extraction phase.

* **Instruction** offers additional guidance or constraints to further clarify the desired behavior.

* **Output** specifies the expected structure and semantics of the LLM's response. While we employed *virtual Large Language Model* (vLLM) [44] to enforce structured outputs, it remained necessary to clarify the meaning of each output field explicitly.

* **Examples** include code samples from the initial project version, driving the model in aligning with existing design and implementation patterns.

To reach the final prompt design, we began with a simple version that included necessary contextual information followed by an instruction to enclose the output within custom tags. However, early experiments revealed that this approach lacked robustness: the expected format was often not respected, making parsing unreliable. To address this issue, we switched to the structured output mode of vLLM, requesting responses in JSON format. This mode leverages a predefined JSON schema specifying the required fields and their types, which helps guide the LLM in generating well-structured outputs. We also explored enriching the prompts with additional contextual elements, such as including example FMs in the code-to-FM direction and employing prompting strategies like *Chain of Thought* (COT) [71]. However, these additions did not improve the reliability of the responses and, in some cases, proved counterproductive.

## 2.3 Response Validation Loop

After receiving the output from the LLM, a validation step is performed to ensure the response adheres to the expected format and to mitigate potential hallucinations. This step is essential for ensuring the reliability and applicability of the generated artifacts.

In the code-to-FM direction, the response is automatically validated against the expected FM syntax—an XML format in our case. Currently, this validation is limited to syntactic correctness. If the format is valid, the FM is extracted and saved. If the validation fails, the output is returned to the LLM along with an explanation of the encountered errors. The model is then prompted to regenerate a corrected version. This process is iterated until a valid response is produced or a predefined maximum number of attempts is reached.

A similar validation mechanism is applied in the FM-to-code direction, where the validation process automatically verifies that each new or modified code element includes a valid, writable file path and contains non-empty content. This ensures the generated code can be seamlessly integrated into the existing project structure.

## 3 Methodology

This section describes the methodology employed to evaluate the ability of LLMs to support the co-evolution between FM and code within our proposed approach. We first present the addressed research questions, followed by a description of the dataset. We then introduce the selected LLMs, outline the experimental setup, and detail the conducted experiments. Finally, we describe the metrics used for benchmarking.

### 3.1 Research Questions

**RQ1:** To what extent can LLMs support co-evolution from source code to FM, and how does model choice influence this process? This research question addresses the most common co-evolution scenario, where developers modify the source code and subsequently need to update the corresponding FM. We aim to investigate how effectively LLMs can support this co-evolution direction and whether variations in the LLM choice influence the process. Investigating this variation not only strengthens our benchmark but also enables us to assess the impact of LLMs on the quality of the generated models, an area that, unlike code generation, has received comparatively less attention, particularly in the context of open-source LLMs.

**RQ2:** To what extent can LLMs support co-evolution from FM to source code, and how does model choice influence this process? This question explores the reverse co-evolution scenario, where changes originate from the FM and must be reflected accordingly in the code. Although less common in practice, this approach aligns with model-driven engineering principles that promote initiating modifications at the variability model level [11], thus ensuring consistency across the system before implementing code-level changes.

### 3.2 Software Product Line under Study

To evaluate our approach, we selected RobocodeSPL,<fnref n="1" /> an SPL designed to generate robots for the Robocode environment,<fnref n="2" /> where robots engage in battles based on predefined behaviors and strategies [52]. RobocodeSPL comprises 92 features, covering core functionalities, such as radar, targeting, movement, enemy selection, and gun systems, along with various optional features. It includes

<footnote n="1">¹https://github.com/but4reuse/RobocodeSPL_teaching</footnote>

<footnote n="2">²https://github.com/robo-code/robocode</footnote>
<figure><img src="image_4.png" /><figcaption>(a) Prompt template for the FM to source code co-evolution.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>(b) Prompt template for the source code to FM co-evolution.</figcaption></figure>

<caption>Figure 2: Prompt templates for bidirectional co-evolution between FM and source code.</caption>

72 leaf features and defines approximately 1.3 million valid configurations [36]. In their foundational work, Martinez et al. [52] reverse-engineered Robocode variants to derive this SPL, providing both the initial and final version of the FM.

RobocodeSPL thus seems well-suited to assess our LLM-based co-evolution approach. However, we identified several limitations. First, realistic evolution changes do not occur in a single step, but rather progressively, ideally grouped by functionalities. Second, real-world FM evolution encompasses more than feature addition, as FM can be subject to four types of modifications: refactorings, specializations, generalizations, and arbitrary edits [69]. Finally, some features in the final FM were not implemented in the associated source code, which would have affected the co-evolution. To address these issues, we adapted the SPL. First, to align with evolution principles and make the SPL more representative of a real-world scenario, we decomposed the evolution into multiple incremental changes, incorporating the four different types of modifications possible in a FM. Second, we removed from the final FM all features that were not implemented in the corresponding source code.

Table 1 summarizes the seven versions of the RobocodeSPL project we ended up with. It shows, for each stage, the number of concrete features and valid configurations computed with the Flamapy tool,<fnref n="3" /> as well as the type of modification applied from a version to the following one. Figure 3a shows the initial version ($v_0$) of the FM, which remained consistent with the original FM proposed by Martinez et al., while Figure 3b depicts the final version ($v_7$) after applying all the evolutions and modifications. During the refactorization phase ($v_1$), we simulated renaming operations. For the generalization evolution, we decomposed the additions made by the original authors of RobocodeSPL, grouping them by

<table><caption>Table 1: Overview of the RobocodeSPL evolution scenarios and versions. Related commits can be found in the replication package.</caption><thead><tr><td>Versions</td><td>Concrete Features</td><td># Config</td><td>Evolution</td></tr></thead><tbody><tr><td>$v_0$</td><td>13</td><td>108</td><td>Initial</td></tr><tr><td>$v_1$</td><td>13</td><td>108</td><td>Refactorization</td></tr><tr><td>$v_2$</td><td>20</td><td>486</td><td>Generalization</td></tr><tr><td>$v_3$</td><td>21</td><td>648</td><td>Arbitrary</td></tr><tr><td>$v_4$</td><td>31</td><td>2,808</td><td>Generalization</td></tr><tr><td>$v_5$</td><td>55</td><td>36,504</td><td>Generalization</td></tr><tr><td>$v_6$</td><td>62</td><td>117,936</td><td>Generalization</td></tr><tr><td>$v_7$</td><td>59</td><td>104,328</td><td>Specialization</td></tr></tbody></table>

functionality. The first generalization ($v_2$) introduced radar-related functionalities, followed by targeting ($v_4$), movement ($v_5$), and gun system ($v_6$). Arbitrary edits ($v_3$) involved both feature additions and removals, while specialization ($v_7$) focused on removing specific features.

## 3.3 Selected LLMs

While prior LLM-based SPL studies often relied on proprietary GPT-based models [1, 4, 5, 19, 20, 33, 41], such models limit reproducibility and transparency. Our study, therefore, focuses on open-source LLMs as more accessible alternatives. We based our LLMs selection on BigCodeBench,<fnref n="4" /> a benchmark designed to evaluate tool use and complex instruction-following capabilities of LLMs [73]. We applied the following filtering criteria: (i) open-source availability, (ii) less than 14B parameters, (iii) support for at least 32K tokens of context, and (iv) compatibility with our hardware for inference.

<footnote n="3">³https://ide.flamapy.org/</footnote>

<footnote n="4">⁴https://bigcode-bench.github.io/</footnote>
<figure><img src="image_6.png" /><figcaption>(a) Excerpt of the initial FM of RobocodeSPL ($v_0$).</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>(b) Excerpt of the final FM of RobocodeSPL ($v_7$).</figcaption></figure>

<caption>Figure 3: Evolution of the RobocodeSPL feature model between versions $v_0$ and $v_7$.</caption>

These criteria were empirically refined during experimentation. We observed that the size of our data required at least 32K tokens of context, and that models up to 14B already offered strong performance with our current machine and dataset. Therefore, there was no need to pursue larger models and more powerful machines, in particular considering environmental issues. For completeness, we also included the Qwen 32B variant to illustrate the performance trade-offs. Table 2 lists the selected LLMs<fnref n="5" />. For the sake of clarity, in the remainder of the paper, we refer to each model name by its family name and size only.

<table><caption>Table 2: LLMs considered in our study. Size is in billions of parameters.</caption><thead><tr><td>LLM Model</td><td>Model family</td><td>Size</td></tr></thead><tbody><tr><td>Phi-3-mini-128k-instruct</td><td>Microsoft</td><td>3.82 B</td></tr><tr><td>Qwen2.5-Coder-7B-Instruct</td><td>Qwen</td><td>7.62 B</td></tr><tr><td>Llama-3.1-8B-Instruct</td><td>Meta-llama</td><td>8.03 B</td></tr><tr><td>DeepSeek-R1-Distill-Qwen-14B</td><td>Deepseek-ai</td><td>14.8 B</td></tr><tr><td>Qwen2.5-Coder-14B-Instruct</td><td>Qwen</td><td>14.8 B</td></tr><tr><td>Qwen2.5-Coder-32B-Instruct</td><td>Qwen</td><td>32.8 B</td></tr></tbody></table>

## 3.4 Experimental Setup

To answer each of our research questions, we adopted a two-phase experimental process. In the first phase, we focused exclusively on Qwen 14B. This LLM was selected as a starting point based on its top ranking in BigCodeBench, following the application of our selection criteria. We evaluated this model across both co-evolution directions—updating the FM based on source code changes, and updating the code based on FM modifications—covering each of the evolution steps outlined in Table 1. To assess consistency and robustness, each experiment was executed 20 times, resulting in a minimum of 140 prompts per model (with the actual number potentially higher due to validation loop iterations). In the second phase, we extended the experiments to include five additional LLMs. This allowed us to analyze the impact of model choice on the co-evolution process in both directions and, expanded the overall evaluation to at least 840 prompts per direction across all models and scenarios.

We executed all these experiments on a node from the Grid5000 platform,<fnref n="6" /> equipped with an AMD EPYC 7513 (Zen 3) processor, 512 GiB of memory, and 4 Nvidia A100-SXM4-40GB (40 GiB) GPUs.

The system operated on Debian 11 (Bullseye) with the Linux 5.10.0-28-amd64 kernel. We also used vLLM [44], a fast and easy-to-use library for serving LLMs efficiently. Beyond inference, it supports features like structured outputs, which we used to better control and format the models' responses.

All the artifacts of this study, including the code, datasets, and results, are publicly available in the following repository: https://doi.org/10.5281/zenodo.15276836

## 3.5 Key Performance Metrics

In our study, we evaluated two tasks: model generation and code generation. For model generation, we compared at each evolution, the FM generated by the LLM with the expected one. FMs were provided as FeatureIDE<fnref n="7" /> XML files and were compared as such. For code generation, exact code content was not expected from the LLM (as we do not specify detailed feature behaviors), thus, we focused on structural template generation. Therefore, we validated the generated template structure by checking file creation, renaming, and deletion operations. The before-and-after project structures were converted into XML format for comparison.

Consequently, in both directions, results were compared against expected outputs represented as XML files. Previous works have explored various techniques for calculating structural similarity between documents [16, 18, 46, 58]. To compare XML structures, we used the *Similarity-based Flexible Tree Matching* (SFTM) algorithm<fnref n="8" /> proposed by Brisset *et al.*, which enables highly accurate tree matching on real-world web pages [17]. We chose to use SFTM, for two reasons: (i) its open-source nature allowed parser customization, and (ii) in contrast to other tools, it demonstrated strong performance on real-world XML data, e.g., it efficiently processes *Document Object Model* (DOM) trees in microseconds. Using the SFTM tool, we can compute the *precision*, *recall*, and *F1 score*, which are standard metrics commonly used to evaluate the performance of binary classification tasks [17]. In our context, *precision* measures the proportion of correctly predicted XML elements among all elements generated by the LLM, while *recall* reflects the proportion of correctly predicted elements among all those expected. The *F1 score* is considered as the harmonic mean of precision and recall scores and is computed as follows:

$$F1 = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}} \quad (1)$$

<footnote n="5">⁵Although Llama 3.1 is freely available, it is still not considered open source by the Open Source Initiative (OSI)</footnote>

<footnote n="6">⁶https://www.grid5000.fr/</footnote>

<footnote n="7">⁷https://featureide.github.io/</footnote>

<footnote n="8">⁸https://github.com/lssol/sftm-csharp</footnote>
<figure><img src="image_8.png" /><figcaption>Figure 4: F1-score across versions during code-to-FM co-evolution, using Qwen2.5-Coder-14B-Instruct (20 samples per version).</figcaption></figure>

# 4 Empirical Results

To address our research questions, we begin by presenting a detailed strip plot of Qwen 14B's performance during the co-evolution from source code to FM in Figure 4, highlighting the variation of F1-scores across different scenarios. Similarly, Figure 7 reports the corresponding results for the reverse co-evolution direction, from FM to source code. We then analyze the per-scenario F1-score distributions across all models using Cumulative Distribution Functions (CDFs) in Figure 6 and Figure 8, for the code-to-FM and FM-to-code directions, respectively.

## 4.1 RQ1: Source Code to FM Co-Evolution

*To what extent can LLMs support co-evolution from source code to FM?* As shown in Figure 4, Qwen 14B consistently achieved high F1-scores, ranging from 0.93 to 1.0 across all evolution steps. An exception was an outlier in version *v*₅, where the F1-score dropped to 0.72. For clarity, this outlier was excluded from the main plot but is retained in Figure 6. Notably, *v*₄ and *v*₅ exhibited slightly greater variation compared to the other versions, likely due to their higher complexity—*v*₄ introduced 10 features simultaneously, while *v*₅ introduced 24. Although the observed variation remains limited, these results show early signs that increasing evolution complexity may have a more pronounced impact.

These results highlight the strong potential of this LLM to support code-to-FM co-evolution, while also indicating that performance stability may decline in more complex scenarios.

*How does model choice influence the co-evolution from source code to FM?* As shown in Figure 6, larger models (14B and 32B) are characterized by steep, right-shifted CDF curves, indicating both high F1-scores and consistent performance across generations. Among them, Qwen 14B and 32B performed exceptionally well, with F1-scores consistently above 0.90. Qwen 14B exhibited a tight distribution, maintaining F1-scores above 0.90, with a slight variation in *v*₅ due to inconsistent XML tags in the generated FM. Moreover, it only triggered the validation loop once during the experiment, as shown

<figure><img src="image_9.png" /><figcaption>Figure 5: Number of validation attempts per LLM, relative to the achieved F1-score during code-to-FM co-evolution.</figcaption></figure>

in Figure 5. Qwen 32B demonstrated an even tighter distribution, maintaining F1-scores above 0.92 across all evolutions, without triggering the validation loop at all. However, this represents only a marginal improvement over Qwen 14B. DeepSeek 14B performed similarly to both Qwen 14B and 32B in most scenarios, but occasionally produced outliers in simpler cases, such as *v*₁ and *v*₇. For example, one generation resulted in an F1-score of 0 in *v*₁, indicating complete failure due to a minor XML formatting issue that made the output invalid. It is also interesting to note that DeepSeek 14B required the validation loop more frequently than the two Qwen models, reflecting a greater need for prompt adjustments and additional time to achieve the same performance. This behavior, as well as the increase in response time, is probably due to the reasoning-oriented nature of DeepSeek 14B. Overall, these results suggest that the reasoning-based LLM did not offer advantages compared to the code-oriented LLM in the model generation task.

In contrast, smaller models revealed clear limitations about their ability to handle the co-evolution from source code to FM. Their CDF curves tend to be flatter and more left shifted, particularly with Phi 3B, indicating lower stability and reliability, especially as the complexity of the evolution increases. Phi 3B did not even handle simple scenarios, like *v*₁, where 20% of its output completely failed (F1=0) due to various instances where the model either failed to generate a response or produced malformed XML. Performance unsurprisingly further degraded in more complex scenarios, like *v*₆, where more than 80% of outputs failed completely. The rare successful generations thus appeared to be outliers rather than indicative of consistent capability. Llama 8B performed well in simple scenarios, such as *v*₁, *v*₃ and *v*₇, but its stability and performance decreased with increasing evolution complexity. This resulted in several F1-scores of zero, mainly due to malformed XML outputs. Qwen 7B outperformed other small models, delivering more consistent and reliable outputs, though it still struggled with stability and performance in certain cases.

Overall, models in the 3–8B range faced two main limitations. First, they were more prone to generate incorrect or invalid outputs. This led to a more frequent use of the validation loop mechanism,
<figure><img src="image_10.png" /><figcaption>Figure 6: *Cumulative Distribution Function* (CDF) of F1-scores during code-to-FM co-evolution (20 samples per version).</figcaption></figure>

as shown in Figure 5, especially for Llama 8B and Phi 3B. In some
cases, the validation process even failed to resolve the issue, with
the model continuing to propose invalid responses, even after reach-
ing the preset limit of 10 attempts. In contrast, larger models rarely
triggered corrections, and when needed, refinements were typically
successful. Second, smaller models displayed higher variation in
F1-scores across runs, indicating a lack of stability in their gener-
ated outputs. These results show the influence of both model size
and architecture on code-to-FM co-evolution performance, as they
impact the quality and consistency of generated outputs. While
14B and 32B models generally outperformed the 3B to 8B ones,
model architecture also played a role—Qwen 7B, for instance, out-
performed Llama 8B despite being smaller. Although benchmark
studies, such as BigCodeBench,<fnref n="9" /> report that smaller models like
Phi 3B can perform better than larger ones on programming tasks,
our findings suggest that it does not apply to the specific task of
model generation.

**RQ1:** Our experimental results show that LLMs can effec-
tively support co-evolution from source code to FM, achieving
F1-scores ranging from 0.93 to 1.0. However, performance and
stability highly depend on the LLM used and the complexity
of the evolution scenario. Larger models generally produce
more accurate and consistent results than smaller ones. Nev-
ertheless, all models exhibit some degree of instability as the
complexity of the evolution increased, suggesting that perfor-
mance stability may further decline on more complex datasets.
Moreover, models with more than 14B parameters demon-
strate only slight improvements in accuracy and consistency,
indicating that further scaling provides marginal benefits.

## 4.2 RQ2: FM to Source Code Co-evolution

To what extent can LLMs support co-evolution from FM to source code? As illustrated in Figure 7, Qwen 14B achieved F1-scores ranging from 0.41 to 0.99. A lower score (F1=0), observed during $v_7$, was considered as an outlier and omitted from the figure for clarity, but it is still present in Figure 8. Compared to the code-to-FM direction, we observe lower performance and greater variation, with impactful evolutions showing the most instability, though simpler cases are also affected. These findings demonstrate some potential in using LLMs to co-evolve from FM to source code, but also suggest that it is a more challenging task than the reverse direction, highlighting the need for additional context and potentially more advanced techniques.

How does model choice influence the co-evolution from FM to
source code? In contrast to the reverse co-evolution direction, the
curves in Figure 6 show that larger models no longer consistently
produce the steepest or most right-shifted CDFs, indicating a re-
duced dominance in this co-evolution direction. Nonetheless, in
more complex evolution scenarios—like $v_4$, $v_5$, and $v_6$—larger mod-
els still demonstrated more stability and reliability. Among these
models, performance remained comparable in simpler scenarios,
but differences became more noticeable as complexity increased.
For instance, Qwen 32B performed best in $v_4$, while Qwen 14B per-
formed best in $v_5$. Qwen 32B struggled with stability as the number
of simultaneous changes increased: nearly 50% of its generations
had an F1-score around 0.40 in $v_5$. In contrast, Qwen 14B main-
tained consistent performance, with F1-scores generally around
0.94. These results suggest that larger LLMs are not always the best
choice for code generation tasks. Moreover, as in the code-to-FM
co-evolution task, DeepSeek 14B relied on the validation loop more
frequently than both Qwen models, as illustrated in Figure 9. It was

<footnote n="9">⁹https://bigcode-bench.github.io/</footnote>
<figure><img src="image_11.png" /><figcaption>Figure 7: F1-score across versions during FM-to-code co-evolution, using Qwen2.5-Coder-14B-Instruct (20 samples per version).</figcaption></figure>

among the models that relied on it the most, alongside Llama 8B and Phi 3B. Notably, when DeepSeek 14B produced an incorrect output, it often failed to correct itself on the first retry, unlike other models. This again shows that reasoning-focused LLMs did not offer significant advantages.

Among the smallest models, Qwen 7B produced steeper and more right-shifted curves than larger models in simpler evolutions, such as $v_1$ and $v_3$, indicating better performance and stability in these scenarios. However, its performance degraded in more complex scenarios, e.g., $v_4$ and $v_5$, where Llama 8B demonstrated better stability and achieved higher F1-scores. Still, Llama 8B did not match the performance of larger models in these more complex scenarios. As for the code-to-FM direction, Phi 3B consistently struggled across most evolutions, even ending up in timeout for all $v_7$ runs. The only exception was $v_1$, the simplest scenario, where it matched the performance of other models.

These results demonstrate that, while model size and architecture continue to affect performance and stability, their impact is less pronounced in the FM-to-code direction than in the reverse co-evolution task, as the largest models did not consistently outperform smaller ones across all scenarios.

**RQ2:** Our results show the potential of using LLMs to support FM-to-code co-evolution although this direction proved to be more challenging than the reverse one. Overall, performance and stability are generally lower, with F1-scores ranging from 0.41 to 0.99, particularly in complex scenarios. While LLM influence is less pronounced than in the reverse direction, larger models still tend to perform better in complex evolution scenarios. However, relying on increasingly large LLMs does not guarantee better performance and stability across all scenarios.

# 5 Threats to Validity

**Prompt structure.** Our prompt design was iteratively refined based on initial experiments. An ablation study revealed that increasing prompt complexity did not enhance response reliability and, occasionally, even reduced its effectiveness. However, such additions may prove beneficial for more complex datasets, so we cannot assert that our prompt structure is optimal for all scenarios.

**Size and complexity of the dataset.** Our evaluation was conducted on a real-world dataset that, while relevant, remains relatively small and simple compared to more complex industrial FMs, as it lacks explicit variability constraints. For example, although the dataset contains a few optional features, these were excluded from the analysis because optionality is not explicitly represented in the source code. Instead, it is handled implicitly through configuration and composition mechanisms provided by tools like FeatureIDE, which uses the FeatureHouse composer to enable feature composition at build time. As a result, optional features are not directly visible in the code, making it difficult for the LLM to accurately recover them during the code-to-FM co-evolution process.

**Feature model format.** Our study focused on FMs represented in XML format. While the inherent flexibility of LLMs suggests that our approach could be adapted to alternative formats—such as *Universal Variability Language* (UVL) [67] or DIMACS [15], and although preliminary experiments with these formats yielded promising results, we did not conduct a comprehensive assessment across larger datasets. We thus cannot claim that our approach generalizes to alternative FM representations.

**Generated code evaluation.** We did not conduct a content-based analysis of the code generated by the LLMs, such as assessing its correctness or completeness. This decision stems from the nature of the FM, which does not provide detailed behavioral specifications for newly added or modified features. As such, expecting the LLMs to generate fully accurate and executable implementations would not be realistic. Instead, we consider the LLM's role as assisting the developer by generating a draft that can later be refined through human intervention. In preliminary experiments, we explored the impact of providing additional context, such as unit tests. In these scenarios, LLMs generated more complete and meaningful code rather than simple placeholders, showing their potential when guided by behavioral indications. However, pursuing this direction would shift the study toward test-driven code generation, an area already explored in prior work [22, 48] and away from our primary focus on co-evolution. For this reason, we did not evaluate the quality of the generated code as part of our study.

# 6 Discussion

**FM and corresponding code datasets.** Benchmarking co-evolution on more complex datasets presents significant challenges, particularly due to the limited availability of FM datasets in the configurable systems domain [68]. This limited availability is especially pronounced in studies—such as ours—that require both FMs and their corresponding source code across multiple evolution steps. Some research efforts have started to address this gap. For example, Sundermann et al. [66] conducted a literature survey and assembled a benchmarking dataset containing FMs. However, at the time
<figure><img src="image_12.png" /><figcaption>Figure 8: Cumulative Distribution Function (CDF) of F1-scores during FM-to-code co-evolution (20 samples per version).</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Figure 9: Number of validation attempts per LLM, relative to the achieved F1-score during FM-to-code co-evolution.</figcaption></figure>

of writing, this dataset does not yet include co-evolving source code across versions. That said, the authors explicitly encourage contributions from the research community to extend the dataset with corresponding code, marking it a valuable resource for future research in this area.

**Open-source and small LLMs are underexplored.** By conducting our experiments with several open-source and smaller LLMs, we observed that while model type and size influence the stability and effectiveness of co-evolution (with larger models often being more stable and reliable for model generation), there are diminishing benefits beyond a certain LLM size. In fact, for the code generation task, larger models sometimes performed worse. This trend can be observed when comparing the performances of

Qwen 32B and Qwen 14B. As such, we encourage the SPL research community to further explore smaller, open-source LLMs, where model size can be fixed depending on the complexity of the use case and dataset. These models offer an attractive balance between performance, efficiency, cost, and resource consumption—while also improving accessibility and reproducibility. This observation aligns with current trends where the focus is shifting toward (i) optimizing inference rather than training, and (ii) orchestrating model selection based on task requirements instead of relying on a single monolithic model. This resonates with the emerging concept of *multi-agent systems*, where smaller, specialized models or agents are selected dynamically depending on the task to execute [61].

**Areas for improvement.** As noted in previous sections, LLMs begin to exhibit some instability in more complex co-evolution scenarios, particularly when multiple features are introduced simultaneously, resulting in a larger diff. This reflects a common situation in real-world projects, where commits are not always fine-grained, posing challenges for applying this approach in more industrial settings. Moreover, the FM-to-code direction requires more contextual information to improve performance or achieve full automation. Addressing these challenges requires not only mitigating current limitations, such as context length restrictions and hallucinations, but also exploring more advanced prompting strategies to better guide the models in complex scenarios.

A recent taxonomy of prompt engineering strategies proposed by Zhao *et al.* [60] outlines several techniques worth exploring. Among them, RAG and *Chain of verification* (CoVe) [26] aim to reduce hallucination, while *Chain of Code* (CoC) prompting [45] is suggested to improve code generation. LLM-based agents [39] can also be considered to improve automation, enabling models to perform multi-step reasoning, interact with tools, or iteratively refine
their outputs. Finally, for scaling to more complex datasets, alternative FM formats such as UVL and DIMACS may offer advantages over XML, especially due to their more compact representations, better suited to the token limitations of current LLMs.

# 7 Related Work

**Reverse engineering LMs.** Several studies have investigated the recovery of LMs from existing artifacts. However, many of these approaches relied on high-level abstractions or failed to fully capture the variability constraints commonly present in complex LMs. For example, Acher et al. extracted LMs from structured product descriptions by leveraging parameterized directives to preserve valid feature combinations [3]. Lopez-Herrejon et al. employed Evolutionary Algorithms (EAs) to infer LMs from feature sets, demonstrating the potential of search-based approaches [49]. Similarly, Haslinger et al. introduced a lightweight technique for reverse engineering basic LMs from feature sets that describe program variants [38]. Ryssel et al. applied Formal Concept Analysis (FCA) to generate complete LMs from incidence matrices, though their approach does not handle variability constraints [59]. Finally, Acher et al. proposed a synthesis approach that integrates user knowledge to produce more meaningful and maintainable LMs from configurations [2]. Among the more advanced works, Al-Msie'Deen et al. used FCA to extract detailed LMs with its variability constraints [6].

**SPL evolution.** Several studies have addressed the challenges of SPL evolution. In particular, some focused on identifying and characterizing evolution scenarios. For example, Passos et al. studied the co-evolution of variability models and related artifacts in variability-aware systems [55], while Thüm et al. proposed an algorithm to find how two LMs are related within an SPL [69], and classified FM changes into four types: *refactorings*, *specializations*, *generalizations*, and *arbitrary edits*. Other studies focused on consistency management. Guo et al., for instance, proposed an ontology-based formalization combined with predefined strategies to preserve FM consistency and reduce change impact [37]. Several contributions addressed anomaly detection during evolution. Feichtinger et al. combined feature-to-source-code mappings with source code dependency analysis to identify potential inconsistencies during evolution [30]. Lity et al. extended delta-oriented modeling to assess the impact of higher-order changes [47]. Maazoun et al. proposed a change impact analysis approach that identifies elements potentially affected by FM modifications [50], while Nieke et al. introduced a method that encodes the evolution history to detect and explain inconsistencies [53]. Finally, some works focused on co-evolution management strategies. Seidl et al. introduced mapping operators and evolution scenarios to maintain consistency between feature and solution space models during SPL evolution [63], while Kebaili et al. proposed an automated approach for the co-evolution of metamodels and source code [42]. To the best of our knowledge, no existing approach supports automated and bidirectional co-evolution between LMs and source code, as addressed in this work.

**LLMs for variability management.** With the advent of LLMs, research has recently shifted to investigating their potential in Software Product Line Engineering (SPLE) [35]. For instance, Acher et al.

explored how LLMs can assist in re-engineering software variants into an SPL [5]. Stümpfle et al. proposed a retrieval-augmented approach that uses LLMs to generate reusable artifacts and feature mappings from existing models [65], and further examined LLMs role in transforming cloned software variants into an SPL [64]. Furthermore, LLMs have been applied to variability modeling and implementation. Acher et al. explored how LLMs can assist developers in managing variability by generating and adapting source code across different artifacts [4]. Galindo et al. proposed leveraging LLMs to generate realistic and meaningful variants of LMs [33]. In the specific area of co-evolution, Kebaili et al. explored how LLMs can support the co-evolution of metamodels and source code [41]. However, no study investigated the ability of LLMs to co-evolve FM and source code in both directions. Yet, their recent use in variability management, along with their broad application across various software engineering tasks, highlights their potential to tackle more advanced challenges like the co-evolution of FM and code addressed in this paper.

# 8 Conclusion and Future Work

In this paper, we investigated the potential of LLMs to automate the co-evolution between FM and source code. Our experiments yielded promising results, particularly in the co-evolution from source code to FM, where F1-scores ranged from 0.93 to 1.0. In contrast, the reverse direction proved more challenging, with F1-scores ranging from 0.41 to 0.99, as this task inherently requires more contextual understanding.

While the results are encouraging, we observed early stages of instability in LLM outputs, particularly in more complex evolution scenarios. Our evaluation of various open-source LLMs also revealed significant differences in their capabilities to generate both models and code. Larger models generally exhibited superior performance and stability—especially in the code-to-FM task—but performance gains plateaued beyond a certain model size. For the FM-to-code direction, larger models did not always perform best. These observations suggest the need to reconsider the default reliance on large proprietary models. Instead, model selection should be guided by task complexity and dataset characteristics, as smaller open-source models may provide a more efficient and cost-effective balance without significantly compromising performance.

Looking forward, we plan to extend this work by: (i) addressing hallucination and context window limitations to enhance the consistency of LLMs outputs on more complex datasets; (ii) moving toward full automation, evolving from template generation to complete code generation; and (iii) evaluating the improved approach on larger and more complex datasets. To achieve these goals, we aim to further investigate advanced strategies such as those outlined in Section 6, including prompt engineering techniques, retrieval-augmented generation, and the use of LLM-based agents for improved reasoning and orchestration.

# Acknowledgments

We thank the anonymous reviewers for their insightful comments and feedback. This work received funding from the France 2030 program, managed by the French National Research Agency under grant agreement No. ANR-23-PECL-0003.
## References

[1] Seif Abukhalaf, Mohammad Hamdaqa, and Foutse Khomh. 2023. On Codex prompt engineering for OCL generation: an empirical study. In *IEEE/ACM 20th International Conference on Mining Software Repositories (MSR)*. IEEE, 148–157.

[2] Mathieu Acher, Benoit Baudry, Patrick Heymans, Anthony Cleve, and Jean-Luc Hainaut. 2013. Support for reverse engineering and maintaining feature models. In *Proceedings of the 7th International Workshop on Variability Modelling of Software-intensive Systems*. 1–8.

[3] Mathieu Acher, Anthony Cleve, Gilles Perrouin, Patrick Heymans, Charles Van-beneden, Philippe Collet, and Philippe Lahire. 2012. On extracting feature models from product descriptions. In *Proceedings of the 6th International Workshop on Variability Modeling of Software-Intensive Systems*. 45–54.

[4] Mathieu Acher, José Galindo Duarte, and Jean-Marc Jézéquel. 2023. On programming variability with large language model-based assistant. In *Proceedings of the 27th International Systems and Software Product Line Conference-Vol. A*. 8–14.

[5] Mathieu Acher and Jabier Martinez. 2023. Generative AI for reengineering variants into software product lines: an experience report. In *Proceedings of the 27th International Systems and Software Product Line Conference-Vol. B*. 57–66.

[6] Ra'Fat Ahmad Al-Msie'Deen, Marianne Huchard, Abdelhak-Djamel Seriai, Christelle Urtado, and Sylvain Vauttier. 2014. Reverse engineering feature models from software configurations using formal concept analysis. In *CLA: Concept Lattices and their Applications*, Vol. 1252. 95–106.

[7] Sven Apel, Don Batory, Christian Kästner, and Gunter Saake. 2013. Feature-oriented software product lines. (2013).

[8] Wesley K. Assunção, Roberto E. Lopez-Herrejon, Lukas Linsbauer, Silvia R. Vergilio, and Alexander Egyed. 2017. Reengineering legacy applications into software product lines: a systematic mapping. *Empirical Software Engineering* 22, 6 (2017), 2972–3016.

[9] Jonas Ferreira Bastos, Paulo Anselmo da Mota Silveira Neto, Pádraig O'Leary, Eduardo Santana de Almeida, and Silvio Romero de Lemos Meira. 2017. Software product lines adoption in small organizations. *Journal of Systems and Software* 131 (2017), 112–128.

[10] Don Batory. 2005. Feature models, grammars, and propositional formulas. In *International Conference on Software Product Lines*. Springer, 7–20.

[11] Martin Becker, Rick Rabiser, and Goetz Botterweck. 2024. Not Quite There Yet: Remaining Challenges in Systems and Software Product Line Engineering as Perceived by Industry Practitioners. In *Proceedings of the 28th ACM International Systems and Software Product Line Conference (SPLC '24)*. 179–190.

[12] David Benavides, Alexander Felfernig, José A Galindo, and Florian Reinfrank. 2013. Automated analysis in feature modelling and product configuration. In *Safe and Secure Software Reuse: 13th International Conference on Software Reuse, ICSR 2013, Pisa, June 18-20. Proceedings* 13. Springer, 160–175.

[13] David Benavides, Sergio Segura, and Antonio Ruiz-Cortés. 2010. Automated analysis of feature models 20 years later: A literature review. *Information Systems* 35, 6 (2010), 615–636.

[14] Thorsten Berger, Ralf Rublack, Divya Nair, Joanne M Atlee, Martin Becker, Krzysztof Czarnecki, and Andrzej Wąsowski. 2013. A survey of variability modeling in industrial practice. In *Proceedings of the 7th International Workshop on Variability Modelling of Software-intensive Systems*. 1–8.

[15] Thorsten Berger, Steven She, Rafael Lotufo, Andrzej Wasowski, and Krzysztof Czarnecki. 2013. A study of variability models and languages in the systems software domain. *IEEE Trans. on Software Engineering* 39, 12 (2013), 1611–1640.

[16] Philip Bille. 2005. A survey on tree edit distance and related problems. *Theoretical computer science* 337, 1-3 (2005), 217–239.

[17] Sacha Brisset, Romain Rouvoy, Lionel Seinturier, and Renaud Pawlak. 2023. Sftm: Fast matching of web pages using similarity-based flexible tree matching. *Information Systems* 112 (2023), 102126.

[18] David Buttler. 2004. A short survey of document structure similarity algorithms. Technical Report. Lawrence Livermore National Lab.(LLNL), Livermore, CA (United States).

[19] Javier Cámara, Javier Troya, Lola Burgueño, and Antonio Vallecillo. 2023. On the assessment of generative AI in modeling tasks: an experience report with ChatGPT and UML. *Software and Systems Modeling* 22, 3 (2023), 781–793.

[20] Kua Chen, Yujing Yang, Boqi Chen, José Antonio Hernández López, Gunter Mussbacher, and Dániel Varró. 2023. Automated domain modeling with large language models: A comparative study. In *ACM/IEEE 26th International Conference on Model Driven Engineering Languages and Systems (MODELS)*. 162–172.

[21] Lianping Chen, Muhammad Ali Babar, and Nour Ali. 2009. Variability management in software product lines: a systematic review. (2009).

[22] Jianlei Chi, Xiaotian Wang, Yuhan Huang, Lechen Yu, Di Cui, Jianguo Sun, and Jun Sun. 2024. REACCEPT: Automated Co-evolution of Production and Test Code Based on Dynamic Validation and Large Language Models. *Preprint arXiv:2411.11033* (2024).

[23] Paul Clements and Linda Northrop. 2002. *Software product lines*. Vol. 1. Addison-Wesley Boston.

[24] Krzysztof Czarnecki, Paul Grünbacher, Rick Rabiser, Klaus Schmid, and Andrzej Wąsowski. 2012. Cool features and tough decisions: a comparison of variability modeling approaches. In *Proceedings of the 6th International workshop on variability modeling of software-intensive systems*. 173–182.

[25] Krzysztof Czarnecki and Andrzej Wasowski. 2007. Feature diagrams and logics: There and back again. In *11th International Software Product Line Conference (SPLC 2007)*. IEEE, 23–34.

[26] Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston. 2023. Chain-of-verification reduces hallucination in large language models. *Preprint arXiv:2309.11495* (2023).

[27] Yael Dubinsky, Julia Rubin, Thorsten Berger, Slawomir Duszynski, Martin Becker, and Krzysztof Czarnecki. 2013. An exploratory study of cloning in industrial software product lines. In *2013 17th European Conference on Software Maintenance and Reengineering*. IEEE, 25–34.

[28] Jorge Echeverría, Francisca Pérez, José Ignacio Panach, and Carlos Cetina. 2021. An empirical study of performance using Clone & Own and Software Product Lines in an industrial context. *Information and Software Technology* 130 (2021), 106444.

[29] Alexander Egyed. 2010. Automatically detecting and tracking inconsistencies in software design models. *IEEE Trans. on Soft. Engineering* 37, 2 (2010), 188–204.

[30] Kevin Feichtinger, Daniel Hinterreiter, Lukas Linsbauer, Herbert Prähofer, and Paul Grünbacher. 2021. Guiding feature model evolution by lifting code-level dependencies. *Journal of Computer Languages* 63 (2021), 101034.

[31] Stefan Fischer, Lukas Linsbauer, Roberto Erick Lopez-Herrejon, and Alexander Egyed. 2014. Enhancing clone-and-own with systematic reuse for developing software variants. In *2014 IEEE International conference on software maintenance and evolution*. IEEE, 391–400.

[32] José A Galindo, David Benavides, Pablo Trinidad, Antonio-Manuel Gutiérrez-Fernández, and Antonio Ruiz-Cortés. 2019. Automated analysis of feature models: Quo vadis? *Computing* 101 (2019), 387–433.

[33] José A Galindo, Antonio J Dominguez, Jules White, and David Benavides. 2023. Large language models to generate meaningful feature model instances. In *Proceedings of the 27th ACM International Systems and Software Product Line Conference-Volume A*. 15–26.

[34] Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, et al. 2023. Retrieval-augmented generation for large language models: A survey. *Preprint arXiv:2312.10997* 2 (2023).

[35] Sandra Greiner, Klaus Schmid, Thorsten Berger, Sebastian Krieter, and Kristof Meixner. 2024. Generative ai and software variability-a research vision. In *Proceedings of the 18th International Working Conference on Variability Modelling of Software-Intensive Systems*. 71–76.

[36] Édouard Guégain, Clément Quinton, and Romain Rouvoy. 2021. On reducing the energy consumption of software product lines. In *Proceedings of the 25th ACM International Systems and Software Product Line Conference-Volume A*. 89–99.

[37] Jianmei Guo, Yinglin Wang, Pablo Trinidad, and David Benavides. 2012. Consistency maintenance for evolving feature models. *Expert Systems with Applications* 39, 5 (2012), 4987–4998.

[38] Evelyn Nicole Haslinger, Roberto E Lopez-Herrejon, and Alexander Egyed. 2011. Reverse engineering feature models from programs' feature sets. In *2011 18th working conference on reverse engineering*. IEEE, 308–312.

[39] Haolin Jin, Linghan Huang, Haipeng Cai, Jun Yan, Bo Li, and Huaming Chen. 2024. From llms to llm-based agents for software engineering: A survey of current, challenges and future. *Preprint arXiv:2408.02479* (2024).

[40] Georgios Kanakis, Djamel Eddine Khelladi, Stefan Fischer, Michael Tröls, and Alexander Egyed. 2019. An Empirical Study on the Impact of Inconsistency Feedback during Model and Code Co-changing. *The Journal of Object Technology* 18, 2 (2019), 10–1.

[41] Zohra Kaouter Kebaili, Djamel Eddine Khelladi, Mathieu Acher, and Olivier Barais. 2024. An Empirical Study on Leveraging LLMs for Metamodels and Code Co-evolution. In *European Conference on Modelling Foundations and Applications (ECMFA 2024)*, Vol. 23. Journal of Object Technology, 1–14.

[42] Zohra Kaouter Kebaili, Djamel Eddine Khelladi, Mathieu Acher, and Olivier Barais. 2025. Automated co-evolution of metamodels and code. *IEEE Transactions on Software Engineering* (2025).

[43] Elias Kuiter, Jacob Krüger, Sebastian Krieter, Thomas Leich, and Gunter Saake. 2018. Getting rid of clone-and-own: moving to a software product line for temperature monitoring. In *Proceedings of the 22nd International Systems and Software Product Line Conference-Volume 1*. 179–189.

[44] Woosuk Kwon, Zhuoohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In *Proceedings of the 29th Symposium on Operating Systems Principles*. 611–626.

[45] Chengshu Li, Jacky Liang, Andy Zeng, Xinyun Chen, Karol Hausman, Dorsa Sadigh, Sergey Levine, Li Fei-Fei, Fei Xia, and Brian Ihchter. 2023. Chain of code: Reasoning with a language model-augmented code emulator. *Preprint arXiv:2312.04474* (2023).
[46] Yuanzhang Lin, Guoyao Wen, and Xiang Gao. 2023. Automated fixing of web ui tests via iterative element matching. In *2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)*. IEEE, 1188-1199.

[47] Sascha Lity, Matthias Kowal, and Ina Schaefer. 2016. Higher-order delta modeling for software product line evolution. In *Proceedings of the 7th International Workshop on Feature-Oriented Software Development*. 39-48.

[48] Jun Liu, Jiwei Yan, Yuanyuan Xie, Jun Yan, and Jian Zhang. 2024. Fix the Tests: Augmenting LLMs to Repair Test Cases with Static Collector and Neural Reranker. In *2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE)*. IEEE, 367-378.

[49] Roberto Erick Lopez-Herrejon, José A Galindo, David Benavides, Sergio Segura, and Alexander Egyed. 2012. Reverse engineering feature models with evolutionary algorithms: An exploratory study. In *Search Based Software Engineering: 4th International Symposium, SSBSE September 28-30, 2012*. Springer, 168-182.

[50] Jihen Maàzoun, Nadia Bouassida, and Hanène Ben-Abdallah. 2016. Change impact analysis for software product lines. *Journal of King Saud University-Computer and Information Sciences* 28, 4 (2016), 364-380.

[51] Maira Marques, Jocelyn Simmonds, Pedro O Rossel, and María Cecilia Bastarrica. 2019. Software product line evolution: A systematic literature review. *Information and Software Technology* 105 (2019), 190-208.

[52] Jabier Martinez, Xhevahire Tërnava, and Tewfik Ziadi. 2018. Software product line extraction from variability-rich systems: the robocode case study. In *Proceedings of the 22nd International Systems and Software Product Line Conference*. 132-142.

[53] Michael Nieke, Jacopo Mauro, Christoph Seidl, Thomas Thüm, Ingrid Chieh Yu, and Felix Franzke. 2018. Anomaly analyses for feature-model evolution. *ACM SIGPLAN Notices* 53, 9 (2018), 188-201.

[54] Leandro Oliveria de Souza, Eduardo Santana de Almeida, Paulo Anselmo da Mota Silveira Neto, Earl T. Barr, and Justyna Petke. 2025. Software Product Line Engineering via Software Transplantation. *ACM Trans. Softw. Eng. Methodol.* 34, 2, Article 31 (Jan. 2025), 27 pages.

[55] Leonardo Passos, Jianmei Guo, Leopoldo Teixeira, Kryszsztof Czarnecki, Andrzej Wąsowski, and Paulo Borba. 2013. Coevolution of variability models and related artifacts: A case study from the linux kernel. In *Proceedings of the 17th International Software Product Line Conference*. 91-100.

[56] Klaus Pohl, Günter Böckle, and Frank Van Der Linden. 2005. *Software product line engineering: foundations, principles, and techniques*. Vol. 1. Springer.

[57] Clément Quinton, Andreas Pleuss, Daniel Le Berre, Laurence Duchien, and Goetz Botterweck. 2014. Consistency checking for the evolution of cardinality-based feature models. In *Proceedings of the 18th International Software Product Line Conference - Volume 1* (Florence, Italy) (SPLC '14). 122-131.

[58] Davi Castro Reis, Paulo Braz Golgher, Altigran Soares Silva, and Alberto F Laender. 2004. Automatic web news extraction using tree edit distance. In *Proceedings of the 13th international conference on World Wide Web*. 502-511.

[59] Uwe Ryssel, Joern Ploennigs, and Klaus Kabitzsch. 2011. Extraction of feature models from formal contexts. In *Proceedings of the 15th International Software Product Line Conference, Volume 2*. 1-8.

[60] Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. 2024. A systematic survey of prompt engineering in large

language models: Techniques and applications. Preprint arXiv:2402.07927 (2024).

[61] Brell Peclard Sanwouou, Clément Quinton, and Paul Temple. 2025. Breaking the Loop: AWARE is the new MAPE-K. In *FSE 2025 - ACM International Conference on the Foundations of Software Engineering*. ACM, Trondheim, Norway, 1-5.

[62] Ina Schaefer, Rick Rabiser, Dave Clarke, Lorenzo Bettini, David Benavides, Goetz Botterweck, Animesh Pathak, Salvador Trujillo, and Karina Villela. 2012. Software diversity: state of the art and perspectives. *International Journal on Software Tools for Technology Transfer* 14 (2012), 477-495.

[63] Christoph Seidl, Florian Heidenreich, and Uwe Aßmann. 2012. Co-evolution of models and feature mapping in software product lines. In *Proceedings of the 16th International Software Product Line Conference-Volume 1*. 76-85.

[64] Johannes Stümpfle, Devansh Atray, Nasser Jazdi, and Michael Weyrich. 2025. Large Language Model assisted Transformation of Software Variants into a Software Product Line. In *2025 IEEE/ACM 22nd International Conference on Software and Systems Reuse (ICSR)*. IEEE, 12-20.

[65] Johannes Stümpfle, Sebastian Baum, Daniel Dittler, Nasser Jazdi, and Michael Weyrich. 2024. Automating Software Product Line Adoption Based on Feature Models Using Large Language Models. In *2024 IEEE 29th International Conference on Emerging Technologies and Factory Automation (ETFA)*. IEEE, 1-4.

[66] Chico Sundermann, Vincenzo Francesco Brancaccio, Elias Kuiter, Sebastian Krieter, Tobias Heß, and Thomas Thüm. 2024. Collecting Feature Models from the Literature: A Comprehensive Dataset for Benchmarking. In *Proceedings of the 28th ACM International Systems and Software Product Line Conference*. 54-65.

[67] Chico Sundermann, Kevin Feichtinger, Dominik Engelhardt, Rick Rabiser, and Thomas Thüm. 2021. Yet another textual variability language? a community effort towards a unified language. In *Proceedings of the 25th ACM International Systems and Software Product Line Conference-Volume A*. 136-147.

[68] Chico Sundermann, Tobias Heß, Rahel Sundermann, Elias Kuiter, Sebastian Krieter, and Thomas Thüm. 2024. Generating Feature Models with UVL's Full Expressiveness. In *Proceedings of the 28th ACM International Systems and Software Product Line Conference*. 61-65.

[69] Thomas Thum, Don Batory, and Christian Kastner. 2009. Reasoning about edits to feature models. In *IEEE 31st International Conference on Software Engineering*. 254-264.

[70] Xindi Wang, Mahsa Salmani, Parsa Omidi, et al. 2024. Beyond the limits: A survey of techniques to extend the context length in large language models. Preprint arXiv:2402.02244 (2024).

[71] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. *Advances in neural information processing systems* 35 (2022), 24824-24837.

[72] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. [n. d.]. A survey of large language models. ([n. d.]).

[73] Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, et al. 2024. Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions. Preprint arXiv:2406.15877 (2024).