Venue: NAACL 2025

Can Large Language Models Invent Algorithms to Improve Themselves?: Algorithm Discovery for Recursive Self-Improvement through Reinforcement Learning

Yoichi Ishibashi NEC yoichi-ishibashi@nec.com &Taro Yano NEC taro_yano@nec.com &Masafumi Oyamada NEC oyamada@nec.com

###### Abstract

Large Language Models (LLMs) have shown remarkable performance improvements and are rapidly gaining adoption in industry. However, the methods for improving LLMs are still designed by humans, which restricts the invention of new model-improving algorithms to human expertise and imagination. To address this, we propose the _Self-Developing_ framework, which enables LLMs to autonomously generate and learn model-improvement algorithms. In this framework, the seed model generates, applies, and learns model-improving algorithms, continuously improving both the seed model and the algorithms themselves. Among model-improving strategies, we focus on model merging algorithms. In mathematical reasoning tasks, Self-Developing discovers novel merging strategies and outperforms human-designed methods. On GSM8k, the discovered algorithms improve the seed model by 6% and surpass human-designed methods by 4.3%. Moreover, they exhibit strong transferability, achieving a 7.4% performance gain on out-of-domain models. These results suggest that LLMs can autonomously develop effective model-improvement techniques beyond human intuition.

## 1 Introduction

The advancement of Large Language Models (LLMs) is having a significant impact on society Vaswani et al. (2017); Brown et al. (2020); Dubey et al. (2024); Guo et al. (2025). LLMs have been continuously improved by human experts' knowledge and experience, realizing advanced capabilities such as mathematical reasoning or code generation OpenAI (2023). Building on these advanced capabilities, researchers are increasingly focusing on developing self-improving methods for LLMs to autonomously improve their performance without human intervention, with the goal of automating the LLM development process itself. Research on self-improvement of LLMs includes approaches such as fine-tuning using self-generated data Yuan et al. (2024); Gulcehre et al. (2023); Zhang et al. (2024); Wang et al. (2023); Xu et al. (2024), self-play Tu et al. (2024); Cheng et al. (2024), and planning using feedback from environment Shinn et al. (2023); Madaan et al. (2023). However, a fundamental limitation is that the exploration of the strategies to improve LLMs (model-improving algorithms) remains constrained by human knowledge and imagination.

Regarding this, as an extreme form of self-improvement, one can ask a question: _Could we empower LLMs to autonomously discover and develop algorithms to improve themselves?_ This approach could potentially uncover novel, high-performance algorithms beyond human knowledge and imagination, as exemplified by AlphaGo's _'Move 37'_Silver et al. (2016), thus expanding the frontiers of AI capabilities beyond the limitations of humanized algorithms.

In this paper, we propose _Self-Developing_, an LLM-based framework that invents model-improving algorithms without the use of human expertise or feedback from external stronger models. While our framework can be applied to various types of model improvement algorithms, in this study we specifically focus on model merging algorithms Ilharco et al. (2023) that create a single improved model from multiple input models, as a concrete instance of model-improving algorithms. Our approach iteratively refines two components: the seed model, which is improved using LLM-generated algorithms, and the algorithm factory, which generates these algorithms.

In experiments on mathematical reasoning, Self-Developing invents new model-improving algorithms, which can be considered novel model merging strategies. On the GSM8k task, LLM-discovered algorithms surpass human-designed methods such as Task Arithmetic Ilinarco et al. ( 

2023), enhancing the seed model by 6% and outperforming human-designed algorithms by 4.3%. Furthermore, the discovered algorithms demonstrate strong transferability to out-of-domain models not used in algorithm generation, surpassing the performance of Task Arithmetic optimized for these models by 7.4%. Notably, our experiments reveal that the iterative refinement of both the seed model and the algorithm factory plays a crucial role in generating increasingly effective algorithms.

## 2 Self-Developing: Learning to Generate Model-Improvement Algorithms

The main objective of this research is to enable LLMs to autonomously generate and apply model-improvement algorithms. Specifically, we address the following challenge: Given a _seed model_\(M_{0}\), our aim fit to generate models that exceeds \(M_{0}\) without guidance from superior teacher models (e.g., GPT-4 (OpenAI, 2023)) or human intervention. This challenging task requires the seed model to devise and implement self-improvement strategies using only its inherent capabilities and knowledge. Success in this endeavor is defined by the generated model demonstrating higher performance on specified tasks compared to the seed model. To achieve this goal, we propose a framework that iterates through an improvement cycle, as illustrated in Figure 1. The cycle consists of the following steps:

1. **Algorithm Factory Initialization:** We initialize an algorithm factory \(\pi^{g}_{1}\) by cloning the seed model \(M_{0}\) (i.e., \(\pi^{g}_{1}=M_{0}\)). Both the seed model \(M_{0}\) and the algorithm factory \(\pi^{g}_{1}\) are iteratively enhanced.
2. **Algorithm Generation (SS2.1):** In the \(t\)-th iteration, algorithm factory\(\pi^{g}_{t}\) generates model improving algorithms (\(a^{(1)}_{t},a^{(2)}_{t},\ldots,a^{(N)}_{t}\)).
3. **Algorithm Evaluation (SS 2.2):** We apply the generated algorithms to \(M_{0}\) to create new models (\(M^{(1)}_{t},M^{(2)}_{t},\ldots,M^{(N)}_{t}\)). By evaluating these on target tasks, we can measure the effectiveness of the algorithms.
4. **Algorithm Factory Refinement (SS 2.3):** Based on the evaluation results of the models created by applying the generated algorithms, we refine algorithm factory \(\pi^{g}_{t}\). We create preference data from effective and ineffective algorithms and train using DPO. This enables the algorithm factory to acquire the ability to generate superior algorithms.
5. **Iterative Improvement (SS2.4):** By repeating this process, we simultaneously improve the quality of the algorithms and the performance of the generated models.

### Algorithm Generation

The algorithm factory is a language model that generates _model-improving algorithms_ in the form of _programming code_, which enhance the performance of the _seed model_. Formally, the algorithm factory \(\pi^{g}_{t}\) used at iteration \(t\geq 1\) takes a prompt \(x\) that encourages algorithm generation as input and outputs an algorithm \(a\): \(a_{t}\sim\pi^{g}_{t}(a~{}|~{}x)\).

A model-improving algorithm \(a\) can be arbitrary (Python) code that receives a model \(M\) and returns a model \(M^{\prime}\), which is expected to surpass the performance of the original model \(M\). For example, a model-improving algorithm might be code that receives a model and adds some random vectors to its weights. Alternatively, a model-improving algorithm might receive multiple models rather than a single seed model and compute the average of those models to generate a robust model. Previously, a lot of work has human-designed such model-improving algorithms, such as Task Arithmetic (Ilharco et al., 2023), TIES merging (Yadav et al., 2023), and Model Stock (Jang et al., 2024). In this paper, the proposed algorithm factory aims to generate such model-improving algorithms.

In our method, we use seed model \(M_{0}\) as the base model for merging the task vectors (Ilharco et al., 2023) of merge candidate models \(\{C_{j}\}_{j=1}^{K}\), which are fine-tuned on different datasets. The task vector \(\tau_{C_{j}}\) is defined as the weight difference between the seed model \(M_{0}\) and the merge candidate model \(C_{j}\): \(\tau_{C_{j}}=C_{j}-M_{0}\). Figure 2 illustrates an example of a model-improving algorithm. This simple algorithm implements a merge strategy using a weighted sum of task vectors in Python. Formally, given the set of task vectors \(\{\tau_{C_{j}}\}_{j=1}^{K}\), the model-improving algorithm \(a_{t}\) outputs a merged task vector \(\tau_{t}\):

\[\tau_{t}=a_{t}(\tau_{C_{1}},\ldots,\tau_{C_{K}}). \tag{1}\]

We obtain a merged model by adding \(\tau_{t}\) to the seed model:

\[M_{t}=M_{0}+\tau_{t}.\] 



### Algorithm Evaluation

The objective of the algorithm factory \(\pi_{t}^{g}\) is to generate algorithms that enhance the seed model's performance on target tasks. However, in the initial iteration, \(\pi_{1}^{g}\) is untrained and unable to generate effective algorithms. Therefore, in subsequent iterations, we train the algorithm factory to generate more effective algorithms. We evaluate the merged models obtained from the generated algorithms on the target tasks, and based on these evaluations, we create preference data to train the algorithm factory.

We assess the effectiveness of the algorithm by evaluating the model generated with the algorithm on the target tasks. First, from the set of algorithms generated as Python functions, we remove those that are non-executable or result in timeouts, obtaining a set of executable algorithms \(\{a_{t}^{(i)}\}_{i=1}^{N}\). Second, these algorithms are applied to the task vectors \(\{\tau_{C_{j}}\}_{j=1}^{K}\) and merged with \(M_{0}\) to generate new models \(\{M_{t}^{(i)}\}_{i=1}^{N}\). Then, we evaluate the new models on the development set of the downstream tasks, and a task score \(s_{t}^{(i)}\in\mathbb{R}\) is calculated for each model. These scores indicate the effectivenesses of the algorithms. The set of evaluation results \(\{s_{t}^{(i)}\}_{i=1}^{N}\) obtained for all executable algorithms is used to create the preference data as described in SS2.3.

### Algorithm Factory Refinement

To generate increasingly superior algorithms, we train the algorithm factory using Direct Preference Optimization (DPO; Rafailov et al., 2023). The key to this learning process lies in the utilization of preference data based on the performance of the generated algorithms. We evaluate the set of generated algorithms \(\{a_{t}^{(i)}\}_{i=1}^{N}\), selecting high-performance algorithms \(a_{t,w}^{(i)}\) (_chosen_) and low-performance algorithms \(a_{t,l}^{(j)}\) (_rejected_) based on their evaluation scores \(\{s_{t}^{(i)}\}_{i=1}^{N}\). The preference information \(a_{t,w}^{(i)}\succ a_{t,l}^{(j)}\) is then incorporated into the model's learning process. This allows \(\pi_{t}^{g}\) to learn the characteristics of effective algorithms, thereby enhancing its ability to generate superior algorithms in subsequent iterations. Specifically, we select the top-ranked and bottom-ranked algorithms based on a threshold and construct the training data as follows:

\[\mathcal{D}=\{(x,a_{t,w}^{(i)},a_{t,l}^{(j)})~{}|~{}s_{t,w}^{(i)}\geq s_{p_{w} }~{}\text{and}~{}s_{t,l}^{(j)}\leq s_{p_{l}}\}, \tag{3}\]

where \(s_{p_{w}}\) and \(s_{p_{l}}\) represent the score threshold for the top \(p_{w}\%\) and bottom \(p_{l}\%\), respectively. Then, we train the algorithm factory \(\pi_{t}^{g}\) using \(\mathcal{D}\) as the preference dataset.

### Iterative Improvement

Our Self-Developing framework focuses on mutually reinforcing the algorithm factory and see

Figure 1: The overview of **Self-Developing**. This framework involves the simultaneous improvement of the seed model and its self-improvement algorithms by repeating the following steps: First, the algorithm factory \(\pi_{t}^{q}\) is initialized by seed model \(M_{0}\) (Step 1). In \(t\)-th iteration, the algorithm factory \(\pi_{t}^{g}\) takes a prompt \(x\) and generates Python code for model-improvement algorithms (Step 2). Then we apply generated algorithms to the seed model \(M_{0}\) to create improved models. The improved models are evaluated on the target task to measure the algorithm’s effectiveness using task scores \(s_{t}^{(i)}\) (Step 3). Based on the scores, preference data consisting of high-performance and low-performance algorithm pairs are created, and the next generation of the algorithm factory \(\pi_{t+1}^{g}\) is trained using DPO (Step 4). In the next iteration, \(\pi_{t+1}^{g}\) is used as an algorithm factory (Step 5).



model through iterative learning and evaluation. In the \((t+1)\)-th iteration, we use \(\pi^{g}_{t+1}\) as the algorithm factory, which has been trained using DPO from \(\pi^{g}_{l}\). Note that the generated algorithms are always applied to the seed model \(M_{0}\), as the algorithm factory is trained specifically to improve \(M_{0}\). By repeatedly performing this improvement cycle, the algorithm factory gradually generates more efficient algorithms, creating a cycle where the seed model \(M_{0}\) is simultaneously enhanced along with the evolution of the algorithms.

## 3 Main Results

In this section, we demonstrate that Self-Developing can generate algorithms that improve the model itself, and furthermore, these automatically discovered algorithms overperforms the conventional human-designed ones.

### Setup

TasksWe evaluate our approach using the mathematics-related tasks GSM8k Cobbe et al. (2021) and MATH Hendrycks et al. (2021), which have been employed in previous studies Yu et al. (2024); Yuan et al. (2024); Xu et al. (2024b). For GSM8k, we allocate 100 examples from the test set as a development set and use the remaining 1220 examples as the test set. For MATH, we select 100 examples from each of its 6 subsets (totaling 600 examples) for the development set and use the remaining 4400 examples as the test set. To prevent any indirect leakage of test set information into the training data \(\mathcal{D}\) for \(\pi^{g}_{t}\), we evaluate \(\{M^{(i)}_{t}\}_{i=1}^{N}\) exclusively on the development set. After completing all iterations, we conduct a single evaluation on the test set, focusing on the top 15 models across all iterations that demonstrated the highest performance on the development set. We perform evaluations using lm-evaluation-harness1 Gao et al. (2024), employing default prompts and few-shot examples. For both GSM8k and MATH, we use 5-shot examples and evaluate using Pass@1 Chen et al. (2021) with exact match scoring. During the evaluation process, we use greedy decoding for generating responses.

Footnote 1: [https://github.com/EleutherAI/lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness)

ModelsFor the seed model \(M_{0}\), we employ openchat-3.5-1210, a fine-tuned variant of

Figure 2: An example of a model-improving algorithm. This is a model merging function that performs a simple weighted sum of task vectors. The input is a dictionary (with model names as keys and their respective task vectors as values). The algorithm factory produces a Python function that returns the merged task vector. We input up to “#* New * strategies for merging the model weights :n” to ensure that the algorithm factory begins its description from the merging strategy.

 

Mistral-7B-v0.12(Jiang et al., 2023), which has superior code generation capabilities. We also select three Mistral-7B-based fine-tuned models as merging candidates: (1) Abel-7B-0023, which excels in mathematical tasks Chern et al. (2023); (2) OpenHermes-2.5-Mistral-7B4, trained extensively on code instruction data Teknium (2023); and (3) SciPhi-Mistral-7B-32k5, which specializes in scientific domains. These models, fine-tuned for mathematics and science, are expected to enhance the seed model's capabilities 6. We use mergekit7(Goddard et al., 2024) for model merging, applying the algorithm to task vectors in each MLP layer of Transformer Vaswani et al. (2017).

Footnote 2: [https://huggingface.co/mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1)

Footnote 3: [https://huggingface.co/GAIR/Abel-78-002](https://huggingface.co/GAIR/Abel-78-002)

Footnote 4: [https://huggingface.co/teknum/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknum/OpenHermes-2.5-Mistral-7B)

Footnote 5: [https://huggingface.co/SciPhi/SciPhi-Mistral-7B-32k](https://huggingface.co/SciPhi/SciPhi-Mistral-7B-32k)

Self-DevelopingOur process involves 3 iterations, each generating 3000 algorithms8. To effectively balance the exploration-exploitation trade-off in iterative DPO, we decrease the temperature in accordance with the progress of the iteration (see Appendix B). We set the initial temperature \(T_{1}\) to 1.2 with a decay rate \(\beta\) of 0.2, resulting in \(T_{3}=0.85\) for the final iteration. The prompt \(x\), incorporating a one-shot Python implementation example, remains consistent across iterations (see Appendix E). This prompt remains fixed and is used consistently across all iterations. For DPO, we create preference data \(\mathcal{D}\) by selecting the top 3% (\(p_{w}\)) of high-performing algorithms and the bottom 10% (\(p_{l}\)) of low-performing algorithms. We reserve 10% of the training data for development and fine-tune all \(\pi^{g}_{t}\) linear layers using LoRA Hu et al. (2022) with rank \(r=256\). We use a learning rate of \(1e-6\), \(\beta\) of \(0.01\), and cap the training at 5000 steps. All experiments run on NVIDIA A100 GPUs. For iterations where \(t\geq 2\), we augment \(\mathcal{D}\) with the top 3 performing algorithms from each preceding iteration \(\{a^{(i)}_{1}\}_{i=1}^{N_{1}},\cdots,\{a^{(i)}_{t-1}\}_{i=1}^{N_{t-1}}\) (see Appendix D).

Footnote 8: After filtering, the number of executable Python functions typically ranged from 100 to 300 in our experiments.

BaselinesWe compare our Self-Developing with well-established human-designed model-improving algorithms, selecting representative methods that have demonstrated effectiveness in recent literature. Specifically, we include Task Arithmetic Ilharco et al. (2023), TIES Merging Yadav et al. (2023), and Model Stock Jang et al. (2024) as baselines. For Task Arithmetic and TIES Merging, we exhaustively evaluate all combinations of mixing ratios of 20%, 40%, and 60% for candidate models for merging on the development set. For each task, we select the combination that performs best on the development set and then evaluate this optimal combination on the test set.

### Results

Table 1 presents the performance comparison between human-designed algorithms and algorithms discovered by Self-Developing on the GSM8k and MATH tasks. For our approach, we display the top three performances obtained across all iterations of our algorithm discovery process.

**Q1: Can LLMs evolve using self-discovered algorithms?** The results in Table 1 demonstrate that LLMs can improve their own performance using self-discovered model-improvement algorithms. The models applying the top three algorithms discovered by the LLM consistently outperform both the seed model (openchat-3.5-1210) and the three models for merging. Notably, on the

\begin{table}
\begin{tabular}{c c c} \hline \hline
**Models** & **GSMSk (\%)****MATH (\%)** \\ \hline \hline \multicolumn{4}{c}{Base Model (Seed Model)} \\ \hline openchtar-3.5-1210 (\(M_{O}\)) & 70.1 & 0.5 \\ \hline \multicolumn{4}{c}{Models for Merging} \\ \hline Abel-7B-002 (\(C_{1}\)) & 64.8 & 3.7 \\ Opentiermes-2.5-Mistral-7B (\(C_{2}\)) & 60.1 & 1.7 \\ SciPhi-Mistral-7B-32k (\(C_{3}\)) & 56.5 & 1.0 \\ \hline \multicolumn{4}{c}{Human-Designed Algorithms (Best Performances)} \\ \hline Task Arithmetic Ilharco et al. (2023) & 71.9 & **8.5** \\ TIES Merge Yu et al. (2024) & 71.8 & 8.4 \\ Model Stock Jang et al. (2024) & 39.5 & 6.1 \\ \hline \multicolumn{4}{c}{LLM-Designed Algorithms (Top 3 Performances)} \\ \hline
1st (GSMRs: Figure 10, MATH: Figure 20) & **76.1** & **8.5** \\
2nd (GSMRs: Figure 13, MATH: Figure 21) & **76.1** & 8.4 \\
3rd (GSMRs: Figure 14, MATH: Figure 22) & **76.0** & 8.4 \\ \hline \end{tabular}
\end{table}
Table 1: Performance evaluation results of each method on the GSM8k and MATH tasks. The algorithms discovered by Self-Developing outperforms the seed model and existing model-improving algorithms. These results demonstrate that LLMs can invent effective model-improving algorithms that surpass human-designed techniques.

 

GSM8k task, we achieve the highest accuracy of 76.1%, representing a significant performance gain of about 6% over the seed model's 70.1%. For the MATH task, our best model reaches 8.5% accuracy, showing a substantial improvement from the seed model's 0.5%. These results are particularly remarkable considering that powerful external models like GPT-4 were not used in the algorithm generation process.

Q2: Do discovered algorithms surpass human-designed ones?A significant finding is that our proposed method autonomously discovered algorithms that outperform human-designed techniques such as Task Arithmetic and TIES merging. As shown in Table 1, models created using the LLM-discovered algorithms consistently demonstrate higher performance on the GSM8k task compared to Task Arithmetic (76.1% vs 71.9%) and TIES merging (76.1% vs 71.8%). On the MATH task, our best model is comparable to the top performance of Task Arithmetic (8.5%) and slightly outperforms TIES merging (8.4%). Outperforming Task Arithmetic, renowned for its strength in mathematical reasoning Yu et al. (2024), highlights our autonomous algorithm discovery's effectiveness and its potential to surpass well-crafted human-designed algorithms.

Q3: Does training the algorithm factory improve algorithm quality?One of the key contributions of our work is the automatic improvement of model-improving algorithms, which is made possible by training the algorithm factory. Our findings demonstrate that this training leads to a significant enhancement in the quality of generated algorithms, enabling a novel form of LLM self-improvement. Table 2 shows a clear improvement in performance across iterations, particularly for the MATH task. We see substantial performance gain in MATH from 7.0% to 8.5%. This iterative improvement confirms our method's ability to continuously self-improve through the discovery of increasingly effective algorithms.

Figure 3 demonstrates that the quality of algorithms improves with each iteration. This figure shows the distribution of development scores for models created using algorithms generated in each iteration. In the early iterations, low-performing algorithms were dominant, but as learning progressed, we can observe a significant increase in the ratio of high-performing algorithms. By training the algorithm factory, the LLM not only discovers effective model-improving algorithms but also refines these algorithms over time, resulting in increasingly enhanced models.

 

function was not activated in our experimental setting with three models.

In iteration 3, the algorithm evolved to incorporate adaptive weighting mechanisms into the previous mixture strategy of weighted averages and element-wise operations (Figure 5), reaching 73.6% accuracy in GSM8k development set. The weights are dynamically adjusted based on a custom distance metric that considers both the magnitude and direction of the weight vectors.

These results validate the necessity of our framework: while Task Arithmetic and TIES merging are constrained to predefined weight combinations, our framework's ability to explore diverse algorithmic strategies enables it to discover more effective solutions beyond simple weight optimization. The discovered algorithms often involve sophisticated operations that would be difficult to achieve through simpler approaches, such as methods based on custom distance metrics that consider both magnitude and direction of weight vectors, and adaptively adjusted weighting strategies based on model similarity. For detailed analysis, see Appendix C.

## 4 Transferability of Algorithms

We analyze the effectiveness of the algorithms discovered by the algorithm factory on out-of-domain models that were not used in the algorithm generation process.

Experimental SetupTo investigate the transferability of LLM-discovered algorithms, we maintained the original seed model \(M_{0}\) while introducing a new set of candidate models (\(C^{\text{new}}_{1},C^{\text{new}}_{2},C^{\text{new}}_{3}\)) and applied the discovered algorithms to these new models. From models with capabilities similar to the original merge candidates, we selected WizardMath-7B-v1.19Xu et al. (2024a), BioMistral-7B10Labrak et al. (2024), and Starling-LM-7B-alpha11Zhu et al. (2024a) as new candidate models for merging. Although these models differ from the candidate models used in algorithm generation, they are fine-tuned based on Mistral-7BJiang et al. (2023) and can therefore be merged with the seed model. We apply the top 15 algorithms discovered by the algorithm factory (based on their performance on the development set with the original candidate models) to these new models. For comparison, we chose Task Arithmetic, which showed the second-best performance after our proposed method in SS 3, and apply its top 15 mixing ratios (based on development set with the original candidate models) to these new models.

Footnote 9: [https://huggingface.co/WizardLMTeam/WizardMath-7B-V1.1](https://huggingface.co/WizardLMTeam/WizardMath-7B-V1.1)

Footnote 10: [https://huggingface.co/BioMistral/Biódistiral-7B](https://huggingface.co/BioMistral/Biódistiral-7B)

Footnote 11: [https://huggingface.co/berkeley-nest/Starling-LM-7B-alpha](https://huggingface.co/berkeley-nest/Starling-LM-7B-alpha)

Figure 4: A merging algorithm discovered in iteration 1, which combines weighted averages with element-wise maximum and minimum operations. This initial algorithm achieved 71.3% accuracy in GSM8k development set.

 

ResultsFigure 6 is the results of the transferability evaluation for the algorithms. The algorithms discovered by Self-Developing demonstrated transferability on both GSM8k and MATH tasks. In the GSM8k task, many algorithms maintained high performance even when applied to new candidate models for merging. Our LLM-discovered algorithms are positioned above the diagonal line, indicating high scores even when applied to new candidate models. In contrast, the results for Task Arithmetic are concentrated below the diagonal line, suggesting limited transferability. These findings indicate that the algorithm factory not only generates algorithms optimized for specific model sets but also discovers merge algorithms that maintain high performance on similar candidate models. Similar results are obtained for the MATH task, which are provided in Appendix A.

Optimized Task Arithmetic vs. LLM-Discovered AlgorithmsNext, we compare Task Arithmetic that is optimized for the new candidate models, with the LLM-discovered algorithms. For Task Arithmetic, we exhaustively explore all combinations of mixing ratios for the new candidate models, following the same procedure as in SS 3. Table 3 provides a detailed comparison of their performance. It is important to note that the algorithms discovered by the LLM are not optimized for the new candidate models (meaning that these models are out-of-domain for these algorithms).

Our algorithms consistently outperform both the individual models and Task Arithmetic across all tasks. In the GSM8k task, our algorithm achieves a high accuracy of 78.8%, surpassing the best individual model by 3.3 percentage points and the best Task Arithmetic result by 7.4 percentage points. Similarly, in the MATH task, our algorithm reaches 2.5%, more than doubling the performance of Task Arithmetic. These results not only demonstrate the effectiveness of our proposed method but also highlight its robustness when applied to new model sets without re-optimization. The consistent superiority of our approach over Task Arithmetic, particularly on out-of-domain models, underscores the high performance of the discovered algorithms.

Figure 5: A merging algorithm discovered in iteration 3, incorporating adaptive weighting mechanisms based on a custom distance metric. Extends the mixture strategy from iteration 1, achieving 73.6% on GSM8k development set.

 



## 5 Related Work

Self-improvingThe concept of self-improving artificial intelligence was proposed by Minsky (1966) and Good (1965), and later formalized by Schmidhuber (2003). With the rapid development of LLMs, the community has shifted towards practical implementations of self-improvement Huang et al. (2023). Many recent self-improvement approaches primarily focus on the generation of fine-tuning with self-generated training data Yuan et al. (2024); Gulcehre et al. (2023); Zhang et al. (2024); Wang et al. (2023); Xu et al. (2024). Their methods do not generate or learn the improvement strategies themselves. Additionally, agents that modify outputs based on feedback from the environment have been proposed Madaan et al. (2023); Shinn et al. (2023); Ishibashi and Nishimura (2024), but these are different from our goal of improving the LLM itself.

Algorithm Generation using LLMsCode generation by LLMs Jiang et al. (2024) has been proposed for various applications, such as solving reasoning problems Chen et al. (2023) and generating agent actions Wang et al. (2024). Focusing on LLMs' code generation capabilities, several approaches have been suggested where LLMs generate and execute code to enhance their own abilities. For example, Lu et al. (2024) propose a method where LLMs themselves are used to discover loss functions for preference optimization. Zelikman et al. (2023) propose a method to improve a code that makes structured calls to a LLM. These methods generate improvement algorithms using LLMs. Unlike their approaches, we not only generate model-improvement algorithms but also enhance the LLM that generates these algorithms. Another major difference is that we do not rely on external LLMs, like GPT-4 OpenAI (2023), other than the seed model, for algorithm generation. Our work builds upon these foundations but uniquely focuses on the autonomous improvement of both the algorithms and the model generating them, without relying on external models, thus pushing the boundaries of self-improving AI systems.

Figure 6: Transferability of the top 15 merge algorithms for the GSM8k task. The x-axis shows the score on the original set of fine-tuned models used for merging, while the y-axis shows the score when the same merging algorithm is applied to a new set of fine-tuned models. Alphabetic labels (A, B, C, etc.) represent discovered algorithms with high transferability, detailed in Appendix F. Points above the diagonal line indicate better transferability, with higher positions showing greater improvement on new models to be merged.





### Limitations

While our study provides valuable insights, we acknowledge several limitations. First, focusing solely on mathematical reasoning tasks may not fully represent the diverse range of tasks LLMs encounter in real-world applications. Although this choice aligns with standard benchmarks in LLM self-improvement research (e.g., GSM8K and MATH) Zelikman et al. (2022); Zhang et al. (2024); Huang et al. (2023) and allowed for in-depth analysis, extending the evaluation to a broader range of tasks, such as natural language understanding or code generation, could offer additional insights into the generalizability of our findings. Furthermore, due to computational resource constraints, we had to prioritize specific tasks for in-depth analysis, which prevented us from conducting experiments in other domains. While our results demonstrate LLMs' self-improvement capabilities in mathematical reasoning benchmarks, we recognize the importance of validation across broader domains and hope this gap will be addressed in future research.

## Acknowledgments

We would like to thank Takuya Tamura and Daichi Haraguchi at NEC Data Science Laboratories for their valuable discussions and insights throughout this research.

## References

* B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighian, R. Child, A. Ramos, D. M. Ziegler, Jeffrey Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, K. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei (2020)Language models are few-shot learners. In Advances in Neural Information Processing Systems _33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_.
* Chen et al. (2021) Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, H. Ponde de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khalaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. Petroski Such, Dave Cummings, Matthias Plappert, F. Chantzis, Elizabeth Barnes, A. Herbert-Voss, W. Heben Guss, A. Nichol, A. Paino, N. Tezak, Jie Tang, Igor Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. Navar, X. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and Wojciech Zaremba (2021)Evaluating large language models trained on code. CoRRabs/2107.03374. External Links: Link Cited by: SS5.
* W. Chen, X. Ma, X. Wang, and W. W. Cohen (2023)Program of thoughts prompting: disentangling computation from reasoning for numerical reasoning tasks. Trans. Mach. Learn. Res.2023. Cited by: SS5.
* P. Cheng, T. Hu, H. Xu, Z. Zhang, Y. Dai, L. Han, and N. Du (2024)Self-playing adversarial language game enhances llm reasoning. CoRRabs/2404.10642. External Links: Link Cited by: SS5.
* E. Chern, H. Zou, X. Li, J. Hu, K. Feng, J. Li, and P. Liu (2023)Generative ai for math: abei. Cited by: SS5.
* K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jung, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman (2021)Training verifiers to solve math word problems. CoRRabs/2110.14168. External Links: Link Cited by: SS5.
* A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Latman, A. Mathur, A. Schelaten, A. Yang, A. Fan, A. Goyal, A. Hartshorn, A. Yang, A. Mitra, A. Gravnev, A. Hinsvark, A. Rao, A. Zhang, A. Rodriguez, A. Gregerson, A. Spataru, B. Roziere, B. Biron, B. Tang, B. Chern, C. Caucheteux, C. Mayak, C. Bi, C. Marra, C. McConnell, C. Keller, C. Touret, C. Wu, C. Wong, C. Canton Ferrer, C. Nikolaidis, D. Allonsius, D. Song, D. Pintz, D. Livshits, D. Esibou, D. Choudhary, D. Mahajan, D. Garcia-Olano, D. Perino, D. Hughes, E. Lakomkin, E. AlbBadawy, E. Lobanova, E. Dinan, E. Michael Smith, F. Radenovic, F. Zhang, G. Synnaeve, G. Lee, Georgia Lewis Anderson, G. Nail, G. Pang, G. Cucurell, H. Nguyen, H. Korevaar, H. Xu, H. Touvron, I. Zarov, I. Horra, I. M. Klourann, I. Mishra, I. Evtimov, J. Copet, J. Lee, J. Geffert, J. Varnes, J. Park, J. Mahadeokar, J. Shah, J. van der Linde, J. Billock, J. Hong, J. Lee, J. Fu, J. Chi, J. Huang, J. Liu, J. Wang, J. Yu, J. Bitto, J. Spisak, J. Park, J. Rocca, J. Johnston, J. Saxe, J. Jia,

Kalyan Vasuden Alwala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al. 2024. The llama 3 herd of models. _CoRR_, abs/2407.21783.
* Gao et al. (2024) Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Ariya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. 2024. A framework for few-shot language model evaluation.
* Ge et al. (2019) Rong Ge, Sham M. Kakade, Rahul Kidambi, and Praneeth Netrapalli. 2019. The step decay schedule: A near optimal, geometrically decaying learning rate procedure for least squares. In _Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada_, pages 14951-14962.
* Goddard et al. (2024) Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. 2024. Arcee's mergekit: A toolkit for merging large language models. _CoRR_, abs/2403.13257.
* Good (1965) Irving John Good. 1965. Speculations concerning the first ultraintelligent machine. _Adv. Comput._, 6:31-88.
* Gulcehre et al. (2023) Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Maosen Wang, Chenjie Gu, Wolfgang Macherey, Arnaud Doucet, Orhan Firat, and Nando de Freitas. 2023. Reinforced self-training (rest) for language modeling. _CoRR_, abs/2308.08998.
* Guo et al. (2025) Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. _arXiv preprint arXiv:2501.12948_.
* Hendrycks et al. (2021) Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the MATH dataset. In _Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual_.
* Hinton et al. (2015) Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the knowledge in a neural network. _CoRR_, abs/1503.02531.
* Hu et al. (2022) Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In _The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022_. OpenReview.net.
* Huang et al. (2023) Jiaixin Huang, Shixiang Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. 2023. Large language models can self-improve. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023_, pages 1051-1068. Association for Computational Linguistics.
* Ilharco et al. (2023) Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, Ali Farhadi. 2023. Editing models with task arithmetic. In _The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023_. OpenReview.net.
* Ishibashi and Nishimura (2024) Yoichi Ishibashi and Yoshimasa Nishimura. 2024. Self-organized agents: A LLM multi-agent framework toward ultra large-scale code generation and optimization. _CoRR_, abs/2404.02183.
* ECCV 2024
- 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part XLV_, volume 15102 of _Lecture Notes in Computer Science_, pages 207-223. Springer.
* Jiang et al. (2023) Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulinier, Leliao Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothelec Lacroix, and William El Sayed. 2023. Mistral 7b. _CoRR_, abs/2310.06825.
* Jiang et al. (2024) Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A survey on large language models for code generation. _CoRR_, abs/2406.00515.
* Joy et al. (2023) Tom Joy, Francesco Pinto, Ser-Nam Lim, Philip H. S. Torr, and Punenet K. Dokania. 2023. Sample-dependent adaptive temperature scaling for improved calibration. In _Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023_, pages 14919-14926. AAAI Press.
* Labrak et al. (2024) Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. 2024. Biomism!: A collection of open-source pretrained large language models for medical domains. In _Findings of the Association for Computational Linguistics, ACL 2024, Bangskok, Thailand and virtual meeting, August 11-16, 2024_, pages 5848-5864. Association for Computational Linguistics.
* Looshchilov and Hutter (2017) Ilya Looshchilov and Frank Hutter. 2017. SGDR: stochastic gradient descent with warm restarts. In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net 

Chris Lu, Samuel Holt, Claudio Fanconi, Alex J. Chan, Jakob N. Foerster, Mihaela van der Schaar, and Robert Tjarko Lange. 2024. Discovering preference optimization algorithms with and for large language models. _CoRR_, abs/2406.08414.
* 16, 2023_.
* Minsky (1966) Marvin Minsky. 1966. Artificial intelligence. _Scientific American_, 215(3):247-260.
* OpenAI (2023) OpenAI. 2023. Best practices for prompt engineering with the openai api.
* OpenAI (2023) OpenAI. 2023. GPT-4 technical report. _CoRR_, abs/2303.08774.
* 16, 2023_.
* Schmidhuber (2003) Jurgen Schmidhuber. 2003. Goedel machines: Self-referential universal problem solvers making provably optimal self-improvements. _CoRR_, cs.LO/0309048.
* 16, 2023_.
* Silver et al. (2016) David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Vedavyas Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, NaI Kalchbrenner, Ilya Sutskever, Timothy P. Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demirs Hassabis. 2016. Mastering the game of go with deep neural networks and tree search. _Nat._, 529(7587):484-489.
* Teknium (2023) Teknium. 2023. Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants. HuggingFace.
* Tu et al. (2023) Tao Tu, Anil Palepu, Mike Schaekermann, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Nenad Tomasev, Shekoofeh Azizi, Karan Singhal, Yong Cheng, Le Hou, Albert Webson, Kavita Kulkarni, S. Sara Mahdavi, Christopher Semturs, Jurati Gottwick, Joelle K. Barral, Katherine Chou, Gregory S. Corrado, Yossi Matias, Alan Karthikesalingam, and Vivek Natarajan. 2024. Towards conversational diagnostic AI. _CoRR_, abs/2401.05654.
* Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_, pages 5998-6008.
* Wang et al. (2020) Pei-Hsin Wang, Sheng-Jou Hsieh, Shih-Chieh Chang, Yu-Ting Chen, Jia-Yu Pan, Wei Wei, and Da-Chang Juan. 2020. Contextual temperature for language modeling. _CoRR_, abs/2012.13575.
* Wang et al. (2024) Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. 2024. Executable code actions elicit better LLM agents. In _Fortyfirst International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024_. OpenReview.net.
* Wang et al. (2023) Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirizi. 2023. Self-instruct: Aligning language models with self-generated instructions. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023_, pages 13484-13508. Association for Computational Linguistics.
* Xu et al. (2024a) Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Davain Jiang. 2024a. Wizardlm: Empowering large pre-trained language models to follow complex instructions. In _The Twelfth International Conference on Learning Representations, ICLR 2024, Venuna, Austria, May 7-11, 2024_. OpenReview.net.
* Xu et al. (2023) Jing Xu, Andrew Lee, Sainbayar Sukhbataar, and Jason Weston. 2023. Some things are more CRINGE than others: Preference optimization with the pairwise cringe loss. _CoRR_, abs/2312.16682.
* Xu et al. (2024b) Yifan Xu, Xiao Liu, Xinghan Liu, Zhenyu Hou, Yueyan Li, Xiaohan Zhang, Zihan Wang, Aohan Zeng, Zhengxiao Du, Wenyi Zhao, Jie Tang, and Yuxiao Dong. 2024b. Chatglm-math: Improving math problem-solving in large language models with a self-critique pipeline. _CoRR_, abs/2404.02893.
* 16, 2023_
 

Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. Language models are super mario: Absorbing abilities from homologous models as a free lunch. In _Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024_. OpenReview.net.
* Yuan et al. (2024) Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. 2024. Self-rewarding language models. In _Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024_. OpenReview.net.
* Zelikman et al. (2023) Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam Tauman Kalai. 2023. Self-taught optimizer (STOP): recursively self-improving code generation. _CoRR_, abs/2310.02304.
* December 9, 2022_.
* Zhang et al. (2024) Dan Zhang, Singing Zhoubian, Yisong Yue, Yunxiao Dong, and Jie Tang. 2024. Rest-mcts*: LLM self-training via process reward guided tree search. _CoRR_, abs/2406.03816.
* Zhu et al. (2024a) Banghua Zhu, Evan Frick, Tianhao Wu, Hanlin Zhu, Karthik Ganesan, Wei-Lin Chiang, Jian Zhang, and Jiantao Jiao. 2024a. Starling-7b: Improving helpfulness and harmlessness with RLAIF. In _First Conference on Language Modeling_.
* Zhu et al. (2024b) Yuqi Zhu, Jia Li, Ge Li, Yunfei Zhao, Jia Li, Zhi Jin, and Hong Mei. 2024b. Hot or cold? adaptive temperature sampling for code generation with large language models. In _Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada_, pages 437-445. AAAI Press.

## Appendix A Transferability

Our proposed method demonstrated significant transferability in both the GSM8k and MATH tasks, as shown in Figure 6 and Figure 7. These figures showcase the performance of various discovered algorithms on their original test sets and on transfer test sets with new, unseen merge candidate models. For a more detailed breakdown of algorithm performance, we refer to Table 4.

For the MATH task, most Task Arithmetic scores are below \(1\%\) when applied to new models, indicating the challenge of transferability (Figure 7). In contrast, our generated algorithms achieved scores of up to approximately \(2.5\%\) on new models, significantly outperforming Task Arithmetic.

Transferability for GSM8k is particularly strong. Algorithms A (Figure 10) to D (Figure 15) show improved performance when applied to new models. For instance, Algorithm A (Figure 10) improves from \(76.05\%\) on the original test to \(78.75\%\) on the transfer test.

A particularly interesting finding is the remarkable cross-task performance of some algorithms. Notably, Algorithm G (Figure 18), discovered using GSM8k data, achieves an accuracy of \(74.82\%\) on GSM8k and \(7.96\%\) on the MATH task. This performance on MATH is nearly on par with Algorithm I (Figure 20), which was specifically optimized for the MATH task (\(8.50\%\)). Such cross-task effectiveness suggests the potential for discovering algorithms with LLMs that are effective across various problem types. Additionally, it was found that

Figure 7: Transferability of the top 15 merge algorithms for the MATH task. The x-axis shows the test score on original models to be merged, while the y-axis shows the score on new models to be merged. Each point represents a different algorithm, with points above the diagonal line indicating better transferability.

 

algorithms discovered for MATH are also effective on GSM8k, suggesting that exploring algorithms on more challenging tasks may lead to the discovery of algorithms that are effective across a broader range of tasks.

## Appendix B Temperature Decay for Iterative DPO

Iterative DPO (Yuan et al., 2024; Xu et al., 2023; Yuan et al., 2024) has been shown to outperform a single round of DPO by iteratively updating the model through preference optimization steps, thus producing refined outputs.

Temperature (Hinton et al., 2015) is a crucial parameter for controlling the creativity of the generated text. In our method, it also plays a significant role in the generation process of model-improving algorithms. Generally, a higher temperature in LLMs results in more diverse and creative text, while a lower temperature yields more accurate outputs (OpenAI, 2023). This can be viewed as a means to control the trade-off between _exploration_ and _exploitation_. Recent studies have proposed methods to dynamically adjust the temperature based on the input text Zhu et al. (2024b); Joy et al. (2023); Wang et al. (2020). In iterative DPO, the temperature has traditionally been set manually12.

Footnote 12: For instance, in (Yuan et al., 2024), the temperature is fixed at \(T=0.6\) or \(T=0.7\) during data generation step for iterative DPO.

To appropriately balance exploration and exploitation during the algorithm generation process, we introduce a temperature decay inspired by learning rate decay Ge et al. (2019); Loshchilov and Hutter (2017). This approach allows for dynamic adjustment of the exploration strategy as iterations progress.

In the initial iterations, a high initial temperature facilitates the generation of a wide range of creative algorithms, maximizing the opportunity to discover innovative solutions that might be overlooked by conventional approaches. During the mid-phase, a gradual decrease in temperature leverages the effective features of the algorithms learned so far while continuing to explore new variations and combinations. In the later stages, a lower temperature focuses the search around known high-performance algorithms, increasing the likelihood of efficiently discovering superior algorithms.

Specifically, the temperature update at iteration \(t\) is based on the Inverse Time Decay schedule:

\[T_{t}=\frac{T_{1}}{1+\beta(t-1)}, \tag{4}\]

where \(T_{1}\) denotes the initial temperature, and \(\beta\in\mathbb{R}\) is a hyperparameter that controls the decay rate. By adjusting the decay rate \(\beta\), one can regulate the speed of the transition from exploration to exploitation.

ExperimentThis experiment investigates the impact of temperature settings and their decay on the quality and diversity of Python functions generated in an iterative DPO process. Figure 8 visualizes the filtering results to observe qualitative changes in Python functions sampled from the algorithm factory \(\pi^{g}_{t}\) at each iteration. The figure shows the results for different temperature settings with and without temperature decay. The experiment was conducted under the following conditions:

* Initial temperatures: \(\{1.20,0.70,0.20\}\)
* Temperature control: Fixed temperature and temperature decay (\(\beta=0.2\))
* Number of iterations: 3 for each condition

\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & \multicolumn{2}{c}{GSM8k} & \multicolumn{2}{c}{MATH} \\ \cline{2-5} Discovered Algorithms & Original Test & Transfer & Original Test & Transfer Test \\ \hline \hline \multicolumn{5}{c}{Algorithms discovered using GSM8k} \\ \hline A (Figure 10) & **76.05** & 78.75 & 2.01 & **1.72** \\ B (Figure 13) & 76.05 & 78.75 & 1.99 & 1.63 \\ C (Figure 14) & 75.96 & 78.75 & 1.82 & 1.67 \\ D (Figure 15) & 75.96 & **78.84** & 1.82 & 1.70 \\ E (Figure 16) & 75.80 & 76.78 & 0.29 & 0.62 \\ F (Figure 17) & 75.31 & 77.03 & 5.10 & 0.26 \\ G (Figure 18) & 74.82 & 78.10 & **7.96** & 0.36 \\ H (Figure 19) & 74.49 & 74.73 & 6.22 & 1.87 \\ \hline \end{tabular} 
\begin{tabular}{l c c c} \hline \hline I (Figure 20) & 69.48 & 69.48 & **8.50** & 0.08 \\ J (Figure 21) & 70.30 & **78.10** & 8.41 & 0.06 \\ K (Figure 22) & 70.30 & **78.10** & 8.41 & 0.06 \\ L (Figure 23) & 69.32 & 63.41 & 8.13 & 0.17 \\ M (Figure 24) & 69.89 & 53.40 & 7.99 & 1.27 \\ N (Figure 25) & **73.83** & 65.14 & 7.78 & 0.92 \\ O (Figure 26) & 71.29 & 65.87 & 7.48 & **2.40** \\ P (Figure 27) & 69.57 & 65.71 & 7.02 & 1.97 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Performance of merged models on GSM8k and MATH tasks. Algorithms A-H were developed using GSM8k data, and algorithms I-P were developed using MATH data. ‘Original Test’ columns show the performance on merge candidate models used in the algorithm search, while ’Transfer Test’ columns indicate performance on new, unseen merge candidate models, assessing the transferability of each algorithm.

 

The generated Python functions were filtered into the following categories:

* Duplicate Data
* No Function Extracted
* Success (executable functions)
* Non-Executable Function
* Timeout

Key finding 1: Higher temperatures is effective for enhancing data diversityComparing high and low temperature settings, it was found that higher temperatures consistently produce more diverse data. Throughout all iterations, low temperature (\(T=0.20\)) tends to generate a higher proportion of duplicate data, reducing diversity. In contrast, high temperatures (\(T=0.70\), \(T=1.20\)) produce less duplication and more diverse data. The \(T=0.70\) setting generates the highest number of executable functions ('Success' in Figure 8) in the first iteration, but this proportion decreases sharply in later iterations. The \(T=1.20\) setting, while having a lower initial success rate, continues to generate a relatively high number of executable functions in later iterations. These results suggest that higher temperature settings can generate highquality data more consistently over the long term.

Key finding 2: Temperature decay is effective for stable data generationApplying temperature decay tends to be more effective than using a fixed temperature for generating data stably. With fixed temperatures, there is a tendency for the rate of non-executable functions to increase in later iterations. When temperature decay is applied, the rate of duplicate functions shows an increase in later iterations, but the rate of non-executable functions decreases, resulting in a small increase in the number of executable algorithms ('Success'). This phenomenon suggests that temperature decay may shift the generation process from creating more varied data towards generating more accurate data. These findings indicate that an appropriate temperature decay strategy could play a role in optimizing the quality and diversity of generated data in iterative DPO.

## Appendix C Analysis of LLM-Discovered Algorithms

Complexity of Coefficient CalculationIn model merging, coefficients play a crucial role in determining how different models are merged. The coefficients in merge strategies mainly included the following: (1) Weighting Factor, determining the extent to which weights of different models are reflected, (2) Adaptive Coefficient, dynamically adjusted based on model characteristics (e.g., weight norms), and (3) Blend Ratio, determining the ratio

Figure 8: Impact of temperature settings and decay on generated Python functions across iterations. Results are categorized as: Duplicate Data, No Function Extracted (failed to generate a function), Success (executable functions), Non-Executable Function (syntactically incorrect), and Timeout (execution time exceeded).

 

when combining different merge strategies (e.g., multiplication and averaging). For example:

```
@Weightingfactor x+alpha*(y-x) @AdaptiveCoefficient alpha*torch.mul(x,y)+beta*torch.max(x,y) }@BlendRatio (average+element_wise_maximum+ element_wise_minimum)/alpha
```

There was a marked tendency for coefficient calculations to become more complex as iterations progressed. In iteration \(t=1\), relatively simple coefficients (e.g., a fixed value of 0.5) were often used for mixing task vectors, but by iteration \(t=2\), a method was introduced for dynamically calculating coefficients using the cosine similarity of task vectors (Algorithm O; Figure 26), similar to Model Stock [34]. The increasing complexity of coefficient calculations may enable the realization of more precise and adaptive merge strategies. This likely resulted in a tendency to enhance performance by fine-tuning coefficients while maintaining the basic structure of specific strategies.

Diversity of IdeasIn the early iterations, a wide range of ideas were explored. Table 5 shows a portion of the words and their frequencies found in the strategies of algorithms generated by the LLM during iteration \(t=1\). This result demonstrates that diverse methods are proposed. The most frequently used methods are based on 'similarity' and 'distance'. There is a clear tendency to utilize geometric information of vectors ('angle', 'geometric', 'metric', 'norm', 'frobenius', etc.). Additionally, 'element-wise' and 'pairwise' operations are also commonly observed. Furthermore, a wide array of algorithms are proposed, including statistical methods ('kullback', 'leibler', 'gaussian', 'distribution', 'entropy', 'lasso', etc.), learning-based approaches ('learning', 'train'), matrix decomposition ('factorization', 'svd', 'pca'), and grouping techniques ('clustering', 'neighbor', 'kmeans', etc.). Among the creative algorithms, many interesting ones are included. For example, the set similarity-based method is a unique algorithm that treats vectors as sets of values and calculates their set similarity (Figure 9). Although the development scores of models using these methods are not high, there is potential to discover superior algorithms by increasing the number of generated algorithms.

Merging strategy: Algorithm AWe explain the merging algorithm that achieved the best performance on the GSM8k task among the generated algorithms, demonstrating exceptionally high transferability to out-of-domain models (labeled 'A' in Figure 6). The exact function generated can be found in Figure 10. Below, we present a mathematical formulation of the algorithm.

This merging algorithm repeatedly applies a function (implemented as hybrid_merge_strategy in the code) to sequentially merge the task vectors. Starting with the initial vector \({\tau^{\text{merged}}_{1}}=\tau_{1}\), the function \(f\) is applied iteratively to combine each subsequent task vector \(\tau_{i}\) with the current merged vector. This process can be represented as follows:

\[\begin{split}&{\tau^{\text{merged}}_{2}}=f({\tau^{\text{merged}}_{1}}, {\tau_{2}}),\\ &{\tau^{\text{merged}}_{3}}=f({\tau^{\text{merged}}_{2}},{\tau_ {3}}),\\ &\vdots\\ &{\tau^{\text{merged}}_{K}}=f({\tau^{\text{merged}}_{K-1}},{\tau _{K}}).\end{split} \tag{5}\]

Finally, this algorithm outputs the final merged vector \({\tau^{\text{merged}}_{K}}\). Here, the function \(f\) can be defined as:

\[f({\tau^{\text{merged}}_{i-1}},{\tau_{i}})=\frac{1}{2}\left({\tau^{\text{ merged}}_{i-1}}+\mu_{i}\mathbf{1}\right), \tag{6}\]

where \(d\) is the dimension of the task vectors, and \(\mathbf{1}\in\mathbb{R}^{d}\) is a vector with all elements are 1. \(\mu_{i}\in\mathbb{R}\) is the mean of all elements of the task vector \(\tau_{i}\):

\[\mu_{i}=\frac{1}{d}\sum_{j=1}^{d}(\tau_{i})_{j}, \tag{7}\]

where \((\tau_{i})_{j}\in\mathbb{R}\) denotes the \(j\)-th element of \(\tau_{i}\).

 


Discovered diverging function

```
defmerge_models(model_dict,device): '''Developandimplementanovelalgorithm...(omitted)'' #Newstrategiesformergingthemodelweights; #1.Considercalculatingthemowns(sizes)oftheweighttensors. #2.Defineaweightingfunctionthattakesintocountbothmodelconsistencyanddiversity. #3.Introduceaparameter'p',adjustingthenbalancebetweenmodeconsistencyanddiversity. #4.Introduceanotherparameter'alpha'adjustingthenbalancebetweenlinearinterpolationand weightedaveraging.
#Assignmenters'p'and'alpha' p=0.75 alpha=0.5
#Convertmodelweightstensorsandmovententothespecifieddevice(CPUorGPU) weights=[model.detach().to(device)formedinmodel_dict.values()] deweighting_function(x,y,p,alpha): #CalculateweightusingThe Jaccardsimilarity intersection*torch.sum(torch.signoid(x)*torch.sigmaoid(y)) union=torch.sum(torch.simoid(x))+torch.sum(torch.signoid(y)) jaccard=intersection/union #Normalizethewithgitsusinsweigthingparameter'p' normalized_jaccard=jaccard**p  #Combineweightsunagmixofinterpolationandaveragingwithparameter'alpha' returnalpha*x*(l-alpha)*normalized_jaccard*y
#Initializemerged_weightswiththerstomodel'swights merged_weights=weights[8].clone() forwegitinfweights[1:]:  merged_weights= weighting_function(merged_weights,weight,p,alpha) returnerged_weights
```

Figure 9: This algorithm demonstrates a creative approach. The vectors are interpreted as sets of values, with the Jaccard index serving as a similarity measure for adaptive weighting.

 


Figure 10: Discovered algorithm A. This is one of the most effective algorithms discovered by the LLM, generated during iteration \(t=3\).

 

[10] M. C. Gonzalez-Garcia, J. F. Maldacena and A. Strominger, _Phys. Rev. D_**48** (1993) 2576; Phys. Lett. B **282** (1994) 309; Nucl. Phys. B **285** (1994) 357.
* [11] G. Veneziano, _Nucl. Phys. B_**290** (1981) 159.
* [12] E. Witten, _Commun. Math. Phys._**10** (1982) 159.

 

```
0:\(M_{0}\): Seed model, \(\mathcal{T}\): Target Task
1:\(x\): Prompt
2:\(I\): Max iterations, \(N\): Number of algorithm generation
3:\(T_{1}\): Initial temperature, \(\beta\): Decay rate
4:\(p_{w}\), \(p_{l}\). Percentages for DPO data selection
5:\(k\): Number of top-performing algorithms to add from previous iterations
6:\(S\): Number of low-performing algorithms to pair with each high-performing algorithm
7:\(d^{\text{best}}\): Best improved model
8:Initialize algorithm generator: \(\pi_{t}^{g}\gets M_{0}\)
9: Initialize best model: \(M^{\text{best}}\gets M_{0}\)
10: Initialize best score: \(s_{\text{best}}\leftarrow-\infty\)
11:for\(t=1\) to \(I\)do
12:// Algorithm Generation
13: Update temperature: \(T_{t}=\frac{T_{1}}{1+\beta(t-1)}\)\(\triangleright\) Decrease temperature
14:\(A_{t}\leftarrow\{\}\)\(\triangleright\) Initialize empty set for algorithms
15:for\(i=1\) to \(N\)do
16:\(a_{t}^{(i)}\sim\pi_{t}^{g}(a\mid x)\)\(\triangleright\) Generate algorithm with temperature \(T_{t}\)
17:if IsValid(\(a_{t}^{(i)}\)) then
18:\(A_{t}\gets A_{t}\cup\{a_{t}^{(i)}\}\)\(\triangleright\) Add valid algorithm to set
19:endif
20:endfor
21:// Algorithm Evaluation
22:\(S_{t}\leftarrow\{\}\)\(\triangleright\) Initialize empty set for scores
23:for\(a_{t}^{(i)}\in A_{t}\)do
24:\(M_{t}^{(i)}\leftarrow\text{Apply}(a_{t}^{(i)},M_{0})\)\(\triangleright\) Apply algorithm to get improved model
25:\(s_{t}^{(i)}\leftarrow\text{Evaluate}_{\mathcal{T}}(M_{t}^{(i)})\)\(\triangleright\) Evaluate improved model with dev set
26:\(S_{t}\gets S_{t}\cup\{s_{t}^{(i)}\}\)\(\triangleright\) Add task score to set
27:endfor
28:// DPO Data Selection
29:\(s_{pw}\leftarrow\text{Percentile}(S_{t},100-p_{w})\)\(\triangleright\) Top \(p_{w}\%\) score threshold
30:\(s_{p_{l}}\leftarrow\text{Percentile}(S_{t},p_{l})\)\(\triangleright\) Bottom \(p_{l}\%\) score threshold
31:\(A_{t,w}\leftarrow\{a_{t}^{(i)}\in A_{t}\mid s_{t}^{(i)}\geq s_{pw}\}\)\(\triangleright\) High-performing algorithms
32:\(A_{t,l}\leftarrow\{a_{t}^{(i)}\in A_{t}\mid s_{t}^{(i)}\leq s_{p_{l}}\}\)\(\triangleright\) Low-performing algorithms
33:\(A_{\text{pre},w}\leftarrow\bigcup_{j=1}^{t-1}A_{j,w}\)\(\triangleright\) Union of all previous high-performing algorithms
34:\(A_{\text{top}3}\leftarrow\text{TopK}(A_{\text{pre},w},k)\)\(\triangleright\) Select top 3 algorithms based on scores
35:\(A_{t,w}\gets A_{t,w}\cup A_{\text{top}3}\)\(\triangleright\) Add top 3 to high-performing set
36:\(\mathcal{D}\leftarrow\{\}\)\(\triangleright\) Initialize empty DPO dataset
37:for\(a_{t,w}^{(i)}\in A_{t,w}\)do
38:\(L_{i}\leftarrow\text{Sample}(A_{t,l},S)\)\(\triangleright\) Sample \(S\) low-performing algorithms
39:for\(a_{t,l}^{(j)}\in L_{i}\)do
40:\(\mathcal{D}\leftarrow\mathcal{D}\cup\{(x,a_{t,w}^{(i)},a_{t,l}^{(j)})\}\)\(\triangleright\) Add pair to DPO dataset
41:endfor
42:endfor
43:// Update Algorithm Generator
44:Update \(\pi_{t}^{q}\) to \(\pi_{t+1}^{q}\) using DPO with \(\mathcal{D}\)
45:// Update Best Model
46:if\(\max(S_{t})>s_{\text{best}}\)then
47:\(s_{\text{best}}\leftarrow\max(S_{t})\)
48:\(M^{\text{best}}\leftarrow\text{Apply}(a_{t}^{(i^{*})},M_{0})\) where \(i^{*}=\arg\max_{i}s_{t}^{(i)}\)
49:endif
50:endfor
51:return\(M^{\text{best}}\)
```

**Algorithm 1** Self-Developing 

Prompt

Figure 12 shows the prompt template we used for generating model merging algorithms. The prompt uses a relatively creative merging algorithm as a one-shot example (Figure 11). While simpler examples might seem sufficient, our preliminary experiments suggested the need for a more sophisticated example to guide the generation of creative merging algorithms.

During our preliminary experiments, we investigated how the presence or absence of a one-shot example affects algorithm generation. This example serves multiple purposes: demonstrating the expected format of a Python function, showing how to handle model weights as tensors, and illustrating basic weight combination operations.

Our preliminary exploration of zero-shot settings (i.e., without the one-shot example) revealed several important challenges. Many generated outputs failed to be executable Python functions, often containing syntax errors or undefined variables. The generated algorithms also showed less variety in their approaches, mostly converging to simple weighted averaging operations.

These preliminary findings led us to include the one-shot example in our main experiments, as it appeared crucial not only for ensuring the generation of executable code but also for encouraging the exploration of diverse algorithmic strategies. The example helps the LLM understand both the technical requirements (e.g., proper tensor operations) and the potential space of solutions for model merging algorithms 

* ```

Figure 11: One-shot example.

 

* # Task The goal is to merge the weights of multiple pre-trained language models to create a merged model that effectively combines the weights of different models to achieve higher performance. Refer to the code below and devise a new merging strategy to implement.

Reference Code "python importtorch models={ oAIR/Abel-7B-082': torch.rand(dim),# Abel-7B-082isamodelfine-tunedformathematicaltasks, demonstratingstrongperformanceondatasetssuchasGSM8kandMATH. 'SciPhi/SciPhi-Mistral-7B-32K':torch.rand(dim),# SciPih-Mistral-7B-32kisaefine-tunedLLM focusedonscientificreasoningandeducation,optimizedforAlpaca-styleprompts. 'teknium/OpenHermes-2.5-Mistral-7B':torch.rand(dim),# OpenReferences2.5isafinetunedmodel, buildingonOpenHermes2,specificallyenhancedwithadditionalcodedatasets.Trainingoncode improveditsperfromancesonvariousnon-codepenchmarkslikeTruftfulQAandAGIeval. } dermerge_models(model_dict,device): ... Args: model_dict(dict):Adictionarywherekeysaremodelnamesandvaluesarethemodelweights. device(torch.device):Thedevice(CPUorGPU)onwhichthecomputationwillbeperformed. Returns:torch.Tensor:Theweightsofthemergedmodel. .. #Implementthemergestargetyhere
ImplementationInstructions Implement the 'merge_models' function and devise a new strategy for merging the model weights. Consider combining multiple strategies such as weighted averages, element-wise maximums, element-wise minimums, geometric means, Manhattan distances (L1 norm), cosine similarity, Euclidean distances (L2 norm), harmonic means, median merging, matrix factorization, or hadamard product. Document your thought process and the changes you make in the code.

#### Example1 "python [One-shotexaple] ""
Example2 """ python deffmerge_models(model_dict,device):  . Developandimplementanovelalgorithmmorrengthenodelweights.Yourgoalsistocreateauniqueandeffectivestrategythatcombinesvariousexistingtechniquesandintroducesnewapproachestoachieveoptimalperformance.Considerintegratingmethodsuchssadaptiveweighting ,hybridstrategies,oradvancedheuristicstocreateamoreinnovativemergingtechnique. Args: model_dict(dict):Adictionarywherekeysaremodelnamesandvaluesarethemodelweights. device(torch.device):Thedevice(CPUorGPU)onwhichthecomputationwillbeperformed. Returns: torch.Tensor:Theweightsofthemergedmodel. .. #Newwstrategiesformergingthemodelweights:

Figure 12: Prompt template.

 



## Appendix F Discussion

In this appendix we discuss the role of the "classicality" of a quantum field theory in which the classical fields are coupled to the classical fields. We will show that the classical field is coupled to the classical fields, and it is not possible to derive any quantum field theory in terms of the classical fields. The quantum field theory can be formulated as an extension of the classical field theory to the classical fields. In particular, we will consider the case where the classical fields are coupled to the classical fields. This is done by considering the classical fields with the classical fields. We will also study the classical fields on the classical fields.



* #Server_module.com/

Figure 14: Discovered algorithm C.

Figure 13: Discovered algorithm B.

 

deffmerge_models(model_dict,device):  '''Developandimplementanovelalgorithm...(omitted)'' #*Newwstrategiesformergingthemodelweights:  %1.Designahybridstrategybyperformingelement-wisemultiplicationandmean  #2.Definettwoparameters,alphaandbeta,tocontrolthemergingratio
#Convertmodledweightstotensorsandmovethemothespecifieddevice(CPUorGPU) weights=[model.detach().toString(device)formedinmodel_dict.values()]
#Definetwoparameterstocontrolthenemergingratio alpha=0.6 beta=0.4 defmerge_strategy(x,y,alpha=0.5,beta=0.5):  #Performement-wisemultiplication xy=x*y  #Performmeamaggregationoftindheaverageweights returnalpha*x*beta*torch.mean(xy,dim=0)
#Initializernerged_weightswiththefirstmodel'sweights merged_weights=weights[@].clone()
#Iterativelymergeeachsubsequentmodel'sweights forweightinweights[!:]:  merged_weights=merge_strategy(merged_weights,weight) returnmerged_weights ```

Figure 16: Discovered algorithm E.

Figure 15: Discovered algorithm D.

 

* #ServerizedalgorithmF

```
defmerge_models(model_dict,device): '''Developandimplementanovvelalgorithm...(omitted)''  #*New*strategiesformergingthemodelweights: #-Hybridapproach:combinelement-wisemultiplicationwithaverage #Convertmodelweightstotensorsandommovetherothespecifieddevice(CPUorGPU) weights=[model.detach().to(device)formedimplodel_dict.values()] defhybrid_merge_strategy(base,to_merge,alpha=0.5):  #Calculatetheaverageweights  average=(base+torch.mean(to_merge,dim=0)*@.5)  #Scaletheweightsotbeaddedbutkeepthemportantewightsfromthetopmodelslarger returnbase+alpha*(torch.mean(to_merge,dim=0)-base)*@.5 merged_weights=weights[0].clone()
#Sorttheweightsbasedonthemorms(sizes)oftheweighttensorsinddescendingorder weights.sort(key=lambdax:torch.norm(x),reverseTrue)
#Iterativelymergedetheweightswiththecurrentmerged_weights forweightinweights:  merged_weights=hybrid_merge_strategy(merged_weights, weight) returnmerged_weights
```

Figure 17: Discovered algorithm F.

Figure 18: Discovered algorithm G.

 



[MISSING_PAGE_POST]

 

* #Here_models(model_dict,device):  "Developandimplementanovelalgorithm...(omitted)'  #*Newwstrategiesformergingthemodelweights:  #1.Lspiproicwovelstategiesbyconsideringdependencybetweeneachpairofmodels.  #2.Adaptiveweightingbasedonhard/easyexamplesorlayers  #3.Matrixfactorizationwithpriorknowledgeembedding(e.g.domainknoweldgeorontologyinformation)  #4.Hybridstrategythatadaptsweightinghasedomodelrobustnessandperformanceonspecifictasks
#Addmethodsformmatrixfactorizationandotheradvancedmergingtechniques  #Convertmodelweightstotensorsandmovethenotthespecifieddevice(CPUorGPU)  weights=[model.detach().to(device)formedinmodel_dict.values()]  defmerge_strategy(x,y):  #Hard-codedsamplestrategy  return(x+y)/2 ```
#Iterativelymergechsubsequentmodel'sweights  merged_weights=weights[0].clone()  ifi,weightinnumerate{weights[1:],start=1};  ifiX2==1; #Applythemixofelement-wisemaximumsandmedianmerging  merged_weights=merge_strategy(merged_weights,weight)  returnmerged_weights
```

Figure 21: Discovered algorithm J.

Figure 22: Discovered algorithm K.

 



Figure 23: Discovered algorithm L.

Figure 24: Discovered algorithm M.



* #Server_config():
 



## Appendix C

Figure 27: Discovered algorithm P.