MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
Authors: Jing Han
Binwei Yan
Tianyu Guo
Zheyuan Bai
Mengyu Zheng
Hanting Chen
Ying Nie
Venue: ICML 2025 poster (2025)
Keywords: LLMs
Agent
Mixture-of-Roles
Primary area: deep_learning->large_language_models

Abstract: Despite recent advancements of fine-tuning large language models (LLMs) to facilitate agent tasks, parameter-efficient fine-tuning (PEFT) methodologies for agent remain largely unexplored. In this paper, we introduce three key strategies for PEFT in agent tasks: 1) Inspired by the increasingly dominant \textit{Reason+Action} paradigm, we first decompose the capabilities necessary for the agent tasks into three distinct roles: reasoner, executor, and summarizer. The reasoner is responsible for comprehending the user's query and determining the next role based on the execution trajectory. The executor is tasked with identifying the appropriate functions and parameters to invoke. The summarizer conveys the distilled information from conversations back to the user. 2) We then propose the Mixture-of-Roles (MoR) framework, which comprises three specialized Low-Rank Adaptation (LoRA) groups, each designated to fulfill a distinct role. By focusing on their respective specialized capabilities and engaging in collaborative interactions, these LoRAs collectively accomplish the agent task. 3) To effectively fine-tune the framework, we develop a multi-role data generation pipeline based on publicly available datasets, incorporating role-specific content completion and reliability verification.
We conduct extensive experiments and thorough ablation studies on various LLMs and agent benchmarks, demonstrating the effectiveness of the proposed method. This project is publicly available at https://mor-agent.github.io

MoRAgent:
Parameter Efficient Agent Tuning with Mixture-of-Role
Jing Han
Binwei Yan *2
Tianyu Guo?
Zheyuan Bai? Mengyu Zheng? Hanting Chen Ying Nie
Abstract
range of natural language processing (NLP) tasks. Further- more, LLMs have also demonstrated their capability in more challenging tasks, such as function-calling as Al agents. For instance, certain research endeavors (Hong et al., 2023; Shen et al., 2024b; Wang et al., 2024) employ prompt engi- neering methodologies. This approach involves the design of meticulously crafted prompts to effectively activate the agent-related capabilities in LLMs such as reasoning and tool-utilization. These works have shown superior agent performance in large commercial models, like ChatGPT and GPT-4. For open source LLMs (Touvron et al., 2023; Yang et al., 2024), although they have achieved consider- able success across various NLP tasks, they still lag behind commercial models when acting as agents. Consequently, another approach has emerged: fine-tuning models using agent-specific data, which has yielded remarkable results on variety of agent tasks (Qin et al., 2023; Zeng et al., 2023; Chen et al., 2024).
Despite recent advancements of fine-tuning large language models (LLMs) to facilitate agent tasks, parameter-efficient fine-tuning (PEFT) method- ologies for agent remain largely unexplored. In this paper, we introduce three key strategies for PEFT in agent tasks: 1) Inspired by the increas- ingly dominant Reason Actionn paradigm, we first decompose the capabilities necessary for the agent tasks into three distinct roles: reasoner, executor, and summarizer. The reasoner is responsible for comprehending the user's query and determin- ing the next role based on the execution trajec- tory. The executor is tasked with identifying the appropriate functions and parameters invoke. The summarizer conveys the distilled informa- tion from conversations back to the user. 2) We then propose the Mixture-of-Roole (MoR) frame- work, which comprises three specialized Low- Rank Adaptation (LoRA) groups, each designated to fulfill a distinct role. By focusing on their re- spective specialized capabilities and engaging in collaborative interactions, these LoRAs collec- tively accomplish the agent task. 3) To effectively fine-tune the framework, we develop multi-rol role data generation pipeline based on publicly avail- able datasets, incorporating role-specific content completion and reliability verification. We con- duct extensive experiments and thorough abla- tion studies various LLMs and agent bench- marks, demonstrating the effectiveness of the pro- posed method. This project is publicly available at https:://// / /tm///moraeeeeeeeeeeeeee mmortaeent github. io/.
Most existing research on agent fine-tuning focuses on full- parameter fine-tuning, which presents two significant chal- lenges. First, the computational resources required to han- dle billions or tens of billions of model parameters pose substantial barrier to the widespread adoption of agents. Second, full- full-parameter fine-tuning compromises the general capabilities of the original base model, thereby limiting the flexibility of users to seamlessly switch between general tasks and agent-specific tasks. To address these challenges, Prrameter-Efficieen
TuneiTuning (PEFT) methods offer a vi- able solution (Houlsby et al., 2019; Li & Liang, 2021; Lester et al., 2021; Hu et al., 2021). Among these, Low-Rank Adap- tation (LoRA) (Hu et al., 2021) is prominent method that introduces low-rank adaptation matrices to simulate gradi- ent updates while keeping the pre-trained model weights frozen. LoRA achieves the performance comparable to full- parameter fine-tuning across a range of downstream tasks while requiring significantly fewer computational resources.
1. Introduction
Large language models (LLMs), trained extensive cor- pora, have exhibited impressive performance across wide
However, directly applying LoRA for fine-tuning agents often yields performance that is significantly inferior to that achieved through full paraameer fine-tuning, Successfully accomplishing agent tasks typically requires LLMs to si- multaneously exhibit multiple capabilities. For instance, LLMs must first comprehend the user's query and perform a reasonable analysis and planning, demonstrating the ability of reasoning. Subsequently, it needs to invoke the correct
'Equal contribution 'School of Artificial Intelligence, Beijing University Posts and Telccommunications HHuawei Noah's 's Ark Lab. *Correspondence to: Ying Nie <ying nic @ hom>>
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Rolss
want to know the weather in Bali tomorrow.
User
The weather in Bali tomorrow will rainy, the temperature is 86*F, and the quality is excellent.
Reasoner
Thought: I will call get, weather to get the weather information, and the input parameters are Bali and tomorrow. Next Role: Executor
Available A Pls: book fight (city, date), get weather (cily, date), check stock price (company),
Summarizer
Executor
R Reasoner
Thought:
The API has been
bservation
PII: get weather
successfully executed, the observation can fully resolve the user query. Next Role: Summarizer
Rain, 86
Params: (Bali, lomorrow)
Figure 1. Workflow example of multiple roles collaborate to accomplish one agent task.
functions with suitable parameters, reflecting the ability of execution. After multiple rounds of interaction with the real environment, it should organize the conversation history and provide feedback to the user, exhibiting the ability of sum- marization. Learning multiple capabilities simultancously is challenging for the parameter matrix, particularly when constrained a low-rank form.
nal loss between LoRAs are further introduced for better optimization.
To effectively fine-tune the framework, we develop multi- role data generation pipeline based on publicly available datasets, incorporating role-specific content completion and reliability verification. Specifically, we prompt GPT to fill in missing multi-role content in agent data, such thought and summary. The completed execution trajectories are further evaluated for quality by DeepSeek-V V3 (Liu et al., 2024a), with low-quality samples being filtered out. Subse- quently, we unify the data format to facilitate downstream fine-tuning. In practice, we observe certain issues in the outputs of executors within some samples, such as selecting functions outside the candidate list, encountering runtime errors, or failing to resolve user problems despite error-free execution. To address these issues, we adopt hybrid ap- proach combining manual corrections and LLM to further enhance its reliability.
In this paper, we first decompose the capabilities necessary for the agent into three distinct roles: reasoner, executor, and summarizer. Specifically, the reasoner is responsible for comprehending the user's query and determining the next role based on the execution trajectory. The exccutor tasked with identifying the appropriate functions and parameters invoke, informed by the analysis of reasoner. The summarizer organizes the conversations history and conveys the distilled information back to the user. The pipeline of the multi-roles is illustrated in Figure 1. should be noted that the summarizer is engaged to provide user feedback only when the reasoner deems the user's query fully addressed or, after multiple invocations the executor, still cannot resolve the user's query and decides to give up.
We extensively evaluate the proposed method on various benchmarks, including StolleTool (Guo et al., 2024), BFCL (Yan al., 2024), GSM8K (Cobbe et al., 2021), and MATH (Hendryeks al., 2021). For example, on Stable- ToolBench, our method achieves improvements in DFS pass rates of 40.6% and 14.2% for Ilama3.2 1-1 IIIttrutct (Tou- vron et al., 2023) and Thi-3.5-mini-Instruce (Abdin et al., 2024), respectively, while introducing only an additional 0. 16B and 0.36B trainable parameters.
To better characterize these capabilities, we then propose the Mfxture-of-Roles (MoR) architecture, which comprises three specialized LoRA groups, each designated to fulfill a distinct role. By focusing on their respective specialized capabilities and engaging in collaborative interactions, these groups collectively accomplish the overall agent task. In practice, each group consists of different number of LoRA modules. Our guiding principle is allocate more LoRAs to relatively important roles and fewer to less important ones, aiming to achieve better performance with fewer trainable parameters. Additionally, we introduce both a rule-based role-aware gate and learnable token-ane awkee routers to more reasonably allocate LoRAs to the input features. During the training process, auxiliary balance loss and orthogo-
2. Related Work
2. Agent Tuning
Fine-tuning open source LLMs on agent-specific data has shown to be an effective approach for developing agent ca- pabilities. TooILLM (Qin et al., 2023) presents a general
2

---

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
tool-use framework encompassing data construction, model training. and evaluation. 'he automatically constructed instruction-tuning dataset for tool invocation using Chat- GPT has been widely adopted. Agen'Tuning (Zeng et al., 2023) focuses on improving the agent capabilities of L.LMs themselves without compromising their general abilities. To achieve this, they employ a hybrid instruction-tuning strategy by combining AgentInstrict with instructions from general domains. Gorilla (Patil et al., 2023) constructs the APIBench, a large corpus of APIs with complex and often overlapping functionalitty by scraping MI. APIs from public model hubs. Toolformer (Schick et al., 2023) incorporates a range of tools, including a calculator, a Q&A system, a search engine, a translation system, and calendar. KwaiA- gents (Pan et al., 2023) and ModelScope-Agent (Li al., 2023) presents a comprehensive framework spanning over tool-use data collection, tool retrieval, customized model training, and evaluation for practical real-world applications. Many works have also been proposed to synthesize higher quality data for agent tasks. APIGen (Liu et al., 2024d) in- troduces a function-call canction-calling data generation pipeline to facil- itate the fine-tuning of dunetion-calling L.LMs by providing high-quality, diverse datasets that better reflect the variabil- ity and complexity of real-world API use. xLAM (Zhang et al., 2024) and ToolACE (Liu et al., 2024c) further discuss the data pipeline including data unification, augmentation, quality verification, general instruction data synthesis, and preference data generation.
for fine-tuning, specificall employing LoRA for directional updates. QLoRA (Dettmers al., 2024) further reduces memory use without sacrificing performance with 4-bit Nor- malFloat, double quantization and paged optimizers.
To further improve the accuracy. researchers recently pay more attention to using multiple LoRA combinations. Lo- RAHub (Huang et al., 2023) and MoA (Feng et al., 2024) pioneer the approach of training several LoRA weights on downstream tasks and then integrating the LoRA modules into shared LLM using a routing mechanism. MoLE (Wu et al., 2024) treats each layer of trained LoRAs as a dis- tinct expert and implements hierarchical weight control by integrating a learnable gating function to learn optimal com- position weights. OCTAVIUS (Chen et al., 2023) design multimodal LoRA-MoE decoder for modality learning. MoLA (Gao al., 2024) introduces method of layer- wise expert allocation, where each model layer has the flexibility to employ a varying number of LoRA experts. However, these studies overlook the characteristics of agent tasks, making them challenging to apply in practice.
3. Method
In this section, we first introduce the decomposition of capa- bilities in agent tasks, then we elaborate the framework of mixture-of- and the objectives adopted in fine-tuming mee-tuning, The pipeline of preparing multi-role data is discussed at the end.
In addition to single model, multiple models working to- gether to complete agent tasks has recently attracted the interest of many researchers. Mobile-Agent--2 (Wang et al., 2024) presents multi-agent architecture for mobile device operation assistance, which includes planning agent, deci- sion agent, and reflection agent. a-UMi (Shen et al., 2024a) decomposes the agent capabilities into three roles, each role implemented by a single LLM that focuses on a specific capability and collaborates with others to accomplish the task. However, multiple models pose high requirements computing resources, both for training and inference.
3.1. Capabilities Decomposition
ReAct (Yao et al., 2022) introduces a paradigm of Rea- son+Action for LLM inference, which has gradually be- come dominant in agent tasks (Qin et al., 2023; Gou et al., 2023; Shen et al., 2024a; Lu al., 2024; Wang et al., 2024). Given system prompt p. user query g and previous execution trajectory т, the LLM with weights W outputs thought T, and action A, in sequence at t-th step:
Te, = (p, q, TT-);;
2.2. Parameter Efficient Fine-tuning
where T--1 = (Ti, O1, .... Tr-1, Ac-1, 0--1) repre- sents the execution trajectory before 1-th step. Action con- sists of two parts: function name and the corresponding input parameters. O2 indicates the observation of the action As when invoked in a real deployment environment, Learn- ing multiple capabilities simultaneously is challenging for the weights of LLM, especially for low-rank forms. There- fore, we first propose to decompose the agent capability into three roles.
Parameter-efffiientt fine-tuning (PEFT) has emerged as crucial technique for adapting large pre-trained models to downstream tasks while minimizing computational costs and preserving model performance. LoRA (Hu et al., 2021) freezes the pretrained model weights and injects trainable rank decomposition matrices into each layer of the Trans- former architecture, greatly reducing the number of train- able parameters. LoRA+ (Hayou et al., 2024) corrects the suboptimality of LoRA by setting different learning rates for the LoRA adapter matrices with a well-chosen fixed ratio. DoRA (Liu et al., 2024b) decomposes the pre-trained weight into two components i.e., magnitude and direction,
The reasoner starts to understand the user's 's query and generate its analytical reasoning, which is then passed to the executor. Additionally, based on the obser- vation returned after invoking tools the real deployment
Reasoner:
3

---

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
Transformer Block X
Mixture of Roles (MoR)
Zero
Shared LoRA
Output Hidden h
Routed LoRA
Feed-Forward Network with MoR
X
X
X
X
X
X3
Ab
Ah
4h
Reasoner
Executor
Summarizer
Normalization
Token-Aware Router-R
Token-Aware
Taken- Auvar
Attention with MoR
Router-E
Router
Top-Kr
Top-K.
Top-K.
Role-Awer Aware Gate
Normalization
Pretrained
O0O
Weights
O 000 000
Hidden ho
Wo
Input Hidden u
Figure 2. The framework of our method. The capabilities necessary for agent are decomposed into three distinct roles: reasoner, executor, and summarizer. Each role consists of different number of LoRAs according their learning difficulty. The rule-based role-aware gate and learnable token-aware routers are introduced to more reasonably allocate LoRAs.
environment, the reasoner analyzes the execution trajectory determine whether the user's i query has been addressed. If the user query is solved, or if it cannot be solved after enough attempts, the next role is transferred to summarizer. Otherwise, is returned to the executor for further action. Formally,
3.2. Mixture-of-Roles
To characterize the decomposed capabilities, we then pro- pose the framework of Mixture-of-Roles (MoR) as illus- trated in Figure 2.
Forward Procedure. The MoR framework can be equipped on the linear layer of attention or feed-forward network in each transformer block. While freezing the pretrained weights in the backbone, we fine-tune only the parameters in MoR, including LoRAs and routers. Given in- put hidden Rleenxd and the frozen pretrained weights Wo Rd where len is the sequence length of input, dj and de denote the dimension of input and output, re- spectively. The final output hidden h can be calculated by:
Tr, Role, (Pr:
where Role; denotes the activated role in the next step, W, and p. denote the weights of reasoner and the system prompt of reasoner, respectively.
The exccutor is responsible for selecting the appropriate functions and parameters to invoke, based on the analytical thought of reasoner.
Executor:
(5) where hg W t and Ah = Ah, + Ah, + Ah iee. the sum of the outputs of reasoner, executor and summarizer, should be noted that there is only one role that non-zero, and the other two all zeros. Formally, Vi (0. (0...., len 1}:
h = ho + Ah,
Funs, Param, We(Pes Tt,1, Tt-1),
(3)
where Fun, and Param, represent the functions and pa- rameters in the next step, We and ps denote the weights and the system prompt of executor. Then, the functions executed in the real deployment environment, such RapidAPI Hub (Qin et al., 2023), Python IDE, etc.
HA # HAN + #0} + 1(ah 0 -. (6)
The summarizer needs to re-organize the conversations history and convey the distilled information Sum back to the user.
Summarizer:
That is, uli,:] is processed by only one active role, which is achieved through the rule-based role-aware gate, Specifi- cally, when user inputs query, the reasoner is activated first. The next role to be activated is determined based on the output of the reasoner, i.e., Role, in Equation. 2. Be- sides, the observations O, in Equation. always input to the reasoner to decide whether to continue execution or summarize.
Sum W. (Ps, q.T.
(4)
where W. and Ps denote the weights and the system prompt of summarizer, respectively.

---

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
Without loss of generality, taking the output of reasoner as an example: Ah, AW, u,,, where AW, denotes the LoRAs' weights of reasoner, u, denotes the input that is allocated to reasnoer.
Algorithm Fine-tuning LLM with MoR for agent tasks. input A pre-trained LLM M, data D and iterations for fine-tuning,
d1 and dz in loss, the mod- ules need fine-tuning in M, the rank value d3, the number of LoRAs E in each role. repeat
AW, B,AA Σ (u,)
(7)
1. Randomly select a batch of data from D.
2. Conduct the forward process for M accompanied
Each role consists of shared LoRA (superseript 0) and number of E, routed LoRAs (superscript from 1 to E,). Bº, B E R xds, A.: A E Rdaxda, and the rank d3 & min(d1, d2). R. denotes the Top-K token-aware router in reasoner to select the specific LoRAs for different input.
with MoR by Equation 5.
3. Compute the loss function by Equation 8. Freeze all parameters in M, update the parameters of LoRAs and routers.
until iterations T.
Training Objective. We adopt the following objective to guide the processs of supervised fine-tuning:
output A fine-tuned LLM with MoR for agent tasks.
Cimal= Lce Cuux + 0:2 Coth.
(8)
where F represents the frobenius norm. Through appro- priate hyper-parameters to combine different losses, better results can be achieved. The complete fine-tuning process is illustrated in Algorithm 1.
where LCE is the cross-entropy loss, which measures the dif- ference of distribution between predication and truth. Laux is the auxiliary balancing loss to avoid the load imbalance among LoRAs fort is the orthogonal loss between Lo- RAs, which makes the LoRAs as independent as possible to capture features in different directions. on and 02 are nyper-parameters.
3.3. Data Preparation
To effectively fine-tune the framework, we develop multi- role data generation pipeline based on publicly available datasets, incorporating role-specific content completion and reliability verification.
B Σ
CCB
log Pw (1 (v'lz',,
(9)
Role-Specific Content Completion. We adopt the pub- licly available datasets including ToolBench (Qin al., 2023), the combination of APIGen (Liu et al., 2024d) and ToolACE (Liu et al., 2024c) and
where is the batch size, y is the label for input u, and Pw indicates the distribution of predication with weights w. The auxiliary balancing loss is inherited from Switch transformers (Fedus et al., 2022), VV E (r,
MathGenie (Lu et al., 2024) to fine-tune the correspond- ing downstream agent tasks respectively. For ToolBench, which contains implicit multi-role content, we can directly convert it into an explicit multi-role format using rule-based methods. However, many samples in other datasets lack multi-rolc content, such thought and summary. For these samples, we prompt GPT4o to complete them. The detailed prompts are presented in Figure 6 and Figure 7. In practice, we observe many execution trajectories which the rea- soning or summarizing steps are of low quality, a common issue in the outputs of LLM. To address this, we prompt DeepSeek- V3 (Liu al., 2024a) to evaluate both the overall trajectory and the individual reasoning summarization steps.
Laux Eig
Σ fo P
(10)
where fú represents the fraction of tokens dispatched to i-th LoRA reasoner, executor and summarizer,
f
Σ EB
1(Ro(ug)' + 0}
(11)
B
and P denotes the average fraction of the router probability allocated for i-th LoRA in role V:
P
Σ B
Softmax (ug W
(12)
B
where W denotes the weights of router in different roles. In addition, to encourage LoRAs to learn distributions in different directions and reduce redundancy, we propose the orthogonal loss.
Further, we unify the data into JSON format to facilitate downstream fine-tuning. Specifically, the JSON begins with a list of candidate functions, available. Each function is characterized by its name, description, parameters, and whether it is marked as required. This is followed by the system prompt and detailed execution trajectories. The ex-
Er
Eg Σ (14:TAB|1 j=141
B B& B 1
Lonth
Σ
http:///wngnnggieeeeddddaeeessip calling-v2
(13)
5

---

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
Function Call
Malh
StableToolBench comprises a total of 765 questions span- ning across 6 subtasks, evolving from ToolBench (Qin et al., 2023). It introduces virtual API server and stable eval- uation system. BFCL presentes the first comprehensive evaluation on the LLMs' ability to call functions and tools. We report the results on Abstract Syntax Tree (AST) of live and non-live, Executable Function Evaluation (Exec) of non- live, Relevance of live, containing a total of 2797 questions. GSM8K and MATH are designed to be evaluated in the area of mathematics, which contain 1319 and 5000 questions, respectively. Different from directly generating the answer, we solve the math problems by importing python package and then executing it.
Function:
System Conversation
Tunetion name"
desctption: "uactinn description!
peroratse (ape descriitions
"sserquary" quary'
paramnttens]
functions) ..)
neearner
System prompt:
content: thought and rote
Conversation
esseater
content: 'sser queey""
content: "yython code'
sole: neseoner
zle: abservation"
outter: Shought and aest nod
cuttent 'resllt of oode exsoutionn
[aaree conversccons) .
sole: "uscuttr"
ootton:: "anctonn romes and paranttors
cole: summartaa"
content: Summazte and tood book user'
sele: obsorvationn
outtem: 'esalt of functon euscution
During fine-tuning, we choose the corresponding datasets for fine-tuning in their respective downstream tasks. Specif- ically, for StableToolBeech, we sample 120k multi-role execution trajectories from the training set of toolbench. For BFCL, we sample 90k items from the combination of APIGen (Liu et al., 2024d) and ToolACE (Liu et al., 2024c) and glaive-function www.ffioonncallnn--eeee we adopte the 80k execution trajectories MathGGene (Lu et al., 2024). All datasets are filtered and corrected by our multi-role content completion and reliability verification. The detailed prompts for system and roles in our data are illustrated in Figure 8, Figure 9 and Figure 10.
[more consenstion) ...
sele: "ummacttee
cottom:: Sammanize and food bodk to use!
Figure 3. The JSON format on different scenarios in our fine- tuning datasets.
For GSM8K and MATH,
ecution trajectories starts with the user's query and proceeds with the interaction between the reasoner and the execu- tor. If necessary, this includes execution results from the real deployment environment, i.e., observations. Finally, it concludes with the feedback from summarizer. The JSON formats for different scenarios are shown in Figure 3.
Reliability Verification. Although the original publicly available datasets are carefully reviewed, we observe in practice that some samples contain wrong outputs from the executor. To enhance the effectiveness of fine-tuning data, we conduct a series of filtering and correction procedures. The errors can be categorized into the following lypes:1) Function not included in the candidate list. This issue can be identified using rule-based methods and corrected by re- prompting other L.M. 2) Incorrect numbers of parameters and type mismatches. These can be detected through execu- tion failures and subsequently corrected manually. 3) Errors in function selection or parameter assignment. These cases more subtle, it does not trigger execution failures. We examine them by comparing the outputs from prompt- ing other LLM with the execution results. Samples with inconsistencies are then manually reviewed and corrected.
4.2. Implementation Details
0.50
Rewesneer
Coscuiie
0:45-
Summar
0.00
0.40-
0.55
S gas
0.50
0.30
0.15
0.10
0.35
Number of LoBAs
Number of LURAs
Figure 4. The loss and levenshtein accuracy of respective roles with different number of LoRAs
To determine the optimal number of LoRAs per role, aim- ing balance performance and the number of trainable parameters, we conduct toy experiments using the specific capability dataset of each role. The dataset is constructed by first combining all multi-rolc data and then splitting it based on roles. For each role, 80k samples are randomly selected as the training set, while 5k samples are sampled as the validation set. We set the dimension d3 in Equation 7 to 16 and vary the number of LoRAs for each role from I to 8. The modules including query, key, value, out, gate, up, and down are selected as target modules for fine-tuning with MoR. Additionally, routers are removed for simplicity. The loss and levenshtein accuracy after epoch fine-tuning based on Llama3,22 2- -B-Innttuct are shown in Figure 4. We can observe that the trend of loss and accuracy changes
4. Experiments
4.1. Models and Datasets
We conduct experiments on multiple pre-trained LLMs, including 1--aa..-----Inttttet (Touvron et al., 2023), Phi3.5-mini-innstruc (Abdin et al., 2024) and code-specific Qwen2.5-1.5B-Coo (Hui et al., 2024). For evaluations, the benchmarks including
(Guo et al., 2024), BFCL (Yan et al., 2024), GSMSK (Cobbe et al., 2021) and MATH (Hendrycks et al., 2021) are evaluated. Specifically,
6

---

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
Table I. The pass rate and win rate of different LLMs on Brolechh....nch.
I1-Inst
I1-Tool
I1-Cat
12-Inst
12-Cat
I3-Inst
AVG
Model
Setting
Pass
Win
Pass
Win
Pass
Win
Pass
Win
Pass
Win
Pass
Win
Pass
Win
CoT
52.8
48.4
51.9
40.3
56.6
54.2
52.8
36.6
51.9
47.6
52.5
45.2
53.1
45,4
GPT4
DFS
59.2
50.1
65.7
44.2
61.7
57.9
55.2
41.2
55.6
53.7
66.1
45.4
60.6
48.8
TooALLaMA-
CoT
518
39.2
46.4
33.1
53.1
39.9
48,9
31.3
51.6
40.7
37.2
42.2
48.2
37.7
v2-7B
DFS
61.0
43.6
45.6
35.8
58.8
41.3
53.5
34.2
60.3
43.8
48.1
45.1
54.6
40.6
Base Model:
Llama33.-
(1.24B)
CoT
14.4
23.3
10.1
15.8
12.2
15.0
3.9
11.3
5.9
11.3
8.2
13.1
9.1
15.0
Basc
DFS
11.7
20.8
11.3
17.1
14.6
18.3
8.5
9.4
2.3
12.9
11.5
11.5
10.0
15.0
MoRAgent-Llama
CoT
54.9
40.5
58.2
38.0
53.1
39.9
38.1
34.0
42.6
41.1
47.9
29.5
49.1
(+40..0)
37.2 (+22.2)
(1.24B+0.166) 16B)
DFS
54.6
44.8
45.5
32.9
53.2
40.5
46.8
36.8
68.2
46.8
58.8
39.4
54.5 (+44.5)
40.2 (+25.2)
Base Model:
Phi3.
1ii...5minni-mg Instruct (3.828)
CoT
29.7
30.7
37.9
30.4
36
34.6
20.9
17.0
26.7
25.0
17.8
15.8
28.2
25.6
Base
DFS
46.4
35.0
50.6
30.4
51.1
49.0
39.9
30.2
41.5
27.4
37.6
32.8
44.5
34,1
MoRAgent-Phi
CoT
51.8
47.2
54.4
38.6
55.4
51.0
48.1
33.0
55.0
45.2
62.3
42.6
54.5
(+26.3)
42.9 (+17.3)
(3.82B+0.36B)
DFS
55.9
48.5
56.6
41.1
60.9
47.1
55.4
44.3
59.7
48.4
63.6
50.8
58.7 (+14.2)
46.7 (+12.6)
slows down significantly when the number of LoRAs for both the reasoner and the executor is changed from to 5. While for summarizer, it occurs at 3 to 4. Accordingly, in subsequent experiments,
Table 2. The performance of different LLMs on BFCL.
Non-live
Live Relevance
Model
AVG
AST
Exec
AST
set the number of activated LoRAs to 4 and the total number of LoRAs to 5 for reasoner and executor, and set the number of activated LoRAs to 3 and the total number of LoRAs to 4 for summarizer. This is also in line with our perception that the summarizer's 's primary task is to convey the distilled information from the historical conversation to the user, making its learning pro- cess relatively less challenging. Also, we set the learning rate to 5e-5, with 4 epochs of fine-tuning by MoR, and (1 and 02 in Equation 8 set to 1e-3 and le-4, respectively.
Qwen2.5-72B-ir
90.8
92.7
75.3
100.0
89.7
GPT4
88.1
89.4
79.8
83.3
85.2
ToolACE-8B
87,5
89.2
78.5
83,3
84.6
MiniCPM3-4B
80.8
87.6
70.0
72.2
77.7
Llama3.2-3BB.
80,6
83.7
55.8
88.9
77.3
Base Model: Llama3.2 2- 18-Insttrett (1,248)
Base
21.9
19.2
29.8
38.9
27.5
MoRAgent-L. Jama
75.2
80.0
60.7
94.4
77.6 (+50.1)
(1.24B+0.16B)
Base Model:
1ii...5mmni--sttee (3.828)
Base
69.0
57.3
58.2
72.2
64.2
MoRAgent-Phi
83.0
78.2
72.7
94.4
82.1 (+17.9)
(3.22++0.663)
4.3. Experimental Results
evance of live. We also adopt Llama3.2. 1-- IIstrrcct and Phi3.5-mini-Instruet as our base models. As reported in Table 2, with only introduced 0. 06B and 0.36B trainable parameters, we improve the average accuracy of Llama3.2 2- IB- Instructtt and 1hi3.5-mini-Insteeee by 50.1% and 17.9%. respectively.
Results on StableToolBench. We first evaluate the pro- posed method with various LLMs on Boclechho (Guo et al., 2024). Specifically, we report the solvable pass rate and solvable win rate based on the solvable tasks with two settings including Chain-of- Thaouhh (CoT) and Depth-First- Search (DFS). For win rate, we run all models once against GPT3.5-1106+CO0 and evaluate them three times. We re- spectively adopt two LLMs including Llama3.2- IB-Instruct as our base models. As reported
Table 3. The performance of different LLMs on GSM8K and MATH.
Model
GSMSK
MATH
and Phi3.5-mini- in Table 1, for Llama3.2-11 B--ItBInstttutt, with only introduced 0.16B trainable parameters, we improve the pass rate and win rate by 44.5% and 25.2% on DFS setting respectively. Also, for Phi3.5-mini-Instruct with only introduced 0.36B trainable parameters, we improve the pass rate and win rate by 26.3% and 17.3% CoT setting respectively. With fewer parameters, our MoRAgenn-PPi even surpasses the agent-specific model TooLLLaMA-v2, which validates the effectiveness of our method.
LIama-3.1-80 --B-it
54.8
21.4
Alama-3.2-113 IB-it
52.6
29.2
DeepScekMath-7B
64.2
36,2
Base Model: 1.ee2..--15B--ooll (I.54B)
Base (w/ (
54.7
23.5
Base (w/o packages)
41.7
33.5
MoRAgent-Qwen (1.54B+0.27B)
68.5 (+13.8)
45.5 (+12.0)
Results on GSM8K and MATH. We also extend our ap- proach to the mathematical scenarios. Different from di- rectly generating the answer, we solve the math problems by importing python package and then executing it. We use 1-e.e...-1.5--ooo
Results on BFCL leaderboard. We then evaluate our method on BFCL leaderboard. Specifically, We report the results on Abstract Syntax Tree (AST) of live and non-live, Executable Function Evaluation (Exec) of non-live, Rel-
as the base model and the results are reported in Table 3. Specifically, we adpot the frame- work of ToRA (Gou et al., 2023) to infer and evaluate. For
7

---

Loss Functions. We first analyze each loss functions in MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Role
Table 4. Analysis on each loss functions.
Table 5. Analysis on the number of LoRAs.
loss function
Total LoRAs
AVG Acc
Trainable Params
AVG
cross-entropy
balance
orthogonal
reasoner
executor
summarizer
Acc
72.3
4
0.13B
73.2
75.1
5
5
0.16B
77.6
V
74.4
6
6
5
0.19B
79.1
/
77.6
7
7
6
0.23B
80.0
Table 6. The accuracy of different fine-tuning methods.
the Qwen2.5-1.55 1-..5--11---oeeee base model, we evaluate both the methods of with and without python packages. From the results, with only introduced 0.27B trainable parameters, compared with the best accuracy of the two baseline meth- ods, we improve by 13.8% and 12.0% on GSM8K and MATH respectively.
Trainable Params
Non-live
Live
AVG
Method
AST
Exec
AST
Rele
Acc
Base
21.9
19.2
29.8
38.9
27.5
LoRA
0,16B
59.7
64.2
56.3
81.8
65.5 (+38.0)
DoRA
0.16B
61.2
65.7
58.4
82.0
66.8 (+39.3)
SFT
1.24B
72.3
77.6
61.5
92.6
76.0 (+48.5)
Ours
0.16B
75.2
80.0
60.7
94.4
77.6 (450.1)
4.4. Ablation Studies
increasing the number of LoRAs will generally increase the performance on agent task, but will undoubtedly also increase the trainable parameters. For example, when the trainable parameters decreases from 0.16B to 0.13B, the performance decreases from 77.6% to 73.2%. When the trainable parameters increases from 0. 0616B to 0.19B, the per- formance only increases by 1.5%, and further increasing the parameters does not bring significant improvement in per- formance. In order achieve a balance between trainable set the number of LoRAs
We conduct series of ablation studies in this sub-section, the Llamm3.2-11 1------nstttuttt is adopted as the base model, and the results on BFCL leaderboard are repoted.
Equation 8, the results are reported in Table 4. For the loss of balance and orthogonal, the introduction of each loss can bring about an improvement in average accuracy. The combination of them can bring 5.3% improvement. We visualize the similarity of routed LoRAs without and with orthogonal loss in Figure 5. The multiplied weights B and A the query module at layer 7 are used for visualization. With the introduction of orthogonal loss, we encourage different LoRAs to learn features in different directions, therehy reducing parameter redundancy and improving the performance of downstream tasks.
parameters and performance, to 5, 5, and 4 for reasoner, executor, and summarizers in our experiments, respectively.
Comparisons with other methods. We compare the pro- posed method with other methods including parameter- efficient and full-parameter fine-tuning with the same multi- roles dataset in Table 6. From the results, SFT exhibits superior accuracy compared PEFT methods (LoRA and DoRA), which can be attributed to its more trainable pa- rameters, achieving an average accuracy 10.5% higher than LoRA. Notably, DoRA (Liu et al., 2024b) introduces an advanced scheme by decomposing pretrained weight matri- ces into magnitude vectors (m) and directional matrices (V), where LoRA is applied specifically to V while is trained separately. This architectural innovation allows DoRA to surpass LoRA slightly in accuracy. Crucially, our method achieves statistically significant performance improvements through two key innovations: 1) a more rational capacity decomposition strategy, and 2) a novel Mixture-of of-RRoes framework enabling dynamic interaction between decom- posed modules. These enhancements collectively contribute to method's marked accuracy superiority over SFT and PEFT methods.
XXV
RD
Late
AAT
AM0
R2F
SAA
430
411
R000
4:01
4.06
Figure 5. Visualization of the similarity of routed LoRAs without (top) and with (bottom) orthogonal loss. From left to right are reasoner, executor and summarizer.
Number of fine-tuning samples. In order to analyze the impact of various numbers of fine-tuning samples on accu- racy, we conduct corresponding experiments here, and the results are shown in Table 7. Even with only 1 training
Number of LoRAs. We also explore the influence of dif- ferent number of LoRAs for various roles. In practice, we leave only one LoRA inactive by default. From Table 5,
8

---

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Rolss
Table 7. The accuracy of different number of fine-tuning samples.
Sheng, L.,, Ouyang, W., Qiao, Y., and Shao, J. Octavius: Mitigating task interference in milms via moe. arXiv preprint arXiv:2311.02684 3:8, 2023.
Non-live
Live
AVG
#Samples
AST
Exec
AST
Rele
Acc
0
21.9
19.2
29.8
38.9
27.5
Chen, Z., Liu, K., Wang. Q.. Zhang, W., Liu, J., Lin, D., Chen, K., and Zhao, F. Agent-flant Designing data and methods of effective agent tuning for large language mod- els. arXiv preprint arXiv:2403. 10000..2880 /2881, 2024.
1K
49.5
44.6
46.3
79.1
54.9 (+27.44
5K
55.3
51.8
50.6
82.1
60.0 (+32.5)
10K
58.8
57.4
52.7
85.9
63.7 (+36.2)
50K
70.4
74.9
56.5
91.7
73.4 (+45.9)
90K
75.2
80.0
60.7
94.4
77.6 (+50.1)
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M.. Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 1 10. 14168, 2021.
samples, we still achieves a 27.4% improvement in average accuracy, demonstrating its well generalization capability. As the fine-tuning data volume increases, the accuracy fur- ther improves accordingly.
Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, Qlora: Efficient finetuning of quantized 1lms. Advances in Neural Information Processing Systems. 36, 2024.
5. Conclusion
To improve the efficiency of applying PEFT to agent, in this paper, we introduce three strategies: 1) Following the increasingly dominant Reason+Action paradigm, we first decompose the capabilities necessary for the agent tasks into three distinet roles: reasoner, executor, and summarizer. 2) We then propose the Mixture-of- -fffolee-f--Roes (MoR) framework, which comprises three specialized LoRA groups, each desig- nated to fulfill distinct role. By focusing on their respective specialized capabilities and engaging in collaborative inter- actions, these LoRAs collectively accomplish the overall agent task. Additionally, we introduce both rule-based role-aware gate and learnable token-awes awkare routers more reasonably allocate LoRAs to the input features. During the training process, auxiliary balance loss and orthogonal loss between LoRAs are further introduced for better opti- mization. 3) We also develop multi-role data generation pipeline based on publicly available datasets to effectively fine-tune the framework. We conduct extensive experiments and thorough ablation studies on various L.MSs and agent benchmarks, demonstrating the effectiveness of our method.
Fedus, W. Zoph, B., and Shazeer, N. Switch transformers: Scaling to trillion parameter models with simple and ef- ficient sparsity. Journal of Machine Learning Research, 23(120):1-39, 2022.
Feng. W., Hao, C., Zhang, Y., Han, Y., and Wang, H. Mixture-of-lorad An efficient multitask tuning for large language models. arXiv preprint arXiv:2403.0334 03302, 2024.
Gao, C., Chen, K., Rao, J., Sun, B., Liu, R., Peng, D., Zhang, Y., Guo, X., Yang, J., and Subrahmanian, V. Higher layers need more lora experts. arXiv preprint arXiv:2402.08562 2024.
Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Huang, M., Duan, N., and Chen, W. Tora: A tool-integrated reasoning agent for mathematical problem solving. arXiv preprint arXiv:2309.1744 2023.
Guo, Z.,, Cheng, S., Wang. H.,, Liang, S., Qin, Y., Li, P., Liu, Z., Sun, M., and Liu, Y. Stabletoolbench: Towards stable large-scale benchmarking
Impact Statement
tool learning of large lan- guage models. arXiv preprint arXiv:2403.00 007711704..... 2024.
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Hayou, S., Ghosh, N., and Yu, B. Lora+: Efficient low rank adaptation of large models. arXiv preprint 40X 2024.
Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. Measuring math- ematical problem solving with the math dataset. arXiv 2021.
References
Abdin, M., Aneja, J., Awadalla, H., Awadallah, A., Awan, A. A., Bach, N., Bahree, A., Bakhtiari, A., Bao, J., Behl, H., et al. Phi-3 technical report: A highly capable lan- guage model locally on your phone. arXiv preprint arXiv:2404.14211 2024.
preprint arXiv:2
Hong, S., Zheng, X., Chen, J., Cheng, Y., Wang. J., Zhang, C., Wang, Z., Yau, S. K. S., Lin, Z., Zhou, L., et al. Metagpt: Meta programming for multi-agent collabora- tive framework, arXiv preprint arXiv:2308.0 00352, 2023.
Chen, Z., Wang, Z., Wang, Z., Liu, H., Yin, Z.,, Liu, S.,

---

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Rolss
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. Parameter-efficient transfer learning for nlp. In International conference machine learning, pp. 2790- 2799. PMLR, 2019.
with question back- for enhancing mathemati- cal reasoning of llms. arXiv preprint 24..22602..352 2024.
Pan, H., Zhai, Z.., Yuan, H., Lv, Y., Fu, R., Liu, M., Wang, Z., and Qin, B. Kwaiagents: Generalized information- seeking agent system with large language models. arXiv preprini arXiv:2312.04888 2023.
Hu, J., Shen, Y. Wallis, P., Allen-Zhu, Z., Li, Y., Wang,
S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:21066 10.009655 2021.
Patil, S. G., Zhang, T., Wang. X., and Gonzalez, J. E. Gorilla: Large language model connected with massive apis. arXiv preprint arXiv:23355 2023.
Huang, C., Liu, Q., Lin, B. Y., Pang. T., Du, C., and Lin, M. Lorahub: Efficient cross-task generalization via dynamic lora composition. arXiv preprint arXiv:2307.13266. 2023.
Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.1678) 2023.
Hui, B., Yang, J., Cui, Z., Yang, J., Liu, D., Zhang, L., Liu, T., Zhang, J., Yu, B., Lu, K., et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409. 12186, 2024.
Schick, T., Dwivedi- Yu, J., Dessi, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L.., Cancedda, N., and Scialom, T. Toolformer: Language models can teach themselves to tools. Advances in Neural Information Processing Systems, (1466599-65555 2023.
Lester, B., Al-Rfou, R., and Constant, N. The power of scale for paraieter-efffcient prompt tuning. arXiv preprint arXiv:2104.08691, 2021.
Li, C., Chen, H., Yan, M., Shen, W., Xu, H., Wu, Z., Zhang, Z., Zhou, W., Chen, Y., Cheng, C., al. Modelscope- agent: Building your customizable agent system with open-source large language models. arXiv preprint arXiv:2309.00988. 2023.
Shen, W., Li, C.,, Chen, H., Yan, M., Quan, X., Chen, H., Zhang. J., and Huang, F. Small llms are weak tool learn- ers: A multi-llm IIm agent. arXiv preprint arXiv:2401.07324, 2024a.
Shen, Y., Song, K., Tan, X., Li, D., Lu, W., and Zhuang, Y. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Prucessing Systems, 36, 2024b.
Li, X. L. and Liang, P. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00199 2021.
Liu, A., Feng, B., Xue, B., Wang. B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al. Deepseek- v3 technical report. arXiv preprint arXiv:2412.19437, 2024a.
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation lan- guage models. arXiv preprint arXiv:2302. 200.:::23021197777 /3971, 2023.
Liu, S.-Y., Wang, C.-Y., Yin, H., Molchanov, P., Wang, Y.-C. F. Cheng, K.-T,, and Chen, M.-H. Dora: Weight-decomposed low-rank adaptation. arXiv preprint arXiv:2402.09333. 2024b.
Wang. J., Xu, H. Ye, J., Yan, M.. Shen, W., Zhang, J., Huang, F., and Sang, J. Mobile-agent: Autonomous multi- modal mobile device agent with visual perception. arXiv preprint arXiv:2401.166 16158, 2024.
Liu, W., Huang, X., Zeng, X., Hao, X., Yu, S., L., D., Wang, S., Gan, W., Liu, Z., Yu, Y., et al. Toolace: Win- ning the points of Ilm function calling. arXiv preprint wxXiv:240.000920 2024c.
Wu, X., Huang, S., and Wei, F. Mixture of lora experts. arXiv preprint arXiv:2404. 13628, 2024.
Yan, F., Mao, H. Ji, C. C.-J,, Zhang, T., Patil, S. G., Stoica, I., and Gonzalez, J. E. Berkeley function calling leader- board. bl begss/B_berkeeee_furre funct oo on_calling_ leaderboard. html, 2024.
Liu, Z., Hoang, T., Zhang, Zhu, M., Lan, T., Kokane, S., Tan, J., Yao, W., Liu, Z., Feng, Y., et al. Api- gen: Automated pipeline for generating verifiable and diverse function-calling datasets. arXiv preprint arXiv:2406. 185/8, 2024d.
werkeleyy.dde
Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., et al. Qwen2. technical report. arXiv preprint 15Xiv:2412.151155 2024.
Lu, Z., Zhou, A., Ren, H., Wang, K., Shi, W., Pan, J.. Zhan, M., and Li, H. Mathgenie: Generating synthetic data
10

---

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Rod
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, Narasimhan, K. and Cao, Y. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022.
Zeng, A., Liu, M., Lu, R., Wang, B., Liu, X., Dong, Y., and Tang, J. Agenttuning: Enabling generalized agent abilities for Ilms. arXiv preprint arXiv:2310.128233 2023.
Zhang, J., Lan, T., Thu, M., Liu, 7.,, Hoang. T., Kokane, S., Yao, W., Tan, J., Prabhakar, A., Chen, H., et al. xlam: A family of large action models empower agent systems. arXiv preprint
2024.

---

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
A. Detailed Prompts for Completing the Content of Roles.
Your role a reasoner with reasoning capabilities; you have the ability to analyze the current task resolution status based on user queries and conversation history, and decide which role call next. Please note the following:
Reasoner:
1. You do not need directly generate task answers, you only need to analyze and think about the solution approach that can complete the task based the current status.
2. If you believe that the executor has completed the task in the historical conversation, designate the next role as summarizer to provide a summary, and let him output the results the executor.
3. If you believe the executor has not yet completed the task, analyze and output the correct solution approach, and designate the next role as executor to carry out detailed resolution.
Please output in the following format: The solution approach after thoughtful analysis. Next: Choose the next role call, either executor summarizer. Please strictly adhere to the above format for output, and end with either "Next: executor" or "Next: summarizer".
Executor: Your role is executor, and you have the capability to provide specific answers to tasks based on user queries and the analysis provided by the reasoner. If the task involves tool invocation, directly output the correct tool and its parameters in the required format. Please note the following:
1. If you feel that based on the current information, you are not yet able to output the correct answer, designate role as reasoner to continue further reasoning and analysis.
2. Even though you are certain that you have completed the user's task, please designate the next role as reasoner and let him determine further.
Please output in the following format: The function tool called and its parameters (if the user's query is tool invocation task). Next: Choose the next role to call, either reasoner summarizer. Please strictly adhere to the above format for output, and only end with "Next: reasoner".
Your role is summarizer, and you have the ability produce summaries based user queries and the dialogue history. Please strictly adhere to the following rules:
Summarizer:
1. If the task involves tool invocation and there are no actual tool invocation return observation results in the historical conversation, you can examine the correctness of the tools and parameters output by the executor, as well as the output format.
2. If you believe there are errors in the output, correct the output according to the required format.
3. If you believe the output is entirely correct, then directly copy and output the correct answer generated the executor.
4. If the task involves tool invocation and there are actual tool invocation return observation results in the historical conversation, you can provide a concise summary of the tool invocation feedback results in natural language.
5. If the task does not involve tool invocation, you can summarize the information you have received to answer the user's queries.
Please note the format requirements for the output. As you the final step the conversation, there is no need to specify the next role to be called.
Figure 6. Detailed prompts completing content roles function caaeiinncalling scenarios.
B. Detailed Prompts for System and Roles.
C. Visualization of Execution Trajectory.
12

---

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
You're re professional math problem thinking assistant. You can deduce thought process based on given problem and existing code solutions. Please give the thought process based on the above information. Please note that you need to follow the following rules:
Reasoner:
Read and understand the math problems and code solutions above, and give answers.
2. Your thinking process needs to give step-by-step solution the problem, and you need provide formulas and parameters for the code solution, but you don't need to caleulate the results.
3. The first step should be to extract the variables and corresponding values from the problem, and use First, Next, and Finally as transition words between steps.
4. It's best to conclude the thought process by pointing out which parameters lead to the final result we
need.
5. Formulas in the thinking process are best highlighted in parentheses.
6. The thought process you give needs be as concise as possible. Don't 't use mathematical symbols elsewhere to avoid miscalculations.
Figure 7. Detailed prompts for completing the content of roles in mathmatical scenarios.
System:
You have access the following APIs within XML (og::
Your role is reasoner with reasoning capabilities, and you have the ability analyze the current task status based on user queries and conversation history, and decide which role to call next. Please note the following:
Reasoner:
1. You do not need to directly generate task answers, you only need to analyze and think about the solution approach that can complete the task based on the current status.
2. If you believe that the Executor has completed the task in the historical conversation, designate the next role as summarizer to provide a summary.
3. If you believe the executor has not yet completed the task, analyze and output the correct solution approach, and designate the next role as executor to carry out detailed solution.
Please output in the following format: The solution approach after thoughtful analysis. Next: Choose the next role to call, either executor summarizer.
Your role is executor, and you have the capability to provide specific api calling tasks based on user queries and the analysis provided by the reasoner. Please directly output the correct api tool and its parameters with the following format: [unused] I]Action: <function-nane name"> Arguments: <args-dict>> (unusell1] Please strictly adhere to the above format for output.
Executor:
Your role is a summarizer, and you have the ability to produce summaries based on user queries and the dialogue history. Please strictly adhere to the following rules:
Summarizer:
1. If the task involves tool invocation and there are actual tool invocation return observation results in the historical conversation, you can provide a concise summary of the tool invocation feedback results in natural language.
2. If the task does not involve tool invocation, you summarize the information you have received to answer the user's queries.
Figure Detailed prompts of the datasets for fine-tuning in BFCI..
13

---

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
System:
You have access to following APIs within XML tags:
Reasoner: Your role a reasoner with reasoning capabilities; you have the ability to analyze the current task status based on user queries and conversation history, and decide which role to call next. Please note the following:
1. You do not need directly generate task answers; you only need analyze and think about the solution approach that complete the task based the current status.
2. If you believe that the Executor has completed the task in the historical conversation, designate the next role as summarizer to provide a summary.
3. If you believe the exccutor has not yet completed the task, analyze and output the correct solution approach, and designate the next role as executor to carry out detailed solution.
Please output in the following format: The solution approach after thoughtful analysis. Next: Choose the next role call, either executor summarizer. Please strictly adhere to the above format for output, and end with either "Next: executor." or "Next: summarizer.
Your role is executor, and you have the capability to provide specific api calling tasks based on user queries and the analysis provided by the reasoner. Please directly output the correct api tool and its parameters with the following format: [unused] |Action::: <function-name>> Action- <args-dict>> (unused12] Please strictly adhere to the above format for output.
Executor:
Summarizer: Your role is summarizer, and you have the ability to produce summaries based on user queries and the dialogue history. Please strictly adhere to the following rules:
If the task involves tool invocation and there are actual tool invocation return observation results historical conversation, you can provide a concise summary of the tool invocation feedback results in natural language.
2. If the task does not involve tool invocation, you summarize the information you have received to answer the user's queries.
Figure Detailed prompts of the datasets for fine-tuning ToolBench.
System:
You have the capability address the following user's query.
Your role is reasoner with reasoning capabilities; you have the ability analyze the current task status based on user queries and conversation history, and decide which role to call next. Please note the following:
Reasoner:
1. You do not need to directly generate task answers; you only need to analyze and think about the solution approach that can complete the task based on the current status.
2. If you believe that the executor has completed the task in the historical conversation, designate the next role as summarizer provide summary.
3. If you believe the executor has not yet completed the task, analyze and output the correct solution approach, and designate the next role as executor to carry out detailed solution.
Please output in the following format: The solution approach after thoughtful analysis. Next: Choose the next role to call, either executor or summarizer. Please strictly adhere to the above format for output, and end with either "Next: executor." or "Next: summarizer."
Executor: Your role is an executor. You have the capability to generate specific Python code based on the conversation history and the output of the executot to address the user's query. Please directly generate a segment of Python code.
Summarizer: Your role a summarizer, and you have the ability to produce summaries based on user queries and the dialogue history.
Figure 10. Detailed prompts of the datasets for fine-tuning in Math.
14

---

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Ran Roles
I'm planning small outdoor event in Ottawa, and need make sure the weather is going to cooperate. Could you fetch the current weather for latitude 45.4215 and longitude -75.6972 using the Open-Meteo API? Also, I'm running small game at the event, and I'm curious about the chances of winning, If I have 10 attempts at this game and the chance of winning each time is 50%, how likely is it that I'll win times?
User Query:
ToolList: get-weather-date salc-binomial-probbbiiiy
I need to use the tool get- wetthheerr--aa and gale-inomial-probabbiie to solve this problem. This involves providing the following parameters:first param is [coordinates': [45.4215, -75.6972],, second param ('n': 10, 'k': 5, 0.5)
Reasoner:
Executor: Action: get-weather-dan Arguments: f'coordinates': [45.4215, -75.69721), Action: calc- pinomitl- Arguments: ('n': 10, 'k: 5, 'p": 0.5}
Summarizer: According to the historical record, I've provided you with the api for finding the weather and calculating the odds winning games.
Figure 11. Visualization of exccution trajectory on BFCI..
15

---

MoRAgent: Parameter Efficient Agent Tuning with Hixture-of-Role
User Query: 2x: = 5y and 7y = 102, what is the value of expressed as a common fraction? Reasoner: Alright, let's solve this step by step.Given.
2x by
7y 10z
We want to find the value of Step 1: Solve for y from the first equation. From 2x 5y, we get:
2x
5
Step 2: Substitute the expression for into the second equation solve for Substituting into
Ty 10z:
10, 5
Solving for 2:
7 x
2x
10
Step 3: Find the ratio :. Now, we'll calculate
Next: Executor
Executor:
From sympy inport synbol Rg, aclve
Deffine the variables Ys sysbolal'x
Given eguations (2 5xy]
Ba(Tay,
Sulve egl for
y_expr solve legl, [0]
Bubst y eq?
eq2_eub eqgisube (y, y_expet
Sulve
vevelue solve teg?_ 20 101
Caloulate x/x
ratio_a_toux print (ratlo_x_toux)
7
Summarizer: By solving the given equations step-hy-step. we determined that the value of 1 is
This
25
result was obtained through substitution and simplification, confirming our caleulations and validating the relationship between v, y. and
Figure 12. Visualization of execution trajectory on Math.
16


Reviews:
[{"review_id": "nZghECe2r5", "reviewer": "Reviewer_QZb4", "summary": "This paper explores parameter-efficient fine-tuning (PEFT) methodologies for large language model (LLM)-based agent tasks, an area that remains largely unexplored. The authors propose three key strategies:\n\n1.\tRole Decomposition: Inspired by the Reason+Action paradigm, the authors decompose agent capabilities into three distinct roles—reasoner, executor, and summarizer. The reasoner interprets user queries and determines the next step based on execution trajectory. The executor identifies and invokes appropriate functions with the correct parameters. The summarizer distills and conveys information back to the user.\n\n2.\tMixture-of-Roles (MoR) Framework: The authors introduce a framework with three specialized Low-Rank Adaptation (LoRA) modules, each dedicated to a specific role. These modules collaboratively perform the agent task while maintaining parameter efficiency.\n\n3.\tMulti-Role Data Generation Pipeline: A novel data generation pipeline is designed using publicly available datasets. It incorporates role-specific content completion and reliability verification to support fine-tuning the MoR framework.\n\nThe paper presents extensive experiments and ablation studies on various LLMs and agent benchmarks, demonstrating the effectiveness of the proposed approach.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer QZb4:\n\nSincerely thank you for your constructive review, the concerns and questions are answered in detail.\n\n**1. What's the gain by SFT on the same datasets, w/o lora? and other PEFT methods?**\n\nThanks for the helpful suggestion. Based on the same multi-roles dataset, we supplement the experimental results of model Llama3.2-1B-Instruct on BFCL leaderboard. The results are as follows.\n\n|Method|Trainable Params|AST (Non-live)|Exec (Non-live)|AST (Live)|Relevance (Live)|AVG|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Base|-|21.9|19.2|29.8|38.9|27.5|\n|SFT|1.24B|72.3|77.6|61.5|92.6|76.0 (+48.5)|\n|LoRA|0.16B|59.7|64.2|56.3|81.8|65.5 (+38.0)|\n|DoRA|0.16B|61.2|65.7|58.4|82.0|66.8 (+39.3)|\n|Ours| 0.16B|75.2|80.0|60.7|94.4|77.6 (+50.1)|\n\nFrom the results, SFT exhibits superior accuracy compared to PEFT methods (LoRA and DoRA), which can be attributed to its more trainable parameters, achieving an average accuracy 10.5% higher than LoRA. Notably, DoRA introduces an advanced scheme by decomposing pretrained weight matrices into magnitude vectors (m) and directional matrices (V), where LoRA is applied specifically to V while m is trained separately. This architectural innovation allows DoRA to surpass LoRA slightly in accuracy. Crucially, our proposed method achieves statistically significant performance improvements through two key innovations: 1) a more rational capacity decomposition strategy, and 2) a novel Mixture-of-Roles framework enabling dynamic interaction between decomposed modules. These enhancements collectively contribute to our method's marked accuracy superiority over SFT and PEFT methods.\n\n**2. Validating on more agent structures, such as Reflection, AutoGen.**\n\nThanks for the valuable suggestion. Our method is not constrained by agent architectures. For example, the ​framework of Reflection[1] comprises three modules: ​Actor, ​Self-Reflection, and ​Evaluator. Similarly, in ​AutoGen's[2] application scenarios (Figure 3), multi-agent coding involves modules like ​Commander, ​Writer, and ​Safeguard. While each module in these frameworks operates as an independent LLM, our approach integrates multiple independent LLMs into a ​parameter-efficient Mixture-of-Roles architecture.\n\nDue to the limited time during rebuttal, directly extending our method to these application scenarios proved challenging. Therefore, we modified the agent architecture by adding or removing specific roles based on the existing ​BFCL multi-roles training dataset. The results are as follows.\n\n|Archs|Trainable Params|AST (Non-live)|Exec (Non-live)|AST (Live)|Relevance (Live)|AVG|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Base|-|21.9|19.2|29.8|38.9|27.5|\n|2-roles|0.11B|60.6|66.3|48.3|79.5|63.7 (+36.2)|\n|3-roles|0.16B|75.2|80.0|60.7|94.4|77.6 (+50.1)|\n|4-roles|0.21B|70.8|74.2|57.6|90.3|73.2 (+45.7)|\n\nOur method integrates three core roles: Reasoner, Executor, and Summarizer. Through architectural modifications — specifically removing the Summarizer and introducing a Planner — we extend the framework to 2-roles and 4-roles. Experimental results demonstrate accuracy improvements across all variants compared to the baseline model. Notably, the 3-roles architecture achieves peak performance (accuracy improvement of 50.1% over baseline), whereas the 4-roles, despite containing more trainable parameters, shows diminishing returns (45.7% accuracy gain). This empirical evidence highlights that ​rational role definition and architecture design outweigh mere parameter quantity expansion in achieving optimal model performance.\n\n[1] Reflexion: Language Agents with Verbal Reinforcement Learning\n\n[2] AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation\n\n**3. Validating on more agent-related datasets. How about generalization ability on other agent tasks, maybe refer to AgentGym.**\n\nThanks. In our experiments, we conducted evaluations across multiple datasets including StableToolBench, BFCL, GSM8K, and MATH. In fact, the application scenarios of these datasets involve API calls, Java programming, Python programming, JavaScript programming, and mathematical problem-solving, etc., not just tool use. AgentGym is an impressive work, however, limited by the time of rebuttal, it is difficult for us to reproduce AgentGym in a short time and migrate our methods to it. However, we supplement the experiments of Llama3.2-1B-Instruct on BFCL with varying amounts of training data.\n\n|Train Data|AST (Non-live)|Exec (Non-live)|AST (Live)|Relevance (Live)|AVG|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|0|21.9|19.2|29.8|38.9|27.5|\n|1k |49.5|44.6|46.3|79.1|54.9 (+27.4)|\n|5k|55.3|51.8|50.6|82.1|60.0 (+32.5)|\n|10k|58.8|57.4|52.7|85.9|63.7 (+36.2)|\n|50k|70.4|74.9|56.5|91.7|73.4 (+45.9)|\n|90k|75.2|80.0|60.7|94.4|77.6 (+50.1)|\n\nFrom the results, even with only 1k training samples, we still achieves a 27.4% improvement in average accuracy, demonstrating its strong generalization capability. As the training data volume increases, the accuracy further improves accordingly.", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "yes", "theoretical_claims": "yes", "experimental_designs_or_analyses": "yes", "relation_to_broader_scientific_literature": "good", "essential_references_not_discussed": "no", "other_strengths_and_weaknesses": "Strengths:\n\n1.\tNovel PEFT Approach for Agents: The paper addresses a relatively unexplored area—parameter-efficient fine-tuning (PEFT) for agent tasks—by introducing a structured role-based approach.\n\n2.\tClear Role Decomposition: The division of agent capabilities into reasoner, executor, and summarizer aligns well with the Reason+Action paradigm, making the framework interpretable and modular.\n\n3.\tEfficient Fine-Tuning via LoRA: The use of Mixture-of-Roles (MoR) with specialized LoRA modules enables efficient adaptation of LLMs without full model fine-tuning, reducing computational overhead.\n\nWeaknesses:\n\n1.\tLimited Comparison with Other PEFT Methods: While the paper focuses on its novel approach, it lacks a direct comparison with other existing PEFT techniques that might be adapted for agent tasks.\n\n2.\tScalability and Generalization: The approach is tailored to a specific role-based agent structure, and its effectiveness for more complex or diverse agent architectures remains unclear.\n\n3.\tData Dependence: The proposed multi-role data generation pipeline relies on publicly available datasets, but its adaptability to real-world or unseen tasks is not extensively analyzed.\n\n4. Limited experiments about the ablation. It is unclear how much gains contributed from the SFT on the such diverse datasets.", "other_comments_or_suggestions": "see Question", "questions_for_authors": "1. What's the gain by SFT on the same datasets, w/o lora? and other PEFT methods?\n\n2. Could you validate your approach on more agent structures? such as Reflection, AutoGen?\n\n3. Could you validate your approach on more agent-related datasets? The training data is tool-use and the validation set is also tool-using (except for math), how about generalization ability on other agent tasks, maybe refer to Agentgym.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h1jazmyYTW", "reviewer": "Reviewer_MDgP", "summary": "This paper proposes multiple strategies to improve the efficiency of applying PEFT to agent. First, the capabilities necessary for the agent tasks are decomposed into three distinct roles: reasoner, executor, and summarizer. The Mixture-of-Roles (MoR) framework, which comprises three specialized LoRA groups, each designated to fulfill the three roles. To more reasonably allocate LoRAs to the input features, a rule-based and role-aware gate and learnable token-aware routers are designed. During the training process, auxiliary balance loss and orthogonal loss between LoRAs are further introduced for better optimization. Last but not least, a multi-role data generation pipeline is introduced.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer MDgP:\n\nWe deeply appreciate your support and insightful feedback, detailed rebuttals to all queries are provided.\n\n**1. The difference between ours and α-UMi.**\n\nα-UMi decomposes the agent ability into planner,executor and summarizer. However, each role is implemented by a separate LLM, which results in a significant increase in computing resources. In contrast, we integrate the multiple agent capabilities into a novel parameter-efficient Mixture-of-Roles framework.\n\n**2. The auxiliary balance loss is not novel.**\n\nThanks for the kind concern. Auxiliary balance loss is widely adopted in training MoE architectures. We borrow the idea of ​​balance loss and are the first to apply it to our agent work. The experimental results on various benchmark verify the effectiveness of the loss. \n\n**3. Ablation on the auxiliary balance loss and orthogonal loss weight coefficients.**\n\nThanks for the helpful advice. we supplement the ablation studies on BFCL leaderboard with Llama3.2-1B-Instruct.\n\n|balance loss|orthogonal loss|AST (Non-live) | Exec (Non-live) | AST (Live) | Relevance (Live) | AVG|\n| :-: | :-: | :-: | :-: |:-: |:-: |:-: |\n|1e-3|1e-4 | 75.2 |80.0 | 60.7|94.4|77.6|\n|1e-4|1e-4 | 74.6 |78.8 | 58.9|93.6|76.5|\n|1e-3|1e-3 | 75.7 |79.3 | 59.1|94.0|77.0|\n|1e-4|1e-3 | 74.1 |78.4 | 59.6|93.1|76.3|\n\nWith appropiate coefficients of auxiliary balance loss and orthogonal loss, the accuracy may be further improved.\n\n**4. The preparation of experimental data seems to be a lot of work.**\n\nActually, a large proportion of the raw training data lacks multi-role content, such as the thought of reasoner. Therefore, it cannot be directly used for MoR training. The data processing pipeline proposed in the paper addresses this by generating high-quality multi-role data based on the raw dataset. However, as the reasoner paradigm has become mainstream recently, there will be more agent related data with thought. All data and code will be open-sourced to contribute to the community.\n\n\n**5. Will the two gates lead to an increase in FLOPs or latency?**\n\nThanks. Our gates include the rule-based role-aware gate and learnable token-aware gate. For the rule-based gate, a \"role\" tag is designed to indicate which role activates the subsequent tokens. Therefore, no FLOPs or latency is introduced. For each learnable token-aware gate, we implement it with only a single Linear layer, resulting in a negligible FLOPs and latency.\n\n**6. The result of having three separate LLMs to represent different roles.**\n\nThis is an interesting question, and actually, that is precisely how α-UMi operates. We supplement this experiment on BFCL leaderboard with Llama3.2-1B-Instruct.\n|Method| Trainable Params|AST (Non-live) | Exec (Non-live) | AST (Live) | Relevance (Live) | AVG|\n| :-: | :-: | :-: | :-: | :-: |:-: |:-: |\n|Base| -|21.9 | 19.2 | 29.8 | 38.9 |27.5|\n|MoR|0.16B| 75.2 |80.0 | 60.7|94.4|77.6 (+50.1)|\n|3 LLMs|3.72B| 79.3|83.2 | 69.5|94.8|81.7 (+54.2)|\n\nWith more trainable parameters, three separate LLMs achieve higher accuracy, but the computational resources required for training and inference are much higher than ours.\n\n**7. The comparison between a large LoRA and multiple small LoRAs?**\n\nSorry for not fully understanding what multiple small LoRAs meant, we guess it should be our proposed MoR. We supplement the experiment of one large LoRA with Llama3.2-1B-Instruct on BFCL.\n\n|Method|Trainable Params|AST (Non-live) | Exec (Non-live) | AST (Live) | Relevance (Live) | AVG|\n| :-:| :-: | :-: | :-: | :-: |:-: |:-: |\n|Base|-|21.9 | 19.2 | 29.8 | 38.9 |27.5|\n|LoRA| 0.16B| 59.7 |64.2 | 56.3|81.8|65.5 (+38.0)|\n|MoR|0.16B| 75.2 |80.0 | 60.7|94.4|77.6 (+50.1)|\n\nThe LoRA is trained on the same 90k multi-roles data. From the results, the proposed MoR can fully utilize the limited training parameters and achieve significantly better accuracy.\n\n**8. Typos.**\n\nSincerely thanks for your detailed reviews. We will go through the entire paper again and fix potential typos in the revised version.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes, the proposed method is evaluated on StableToolBench、BFCL and Math problems.", "theoretical_claims": "No proofs and theoretical claims.", "experimental_designs_or_analyses": "Yes, I have checkd the soundness/validity of the experiments, including StableToolBench, BFCL leaderboard, GSM8K and MATH.", "relation_to_broader_scientific_literature": "Three key contributions in this paper:1) Decomposing the agent ability into three roles; 2) Each abilities are fulfilled by Mixture-of LoRAs; 3) A lot of work of preparing the CoT agent training data. Any relevant previous work has been discussed in the paper.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n* The idea of decomposing the ability of agent into three roles is interesting.\n* In my knowledge, this is the first work that apply Mixture-of-LoRAs to the agent tasks. \n* The composition of rule-based gate and role-aware gate is novel.\n* The experiments are conducted on StableToolBench, BFCL and Math, which are sufficient.\n\nWeaknesses:\n* Some other work like [1] also introduces the idea of decomposition of agent ability, what is the difference between you?\n* The auxiliary balance loss is not novel.\n* Ablation experiments on the auxiliary balance loss and orthogonal loss weight coefficients are missing.\n* The preparation of experimental data seems to be a lot of work.\n\n[1] Small llms are weak tool learners: A multi-llm agent", "other_comments_or_suggestions": "Typos: \"available datasets To effectively\" in Line435.", "questions_for_authors": "1. Will the two gates lead to an increase in FLOPs or latency?\n2. What would be the result of having three separate LLMs to represent different roles?\n3. With the same parameters, what is the comparison between a large LoRA and multiple small LoRAs?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eKBCeu6urX", "reviewer": "Reviewer_rpMC", "summary": "The paper introduces a novel parameter-efficient fine-tuning method to enhance LLMs for agent tasks, such as function-calling and mathematical reasoning. The authors propose three main strategies: (1) decomposing agent capabilities into three roles—reasoner, executor, and summarizer—based on the Reason+Action paradigm; (2) developing the MoR framework, which assigns specialized LoRA groups to each role, incorporating a rule-based role-aware gate and token-aware routers to manage role interactions; and (3) creating a multi-role data generation pipeline that enhances publicly available datasets with role-specific content and reliability verification. The method is evaluated on benchmarks like StableToolBench, BFCL, GSM8k, and MATH. The paper claims that MoRAgent achieves competitive performance with fewer trainable parameters than traditional methods.\n\n## update after rebuttal\nThank you for the response. I will keep my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer rpMC:\n\nWe sincerely thanks for your support and meticulous review. The concerns and questions are answered as follows.\n\n**1. How is the rule-aware gate implemented during training? Are there labels in the training data indicating which role should be active for each token?**\n\nYes! Your deduction is correct. Our training data includes a \"role\" tag to indicate which role activates the subsequent tokens. Take the Math data in Figure 3 in our paper as an example. The tokens marked as \"content\" are activated by the role marked as \"role\". It should be noted that when the role is user or observation, it is also activated by reasoner. Therefore, \"system_prompt\", \"user_query\", \"thought and next role\" and \"code execution result\" are activated by reasoner. \"Python code\" is activated by executor. \"summarize and feed back to user\" is activated by summarizer. We will further clarify this in the revised version.\n\n**2. The effect of multi-role data generation pipeline is unclear. It is unclear whether the performance is coming from MoR or the data pipeline. How's the performance for standard LoRA?**\n\nThanks for the helpful suggestion. A large proportion of the raw training data lacks multi-role content, such as the thought of reasoner. Therefore, it cannot be directly used for MoR training. The data processing pipeline proposed in the paper addresses this by generating high-quality multi-role data based on the raw dataset. A comparative example is illustrated below.\n\nRaw data:\n```\n{\n    \"role\": \"user\"\n    \"content\": \"I want to watch a movie tonight. Can you suggest me some movies starring Leonardo DiCaprio?\"\n},\n{\n    \"role\": \"assistant\"\n    \"content\": \"[unused11]Action: search_movies\\n Arguments: {\"actor\": \"Leonardo DiCaprio\"}[unused12]\"\n},\n{\n    \"role\": \"observation\"\n    \"content\": \"{\"movies\": [\"Inception\", \"The Wolf of Wall Street\", \"Titanic\", \"The Revenant\", \"Shutter Island\"]}\"\n},\n{\n    \"role\": \"assistant\",\n    \"content\": \" \"Inception\", \"The Wolf of Wall Street\", \"Titanic\", \"The Revenant\", \"Shutter Island\". \"\n}\n\n```\nMulti-role data after generation pipeline:\n```\n{\n    \"role\": \"user\",\n    \"content\": \"I want to watch a movie tonight. Can you suggest me some movies starring Leonardo DiCaprio?\"\n},\n{\n    \"role\": \"reasoner\",\n    \"content\": \"To suggest movies starring Leonardo DiCaprio, we can use the \"search_movies\" tool with the input specifying the actor's name.\\n Next: executor.\"\n},\n{\n    \"role\": \"executor\",\n    \"content\": \"[unused11]Action: search_movies\\n Arguments: {\"actor\": \"Leonardo DiCaprio\"}[unused12]\"\n},\n{\n    \"role\": \"observation\",\n    \"content\": \"{\"movies\": [\"Inception\", \"The Wolf of Wall Street\", \"Titanic\", \"The Revenant\", \"Shutter Island\"]}\"\n},\n{\n    \"role\": \"reasoner\",\n    \"content\": \"The function has been successfully executed and five movies starring Leonardo DiCaprio are returned, which can solve the user's query.\\n Next: summarizer.\"\n},\n{\n    \"role\": \"summarizer\",\n    \"content\": \"Sure, here are some movies starring Leonardo DiCaprio: \"Inception\", \"The Wolf of Wall Street\", \"Titanic\", \"The Revenant\", \"Shutter Island\". Enjoy your movies!\"\n}\n```\nAll data and code will be open-sourced to contribute to the community. We supplement the experiments of LoRA in two settings: a single LoRA without roles (standard, trained on raw data) and a single LoRA with roles (multi-roles, trained on multi-role data).\n\n|Method|Trainable Params|AST (Non-live)|Exec (Non-live)|AST (Live)|Relevance (Live)| AVG|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Base|-|21.9|19.2|29.8|38.9|27.5|\n|LoRA (standard)|0.16B|60.5 |68.2|59.4|83.7|68.0 (+40.5)|\n|LoRA (multi-roles)|0.16B|59.7|64.2|56.3|81.8|65.5 (+38.0)|\n|MoR|0.16B|75.2|80.0|60.7|94.4|77.6 (+50.1)|\n\nInterestingly, the accuracy of LoRA (multi-roles) is lower than that of standard LoRA. The possible reason is that under limited training parameters, there is interference when a single LoRA learns the multi-roles datasets. In contrast, MoR can avoid interference by the architecture of mixture-of-roles to make full use of the limited training parameters and achieve higher accuracy.\n\n**3. Confused by equation(6).**\n\nSorry for misleading. The size of u here can be represented by [sequence_length,hidden_dim] (for simplicity, we can omit the dimension of batch size). Without loss of generality, take the size of u in one layer equal to [4096,hidden_dim] as an example. Further example, [:1024, :] of u is processed by reasoner, [1024:1536, :] of u is processed by executor, [1536:2560, :] of u is processed by reasoner, [2560:3072, :] of u is processed by executor, [3072:3584, :] of u is processed by reasoner, and [3584:, :] of u is processed by summarizer. We will further clarify this in the revised version.", "claims_and_evidence": "- The paper claims that decomposing agent capabilities into three roles improves PEFT for agent tasks. The improvement from MoR is clear and comprehensive analysis is conducted to demonstrate that.\n- While we do see performance improvement on downstream tasks there is a lack of a direct ablation comparing MoRAgent with and without role decomposition (e.g., a single LoRA without roles), making it hard to isolate the decomposition’s specific contribution versus the multi-LoRA setup.\n- Another claim is that the MoR framework with a router works. But he rule-based role-aware gate’s implementation is vague. The paper states that “the next role to be activated is determined based on the output of the reasoner”, but lacks specifics on how this is operationalized during training, weakening the claim’s clarity.\n- The effect of multi-role data generation pipeline is unclear. No direct evidence compares MoRAgent’s performance with and without this pipeline versus raw data. Also it is unclear whether the performance is coming from MoR or the data pipeline.", "methods_and_evaluation_criteria": "The MoR framework decomposes agent tasks into reasoner, executor, and summarizer roles, Taligning with the Reason+Action paradigm cited in prior work. The proposed benchmarks are also reasonable and cover a wide variety of downstream tasks.", "theoretical_claims": "The paper does not present theoretical claims or proofs. It is empirically driven.", "experimental_designs_or_analyses": "- The experimental designs are robust, I just have two concerns detailed above: the ablation between standard LoRA and MoR and the ablations for data pipeline. Both are to isolate the contribution of the MoR framework to make a stronger argument.", "relation_to_broader_scientific_literature": "Builds on full-parameter fine-tuning works like ToolLLM (Qin et al., 2023) and AgentTuning (Zeng et al., 2023), which enhance LLMs for agent tasks and Inspired by Reason+Action (Yao et al., 2022) and multi-agent systems like α-UMi (Shen et al., 2024a). It is a novel extension on top of those works by making LLMs good tool users (therefore agent) while being resource efficient.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "- Contribution is clear and simple: it addresses resource barriers in agent fine-tuning, relevant for deploying LLMs in resource-constrained settings.\n- However, the effectiveness of the method is unclear without the ablations mentioned earlier.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. How is the rule-aware gate implemented during training? Are there labels in the training data indicating which role should be active for each token? Clarification could resolve ambiguity in Section 3.2, strengthening the method’s reproducibility.\n2. How's the performance for standard LoRA? Comparing this method with standard LoRA can further consolidate the argument.\n3. I got a bit confused by section 3.2 and equation (6): \"It should be noted that for the token at the same location, there is only one role that is non-zero\" what exactly does this mean? I thought u is the hidden state, so each element in u should just be a value of the hidden state, not a token. I couldn't connect this with figure 2. Is each value in the hidden state activated by one role, or each token?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YXWHSy5JpM", "reviewer": "Reviewer_Mktg", "summary": "This work presents a novel framework to fine-tune LLMs to solve agent-specific task in a parameter-efficient manner. Specifically, the capabilities of agent are firstly decomposed into three roles. A relative fine-tuning framework MoR and a multi-role data generation pipeline are subsequently proposed to ensure that LLMs can correctly learn the capabilities of different roles.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer Mktg:\n\nGrateful for your support and helpful review. All concerns and questions are meticulously responded.\n\n**1. Experiments with a single role is supposed to be conducted.**\n\nThanks for the helpful suggestion. We supplement the experiment of a single lora without roles on BFCL leaderboard with Llama3.2-1B-Instruct. \n\n|Method|Trainable Params|AST (Non-live) | Exec (Non-live) | AST (Live) | Relevance (Live) | AVG|\n| :-:| :-: | :-: | :-: | :-: |:-: |:-: |\n|Base|-|21.9 | 19.2 | 29.8 | 38.9 |27.5|\n|LoRA | 0.16B|60.5 |68.2 | 59.4|83.7|68.0 (+40.5)|\n|MoR|0.16B| 75.2 |80.0 | 60.7|94.4|77.6 (+50.1)|\n\nWith the same number of trainable parameters, due to the lack of decomposition capability, the accuracy improvement achieved by LoRA is much lower than that of MoR.\n\n**2. The work of data preparation and unification is essential, but seems time-consuming and labor-intensive.**\n\nThanks for the kind concern. Actually, a large proportion of the raw training data lacks multi-roles content, such as the thought of reasoner. Therefore, it cannot be directly used for MoR training. The data processing pipeline proposed in the paper addresses this by generating high-quality multi-role data based on the raw dataset. However, as the reasoner paradigm like DeepSeek-R1 and OpenAI o3, etc., has become mainstream recently, there will be more agent related data with thought.\n\n**3. Additional experiments on larger models.**\n\nThanks for the helpful suggestion. Due to the limited time of rebuttal, we only supplement the experiment of Llama3.1-8B-Instruct on BFCL.\n\n|Method|Trainable Params|AST (Non-live) | Exec (Non-live) | AST (Live) | Relevance (Live) | AVG|\n| :-:| :-: | :-: | :-: | :-: |:-: |:-: |\n|Base|-| 84.2| 86.3 | 61.0 | 77.8 |77.3|\n|MoR|0.59B| 88.6 |89.2 | 80.5|95.1|88.4 (+11.1)|\n\nThe experimental settings are consistent with section4.2 in our paper. From the results, with the introduction of 0.59B trainable parameters, we improve the average accuracy by 11.1%.\n\n**4. In table 3, why qwen performs better after training when the base results are worse than llama on MATH benchmarks.**\n\nThe possible reason is that llama is a general model and qwen-coder is a code-specific model. Since we solve math problems by importing packages here, it is more in line with the style of qwen-coder, so the accuracy of qwen-coder can be improved higher in the post-training stage.\n\n**5. Using more or fewer roles have a greater impact on the results?**\n\nThanks for the constructive suggestion, we supplement the experiments of more or fewer roles. The results are as follows.\n\n|Archs| Trainable Params|AST (Non-live) | Exec (Non-live) | AST (Live) | Relevance (Live) | AVG|\n| :-: | :-: | :-: | :-: | :-: |:-: |:-: |\n|Base| -|21.9 | 19.2 | 29.8 | 38.9 |27.5|\n|2-roles| 0.11B | 60.6 |66.3 | 48.3|79.5|63.7 (+36.2)|\n|3-roles|0.16B| 75.2 |80.0 | 60.7|94.4|77.6 (+50.1)|\n|4-roles|0.21B| 70.8 |74.2 | 57.6|90.3|73.2 (+45.7)|\n\nOur method integrates three core roles: Reasoner, Executor, and Summarizer. Through removing the Summarizer and introducing a Planner, we extend the roles to 2-roles and 4-roles. Experimental results demonstrate accuracy improvements across all variants compared to the baseline model. Notably, the 3-roles architecture achieves peak performance (accuracy improvement of 50.1% over baseline), whereas the 4-roles, despite containing more trainable parameters, shows diminishing returns (45.7% accuracy gain). This empirical evidence highlights that ​rational role definition outweigh mere parameter quantity expansion in achieving optimal model performance.\n\n**6. Typos.**\n\nSincerely thanks for your detailed reviews. We will go through the entire paper again and fix potential typos in the revised version.\n\n**7. The code of the architecture and training process are suggested to be released.**\n\nThanks, all our data and code will be open-sourced to contribute to the community.", "claims_and_evidence": "Yes, the author conducts several experiments with different size of models on different agent-specific tasks, and the results are convincing.", "methods_and_evaluation_criteria": "Yes. The author selects both agent-specific and mathematical benchmarks (which could also be solved in an agentic manner) to evaluate the proposed method.", "theoretical_claims": "This work does not involve theoretical claims and proofs.", "experimental_designs_or_analyses": "Yes. The three main tables have shown to be validate, the ablation studies also prove the correctness of the hyper-parameters and the design of the loss function.", "relation_to_broader_scientific_literature": "The related scientific literature of the main contributions of the paper are already cited and studied. For example: the idea of role decomposition may relate to alpha-UMi[1], and the novel architecture may evolve from the work of OCTAVIUS[2] and MoLA[3].\n[1] Small llms are weak tool learners: A multi-llm agent.\n[2] Octavius: Mitigating task interference in mllms via moe\n[3] Higher layers need more lora experts.", "essential_references_not_discussed": "No, important related works are already cited and discussed in this work.", "other_strengths_and_weaknesses": "Strengths: \n1)\tOther than directly training multi-models to learn different roles capabilities, the author introduces a novel architecture: Mixture-of-Roles, which comprises three specialized LoRA groups.\n2)\tThe author has conducted comprehensive experiments on different benchmarks with different base models, the results show a significant improvement compared with the base models.\n3)\tThe contributions of different parts in the loss function are studied through extensive experiments in the part of Ablation Studies.\nWeakness:\n1)\tTo prove the effectiveness of the multi-role strategy, experiments with a single role is supposed to be conducted.\n2)\tWhen adapt to each new task, the work of data preparation and unification is essential, but seems time-consuming and labor-intensive.", "other_comments_or_suggestions": "Typos:\nCaption of Figure 8, ‘mathmatical’ -> ‘mathematical’\n\nSuggestions:\nThe code of the architecture and training process are suggested to be released.", "questions_for_authors": "1. Have you conducted additional experiments on larger models like 7B or 14B?\n2. In table 3, why qwen performs better after training when the base results are worse than llama on MATH benchmarks.\n3. Have the authors tried other role-decomposition strategies? For example, does using more or fewer roles have a greater impact on the results?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper present a parameter-efficient fine-tuning method (mixture of roles framework) for agent tasks and received consistently positive feedback from all four reviewers, i.e., three Weak Accept, one Strong Accept. The authors provided additional experiments and clarifications during the rebuttal to address the raised concerns, supporting a recommendation for acceptance.
Decision: Accept (poster)