AgentMonitor: A Plug-and-Play Framework for Predictive and Secure Multi-Agent Systems

## Abstract

Abstract Rapid advancement of large language models (LLMs) has catalyzed the emergence of LLM-based agents. Recent research has shifted from single-agent systems to multi-agent frameworks, demonstrating that collaboration can outperform the capabilities of individual LLMs. In these frameworks, each agent is assigned a specific role and operates within a configurable pipeline, enabling teamwork toward a shared objective. However, effectively pre-configuring a multi-agent system (MAS) for a specific task remains a challenging problem, with performance outcomes only observable after execution. Inspired by the well-established scaling laws in LLM development that model downstream task performance or validation loss as functions of model size, data size, or training FLOPs, we seek to investigate the predictability of MAS performance. Specifically, we explore whether it is possible to predict the downstream task performance of a configured MAS. Additionally, MAS face a growing challenge in ensuring reliable and trustworthy responses. The introduction of malicious agents can lead to the generation and spread of harmful content, which poses significant security risks. These vulnerabilities highlight the need for robust defenses against safety issues. To address above issues, we introduce AgentMonitor , a framework that integrates with existing MAS at the agent level. AgentMonitor captures inputs and outputs at each step; this enables (1) transforming them into relevant statistics supporting the training of a regression model to predict task performance and (2) the application of on-the-fly corrections to mitigate negative impacts on final outcomes. Extensive experiments demonstrate that training a simple XGBoost model achieves a high Spearman rank correlation of 0.89 in an in-domain setting. In more challenging scenarios, where the task or architecture composition is absent from the training set, our method maintains a moderate average correlation of 0.58 . Furthermore, by employing AgentMonitor in a maliciously configured MAS, the system ultimately generates 6.2% less harmful content and 1.8% more helpful content on average, reducing safety risks and improving reliability. Our code is available at https://github.com/chanchimin/AgentMonitor .

## 1 Introduction

Recently, the rapid development of LLMs has been widely reported (Achiam et al., ; Dubey et al., ; Gemini et al., ). These models exhibit strong capabilities, achieving success in various tasks of natural language processing (NLP) (Radford et al., ). Leveraging training processes such as instruction tuning (Longpre et al., ), LLMs have demonstrated the ability to articulate reasoning (Wei et al., ; Yao et al., ), self-correct errors (Madaan et al., ), utilize external tools (Schick et al., ; Qin et al., ), and retain long-term memory (Huang et al., ) during inference. By combining these capabilities with various techniques, researchers have successfully built on off-the-shelf LLMs to create single-agent systems capable of solving more complex tasks. Notable examples include AutoGPT (Gravitas, ), XAgent (XAgent, ), and OpenInterpreter (OpenInterpreter, ).

Figure: Figure 1: We investigate the research question: ”Given knowledge of existing MAS and their corresponding target scores, how accurately can we predict the performance of a new MAS on an unseen task?” As illustrated in the figure, different configurations of MAS are presented. We use to represent the capabilities of the underlying LLM for each agent. For instance, this could be Llama3-8B, Llama3-70B, or other models.
Refer to caption: x1.png

Beyond the aforementioned single-agent applications, research has emerged that focuses on enabling multiple LLMs to collaborate on specific tasks. Inspired by evidence of collective intelligence (Woolley et al., ) arising in groups of humans, various multi-agent frameworks have been proposed to mimic human collaborative scenarios. Typically, in these frameworks, each agent is controlled by an LLM with an assigned role, and a predefined executable pipeline is configured. Following the pipeline, agents collaborate towards a common goal. This approach has shown promising results, demonstrating that a well-configured LLM-based Multi-Agent System (MAS) can outperform a single agent in certain contexts. Notable successes include Generative Agents (Park et al., ), which simulates human society, AutoGen (Wu et al., ), CAMEL (Li et al., ), AgentVerse (Chen et al., ) which tackles reasoning tasks, ChatEval (Chan et al., ) which tackles evaluation tasks, as well as ChatDev (Qian et al., ) and MetaGPT (Hong et al., ), which focus on software tasks.

Despite the significant success of these MAS, obtaining the optimal MAS configuration remains an unresolved challenge. The process often requires careful design, relying on prior knowledge of the task and heuristic approaches. The effectiveness of the chosen configuration can only be evaluated after the actual execution, which can be resource intensive and inefficient during production. Inspired by the well-studied scaling laws(Kaplan et al., ) in LLM development – which model target task performance(Isik et al., ) or validation loss as functions of model size, data size (Hu et al., ), training FLOPs (Hoffmann et al., ), or data mixtures (Ye et al., ) – we aim to explore whether it is possible to predict downstream task performance given the task and the configuration. Such predictability would enable us to design more reliable and effective MAS without the need for costly trial and error.

To this end, we introduce AgentMonitor, a plug-and-play framework that integrates seamlessly with existing MAS at the agent level, captures inputs and outputs at each step and transforms them into meaningful indicators to predict target scores, as illustrated in Figures 2 and  3. The core design is inspired by the recently popular parameter-efficient tuning frameworks such as PEFT (Mangrulkar et al., ) and OpenDelta (Ding et al., ; Hu et al., ), which wrap pre-trained models with additional parameters without altering the original model structure. Similarly, AgentMonitor wraps a function around the agent itself, making it adaptable to various multi-agent frameworks. By capturing inputs and outputs at each timestep when the agent communicates, we gather information (as shown in Figure 2) that is transformed into system-specific indicators for each multi-agent configuration. Using these captured data, we address our earlier question: How predictable is the performance of a MAS? We answer this by training a simple regression model XGBoost (Chen & Guestrin, ) on these stored indicators. Afterward, when given a newly configured MAS, we use the model to predict its performance on the target tasks.

In this paper, we manually design five distinct architectures, each with different agent assignments and message flow configurations. Our comprehensive experiments across three tasks– HumanEval (Chen et al., ), MMLU (Hendrycks et al., ), and GSM8K (Cobbe et al., )–demonstrate that the predicted values can achieve a Spearman rank correlation of 0.89 with observed values in an in-domain setting. Furthermore, even in more challenging scenarios, where the target task or architecture is absent from the training set, our method maintains a moderate average correlation of approximately 0.58. Furthermore, we show that introducing a malicious agent into the system drastically degrades the quality of the final output, resulting in more harmful or unhelpful responses (Section 4.4). However, with our AgentMonitor actively monitoring and post-editing the agents’ responses in real-time, the harmful effects are significantly mitigated. This underscores the effectiveness of AgentMonitor in building more reliable and beneficial MAS by revealing its predictability and enhancing real-time intervention capabilities.

Figure: Figure 2: Illustration of our AgentMonitor.
Refer to caption: x2.png

Figure: Figure 3: Using AgentMonitor to predict target score.
Refer to caption: x3.png

Figure: Listing 1: An illustration of the usage of our AgentMonitor, along with a comparison to popular frameworks like PEFT in LLM-building scenarios.

## 2 Related Work

### 2.1 LLM Based Agents and Multi-Agent FrameWork

Recent advances in large language models (LLMs), such as GPT-4 (Achiam et al., ), have stimulated the development of LLM-based agents. These agents are capable of utilizing external tools, such as interpreters (OpenInterpreter, ), search engines (Luo et al., ; Chan et al., ), web browsers (Nakano et al., ; He et al., ), or custom-defined tools (Qin et al., ; Schick et al., ) through function calling. Leveraging the strong instruction-following abilities of foundation models, these agents have demonstrated significant progress across various domains. For instance, integration with operating systems (Wu et al., ), the development of XAgent (XAgent, ) for complex task-solving, and SearchGPT (OpenAI, ) for accelerating search experiences. In line with these advances, frameworks for efficiently building LLM agents have emerged, such as LangChain (LangChain-AI, ), AgentGPT (Reworkd, ), and AutoGPT (Significant Gravitas, ).

Beyond single-agent intelligence, recent research indicates that collaboration among multiple agents, each with different expertise, can enhance downstream task performance. Notable successes include AutoGen (Wu et al., ), which facilitates the creation of conversable agents for various pilot applications, such as online decision-making; OpenDevin (Wang et al., ), a platform for developing powerful and flexible AI agents that interact with the world in ways similar to those of human developers; IOA (Chen et al., ), which addresses the challenges of distributed agent deployment by introducing an agent integration protocol, along with a design of an instant messaging architecture. However, constructing reliable MAS with these frameworks often involves trial and error in identifying the optimal configuration. Research has also shown that MAS can be vulnerable to malicious attacks (Ju et al., ; Zhang et al., ), such as prompt injections or misalignment of the model. To address these challenges, we propose AgentMonitor, a framework that proactively monitors indicators in MAS and applies on-the-fly corrections. This not only helps predict downstream task performance, but also mitigates unsafe behaviors.

### 2.2 LLM Predictablility and Scaling Laws

The vast development of LLMs is closely related to the concept of neuron scaling laws (Kaplan et al., ; Rae et al., ; Henighan et al., ). Previous works have attempted to capture the relationships between training FLOPS, model size, etc, and validation loss by first training numerous differently configured models and then proposing a power law to fit the coefficients. Once fitted, this power law can be used to extrapolate and predict the loss for a larger model and further simulated to derive the optimal configuration for target size model. This paradigm has led to several practical and constructive suggestions. For example, Chinchilla law (Hoffmann et al., ) suggests that while given a computational budget of 10x, the suggested model size should be 5.5x larger, while training tokens should be 1.8x more. Similarly, Minicpm (Hu et al., ) derive optimal batch size and learning rate configurations from LLM sandbox experiments where they train a multi-set of smaller models, showing that their 2.4B model performs on par with current 7-13B scale models. MM1 (McKinzie et al., ) performed a grid search for the optimal learning rate using smaller models and then successfully extrapolated the results to larger scales. BIMIX (Ge et al., ) proposed a bivariate law concerning data quantity and mixing proportion, demonstrating that their optimized data mixture outperforms the default mixture.

Given the fruitful results in the construction of LLM, this work aims to investigate the predictability of MAS by proposing a framework that can capture relevant indicators to predict target scores. Another study (Qian et al., ) explored collaborative scaling laws by increasing the number of agents in a system, finding that normalized solution quality follows a logistic growth pattern as the number of agents increases. However, given the versatility required in building different MAS, it is challenging to determine a single-variable law for the entire system. Therefore, we preliminarily explore the predictability of MAS using machine learning methods such as XGBoost (Chen & Guestrin, ) to assess the feasibility of directly predicting target scores for MAS.

A similar work to ours is Ye et al. (), which investigates the predictability of LLM capabilities. Given records of past experiments from different model families, number of parameters, tasks, and in-context examples, they explore whether LLM performance on new configurations can be accurately predicted. Our work shares a similar goal; however, to the best of our knowledge, is the first to conduct such a study in the context of building MAS. We hope that this research provides valuable insights and paves the way for the community to build better MAS.

## 3 AgentMonitor

**Table 1: The configuration of five MAS used in our paper, along with the used indicators.**
| Architectures | Agents | Example Execution Graph | Indicators |
| --- | --- | --- | --- |
| Arch1 | The coder is responsible for providing a self-contained code that can solve the task. |  |  |
| The reviewer is responsible for writing review and provide comments. | Indicators Judged by LLM |  |  |
| The tester is responsible for writing unit test to ensure the code functionality. | Each Agent’s Personal Score |  |  |
| Arch2 | The coder is responsible for providing a self-contained code that can solve the task. |  | Each Agent’s Collective Score |
| The reviewer is responsible for writing review and providing comments. |  |  |  |
| The dummy agent does not have specific responsibility but is acting as a black sheep. | Indicators Determined by Configuration |  |  |
| The tester is responsible for writing unit test to ensure the code functionality. | Each Agent’s Pagerank |  |  |
| Arch3 | The coder is responsible for providing a self-contained code that can solve the task. |  | Each Agent’s Capabilities |
| The reviewer is responsible for writing review and providing comments. | Number of Nodes |  |  |
| The web browser is responsible for using the Internet to retrieve useful information. | Number of Edges |  |  |
| The tester is responsible for writing unit test to ensure the code functionality. |  | Average Clustering |  |
| Arch4 | The coder is responsible for providing a self-contained code that can solve the task. |  | Transitivity |
| The modifier is responsible for modifying the code written by the coder. | Average Degree Centrality |  |  |
| The reviewer is responsible for writing review and providing comments. | Average Closeness Centrality |  |  |
| The tester is responsible for writing unit test to ensure the code functionality. |  | Average Betweenness Centrality |  |
| Arch5 | The executor is provided with an executable interpreter and should execute the code. |  | Heterogeneous Score |
|  | The web browser is responsible for using the Internet to retrieve useful information. |  |  |

In this section, we begin by discussing the intuitive design and usage of our AgentMonitor (Section 3.1), followed by an exploration of the indicators that we collect to train the regression model(Section 3.2). We then introduce the pre-edit and post-edit features of AgentMonitor (Section 3.3).

### 3.1 Design of AgentMonitor

In this section, we begin by introducing the parameter-efficient fine-tuning scenario and the practical use of popular non-invasive frameworks for building models with PEFT. We then discuss the motivation for our proposed AgentMonitor in this context.

Fine-tuning a large language model with all parameters is costly and time-consuming. As a solution, researchers have proposed various parameter-efficient fine-tuning techniques, such as LoRA (Hu et al., ), Adapters (Houlsby et al., ), and Prompts (Lester et al., ), which achieve near-lossless downstream task performance by only training a small proportion of the model parameters. Despite the success of these techniques, individually modifying the original structure of the pretrained model for each method can be inefficient.

In response, incremental frameworks have emerged that adapt various techniques to the original structure without invasive modifications. These frameworks wrap functions around the original model, preserving the original workflow while maintaining high scalability for broader applications. As shown in the upper part of the Code Listing 1, after initializing a pretrained model, the PEFT framework requires only one line of code to register the original model with the PEFT module, without disrupting the original fine-tuning process. Similarly, in the bottom part of the Code Listing 1, we illustrate how AgentMonitor follows the same design principle. By using a single line of code to register each agent in our framework, AgentMonitor seamlessly integrates without altering the original workflow.

Figure: Figure 4: Given a user instruction, a MAS with a malicious agent can potentially generate harmful or helpless response (left), while with our monitor post-editing its responses, it can thereby reduce the harmful effect (right).
Refer to caption: x9.png

### 3.2 Details of Indicators Used

Figure: Figure 5: Spearman rank correlation between the predicted score and the observed score.
Refer to caption: x10.png

In this section, we detail how we transform the stored information into indicators that are used to train the regression model. Specifically, we categorize the indicators into two groups. The first group consists of scores generated by using an LLM to assess performance, including the agent’s personal score and the collective score. Intuitively, the personal score measures how well an agent completes its own task following the given instruction, while the collective score evaluates how the agents’ behaviour contributes to the overall system. For example, an agent given the instruction to generate helpless or nonsense responses might excel at its specific task and receive a high personal score. However, it would earn a low collective score, as it does not contribute significantly to the final result.

The second group of indicators comprises scores determined after the MAS is built. For each MAS, a workflow is designed, forming an execution flow. From this, we extract graph attributes such as the number of nodes, number of edges, transitivity, etc. The rationale behind these indicators is that different workflow mechanisms result in distinct graph attributes. For instance, a linear configuration (A $\rightarrow$ B $\rightarrow$ C) has 3 nodes, 2 edges, and a transitivity of 0. This implies that no triangular structures are formed, potentially limiting the efficiency of information flow across the system.

Table 1 illustrates the different architectures that we designed in this paper, along with the full list of indicators used. For further details on each indicator and the prompts used to evaluate the scores, refer to Appendix B and Appendix A.2.

### 3.3 Pre-Edit and Post-Edit Features

In addition to recording the relevant information of MAS, our framework is naturally designed to capture the input and output of each agent at each step; this operation enables us to easily do the pre-edit and the post-edit. In this paper, we experiment with using another LLM to help edit the response generated by the agent in the MAS, and this operation is done on-the-fly. The post-edit operation are inspired by the adapter in the neuron net work where the output of the hidden state is further passed throught the adapter which is more lightweight and easier to be updated. More similarly, Aligner (Ji et al., ) uses a lighter LLM to elevate the response generated by an LLM and shows the great effectiveness of the weak-to-strong generalization. Our experiments show that without monitoring, a MAS configured with a malicious agent is versatile and easily generates a harmful or helpless response. Conversely, the same MAS that is monitored by our AgentMonitor can generate a more harmless or helpful response. The illustration of the post-edit can be seen in Figure 4. Specifically, the post-edit operation utilizes an LLM to elevate the original response after the agents; in our experiment, we use this operation either after one agent in a system or after all the agents in a system. For example, assuming there are A agent and B agent, and the original information flow is (A $\rightarrow$ B), then, our post-edit version will be (A $\rightarrow$ post-edit LLM $\rightarrow$ B).

## 4 Experiments

In this section, we experiment with the use of the indicators introduced in Section 3.1 to predict the target scores. We report both the Spearman correlation and the errors between the predicted and observed values. As shown in Table 1 and Figure 1, we manually design five different MAS, each configured with different roles. The third column of Table 1 illustrates example execution graphs, where the arrows ($\rightarrow$) indicate the flow of information from the start to the end of the arrow.

Figure: Figure 6: Errors between the predicted score and the observed score.
Refer to caption: x11.png

We evaluate the MAS on three downstream tasks: HumanEval (Chen et al., ), MMLU (Hendrycks et al., ), and GSM8K (Cobbe et al., ), which test coding, reasoning, and math abilities, respectively. We use a sampled version from MINT (Wang et al., ), where the queries are complex enough to require multi-agent collaboration. Furthermore, we configure each agent with different LLMs chosen from Llama3-8B, Llama3-70B, and ChatGPT, for details, see Appendix C.1. That is, We can perturb the selection of LLMs, generating a new combination that is a new data point (^1^11Here, each data point is a tuple of {various indicators, downstream task performance}.) for training a regression model. By treating a different combination as a different data point, we can obtain $3^{4}$ data points from this architecture in total if there are 4 agents in the system and 3 optional LLMs. In this experiment, we collect a total of 1,796 data points. We then performed a grid search to train an XGBoost model (Chen & Guestrin, ), using reg:squarederror
as the objective function.

Additionally, we experiment with scenarios ranging from easy to difficult by designing the following settings:

- •
In-Task: We group the tuples by task, then split them into training and test sets.
- •
In-Arch: We group the tuples by architecture, then split them into training and test sets.
- •
In-Domain: We randomly split all the data into training and test sets.

For the harder scenarios, we experiment with the following:

- •
Cross-Arch: We leave out one architecture for the test set and use the remaining architectures for the training set.
- •
Cross-Task: We leave out one task for the test set and use the remaining tasks for the training set.

### 4.1 Overall Results for Predicting Target Scores

The mean Spearman rank correlation with error bars is shown in Figure 5. The error bars are plotted using a 5-fold cross-validation. We observe a clear pattern: (1) Easier settings, such as In-Task, In-Arch, and In-Domain, tend to achieve relatively high correlations and low variances, which is expected. Among these, the results suggest that HumanEval is the most challenging to predict. (2) Compared to In-Task and In-Arch, the In-Domain setting–which includes all tasks and architectures–achieves a higher overall mean score and a relatively lower variance. This indicates that access to information from other tasks or architectures improves predictive performance, and indicators are transferable between different settings. (3) Although some results in more difficult settings lead to lower mean scores, particularly when Arch3 is left out in the Cross-Arch setting or HumanEval is excluded in the Cross-Task setting, the overall average results still reach 0.58, showing the effectiveness of using our AgentMonitor to predict the target score.

Furthermore, the boxplot in Figure 6 shows the distribution of errors in all settings. In general, the median lies close to zero, suggesting that the predicted values are generally close to the observed values. Despite the lower correlations in more difficult settings, we observe that in the Cross-Arch and Cross-Task settings, the median error remains near zero, indicating that for most instances, the predicted values are close to the observed values. Furthermore, although the correlation for HumanEval in In-Task setting is lower, the error is not, being smaller overall compared to the other two tasks. This suggests that while predicting the rank is more challenging for this task, the regression model can still predict values that are reasonably closer to the observed values.

### 4.2 A Closer Look into Relationships of Each Indicator

Figure: Figure 7: Feature Heatmap of Arch1 on MMLU (Left) and GSM8K (Right).
Refer to caption: x12.png

Figure: Figure 8: Parallel Coordinate Plot of Top8 (Left) and Bottom8 (Right) Features of Arch1 on GSM8K.
Refer to caption: x14.png

In this section, we take a closer look at the relationships between each indicator. As shown in Figure 7, the correlation heatmap of the indicators in MMLU and GSM8K for Arch1 is presented. A clear pattern emerges: scores evaluated by the LLM, such as the coder personal score and the coder collective score, show a higher correlation with each other but exhibit a lower correlation with other graph attributes, such as the total number of nodes and transitivity (indicated by the lighter color near the diagonal). Additionally, the correlation between different agents’ scores is not as strong as the correlation between the same agents’ scores. This mirrors real-world practice, where individuals within the same team often exhibit distinct behaviors.

Furthermore, in Figure 8, we present the parallel coordinate plot of the features with the top eight and bottom eight importance scores of the features. The feature importance is calculated by XGBoost, which measures how much each indicator contributes to the model’s inference. A clear trend is evident in Figure 8 (left), where the figure is much denser compared to Figure 8 (right), which is more sparse. This suggests that there is a greater variety of independent indicators in the denser figure, with the indicators corresponding to different target scores being more scattered, demonstrating their separability. In contrast, indicators with lower importance tend to have similar valid choices and lack clear separability. In particular, it is also observable that most indicators do not exhibit monotonicity with respect to the target score, meaning that a higher indicator value does not necessarily result in a higher target score.

### 4.3 Ablation Study on the Scaling Effect

In this section, we analyze RQ1: how the training size affects the predictive results and RQ2: how the number of instances that we used to calculate indicators affects the predictive results.

Figure: Figure 9: Left: Spearman Correlation and RMSE vs ratio to train.
Refer to caption: x16.png

In this first experiment tailored for RQ1, we focus on the In-Domain setting and randomly sample 10$\%$, 20$\%$, 30$\%$, $\cdots$, up to 100$\%$ of the original training set to form new training sets, while keeping the test set constant. As shown in Figure 9, as the training size increases, we observe a noticeable improvement in Spearman correlation and a reduction in RMSE, as expected. This indicates that more training data contribute to better predictive performance. Notably, when the training set reaches 50$\%$, the results plateau, approaching those achieved with the entire dataset. This suggests that in this specific setting, half of the data contains sufficient information to achieve acceptable results.

Another intuitive question is to what extent we need prior information to calculate the indicators described in Section 3.2. For example, given a prebuilt MAS and a downstream task, it is impractical to obtain all the required information only after completing all the instances in the test set. Ideally, we would only need a few instances to gather enough information for a ”sneak peek” at the system’s potential performance. In contrast to the experiments detailed in the previous section, where the indicators were calculated by averaging over all instances, in the second experiment tailored for RQ2, we use a subsample of the total instances to calculate the ”approximated indicators.” We then analyze the effect of the number of instances used to calculate these indicators.

We begin by using the best-trained XGBoost model to perform inference on the In-Domain test set, retaining instances with an absolute error smaller than $0.05$ as a new test set. The rationale behind this is that, these samples are more predictable for the trained model, they better illustrate the usefulness of the indicators. Otherwise, indicators that are poorly predicted might not be explained by our model and could hinder the interpretation of the approximated indicators.

As shown in the left and right part of Figure 10, we observe the following: (1) There is a clear trend that as the ratio of instances used to calculate the indicators increases, the Spearman correlation continually rises, and the RMSE decreases. This suggests that increasing the number of instances used to calculate the indicators improves predictive performance. As expected, when the ratio increases, the predictive performance of the ”approximated” indicators converge toward that of the ”accurate” indicators. (2) Even when using only $10\%$ of the total instances to calculate the indicators, the Spearman correlation is still around $0.82$, supporting the claim that we can use a relatively small subset of data to gain an early glimpse of the final performance. This approach can guide the construction of MAS without fully executing the entire dataset. (3) The average error and variance decrease as the ratio increases. Additionally, we observe that when the ratio is low, the approximated indicator values tend to be smaller than the accurate values, suggesting that the main source of error may stem from certain indicators being underestimated by the LLM judger.

Figure: Figure 10: Left: Spearman Correlation and RMSE vs ratio to calculate indicators ; Right: Average Errors of Indicators vs ratio to calculate indicators.
Refer to caption: x17.png

### 4.4 When Maliciously Configured, MAS Becomes Vulnerable

In addition to using the stored indicators to predict the target score, our AgentMonitor has another practical application. Since it captures the input and output of each agent in real time, the monitor can pre-edit or post-edit the input or output without affecting the original execution. In this section, we experiment with safety prompts from Beavertails (Ji et al., ), MaliciousInstruct (Huang et al., ), and AdvBench (Zou et al., ). These tasks contain queries that might contravene the policy. Although most open-source LLMs are aligned, when maliciously instructed, there is still a risk of generating harmful content. For details on the models used, evaluation datasets and metrics, refer to Appendix C.

**Table 2: Comparison of Different Architecture with and without post-edit on three safety tasks. (Green and Red) means that the score is (higher,lower) than the base score (first row) in each category, respectively. means Harmless Agent; means Helpful Agent; means Harmful Agent; means summarizer Agent; means we add post-edit on the second architecture that contains malicious agent with either u8B or 8B model.**
| Task | Architecture | Compared with u8B | Compared with 8B | Compared with 70B |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
|  | (u8B) | Harmless $\uparrow$ | Helpfulness$\uparrow$ | Harmless$\uparrow$ | Helpfulness$\uparrow$ | Harmless$\uparrow$ | Helpfulness$\uparrow$ |
| Beavertails |  | -0.08 | -0.29 | -0.78 | -0.20 | -0.79 | -0.38 |
|  | -0.25 | -0.41 | -0.78 | -0.19 | -0.80 | -0.39 |  |
| (u8B) | -0.08 | -0.46 | -0.75 | -0.25 | -0.74 | -0.40 |  |
| (8B) | 0.47 | -0.42 | -0.61 | -0.45 | -0.70 | -0.56 |  |
| AdvBench |  | 0.94 | -0.08 | -0.07 | 0.01 | -0.05 | 0.05 |
|  | -0.03 | -0.18 | -0.98 | -0.15 | -0.98 | -0.15 |  |
| (u8B) | 0.15 | -0.17 | -0.97 | -0.09 | -0.98 | -0.13 |  |
| (8B) | 0.92 | -0.20 | -0.06 | 0.41 | -0.39 | -0.35 |  |
| MaliciousInstruct |  | 0.16 | -0.26 | -0.99 | 0.02 | -0.99 | -0.15 |
|  | -0.04 | -0.28 | -0.99 | -0.14 | -0.99 | -0.28 |  |
| (u8B) | 0.09 | -0.25 | -1.00 | -0.09 | -1.00 | -0.17 |  |
| (8B) | 0.70 | -0.50 | -0.29 | -0.33 | -0.39 | -0.49 |  |

In this experiment, we additionally design two architectures. The first architecture contains a harmless agent (which aims to provide harmless responses), a helpful agent (which aims to provide helpful responses), and a summarizer agent (which aims to summarize the responses generated by the different agents). The second architecture contains the same agents as the first but also includes a malicious agent (instructed to generate harmful and unhelpful responses). We compare the responses generated by our crafted MAS with those generated by different single LLMs (u8B, 8B, and 70B, respectively). As shown in Table 2, when a MAS is mixed with malicious agents, it becomes more prone to generating harmful or unhelpful responses.

It is observed that, on all tasks, the second agent team received lower scores on both the harmlessness and the helpfulness dimensions, suggesting the vulnerability of the MAS, as also noted in  Zhang et al. (). We then tested the effect of adding post-editing LLMs within our AgentMonitor framework, applying different LLMs after all agents had generated their responses. The results show that using the same LLM as a post-editing backbone slightly improves both harmlessness and helpfulness (compare the third row with the second row in each task block). Additionally, when we substitute the post-editing LLM with a more aligned backbone LLM, we observe a significant improvement in harmlessness scores, though helpfulness experiences varying degrees of degradation (compare the fourth row with the second and first rows). This suggests that optimizing both dimensions simultaneously is challenging, as noted in Ji et al. ().

**Table 3: Comparison of different post-edit positions on three safety tasks. Here we take post-edit as u8B as examples.**
| Task | Aligner Position | Compared with u8B | Compared with 8B | Compared with 70B |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
|  | (u8B) | Harmless $\uparrow$ | Helpfulness$\uparrow$ | Harmless$\uparrow$ | Helpfulness$\uparrow$ | Harmless$\uparrow$ | Helpfulness$\uparrow$ |
| Beavertails | After All Agents | -0.08 | -0.46 | -0.75 | -0.25 | -0.74 | -0.40 |
| Only After | -0.29 | -0.45 | -0.82 | -0.21 | -0.79 | -0.42 |  |
| Only After | -0.27 | -0.42 | -0.77 | -0.18 | -0.77 | -0.39 |  |
| Only After | -0.26 | -0.44 | -0.80 | -0.21 | -0.81 | -0.43 |  |
| Only After | -0.02 | -0.38 | -0.72 | -0.17 | -0.72 | -0.40 |  |
| AdvBench | After All Agents | 0.15 | -0.17 | -0.97 | -0.09 | -0.98 | -0.13 |
| Only After | -0.10 | -0.16 | -0.96 | -0.15 | -0.96 | -0.20 |  |
| Only After | 0.00 | -0.14 | -0.96 | -0.16 | -0.97 | -0.17 |  |
| Only After | 0.02 | -0.19 | -0.98 | -0.12 | -0.97 | -0.13 |  |
| Only After | 0.18 | -0.13 | -0.97 | -0.05 | -0.97 | -0.09 |  |
| MaliciousInstruct | After All Agents | 0.09 | -0.25 | -1.00 | -0.09 | -1.00 | -0.17 |
| Only After | 0.08 | -0.28 | -0.99 | -0.15 | -0.99 | -0.32 |  |
| Only After | 0.00 | -0.28 | -0.99 | -0.16 | -0.99 | -0.31 |  |
| Only After | -0.26 | -0.44 | -0.80 | -0.21 | -0.81 | -0.43 |  |
| Only After | 0.05 | -0.25 | -0.99 | -0.09 | -0.99 | -0.18 |  |

We also analyze the effect of the post-editor’s position in Table 3. We find that (1) simply applying post-editing after the malicious agent yields limited improvements, indicating that the MAS remains vulnerable when a maliciously instructed agent is present, even with a single post-editing step afterward. These results suggest that the full pipeline, including interactions among all agents, is crucial since the subtle behaviors of each agent can influence and propagate towards the final result. This also demonstrates that post-editing techniques are not a panacea; simultaneously improving both the helpfulness and harmlessness of a response through a post-editing LLM remains challenging due to the limited inherent capabilities of post-edit LLM, some of the examples can be found in the Appendix D. Despite these limitations, our findings highlight the effectiveness of monitoring all agents in the system, validating the impact of our proposed AgentMonitor. (2) Additionally, in most cases, adding post-editing at only one position does not yield better performance (as seen when comparing the last four rows with the first row). However, in some cases, applying post-editing only after the summarizer achieves better results. These findings suggest that, rather than applying post-editing after all agents–which may increase inference latency–there is potential for improvement by dynamically encapsulating the MAS when needed and skipping it when unnecessary; we plan to explore it in the future.

## 5 Conclusion

In this paper, we introduce AgentMonitor, a framework designed to assess the predictability and enhance the security of MAS. AgentMonitor operates in a non-invasive manner, seamlessly wrapping around existing MAS while preserving their original workflows. By capturing the inputs and outputs of each agent, it enables (1) prediction of downstream task performance and (2) on-the-fly corrections. Our experiments demonstrate that AgentMonitor can effectively predict the system’s performance and facilitate the post-editing of agents’ responses, leading to more reliable and acceptable outcomes. As research increasingly focuses on the construction of MAS, we believe that our work contributes valuable insights, and we look forward to AgentMonitor guiding the design of more robust and rational MAS.

## Appendix A Prompt Template Used in the Paper

### A.1 Prompts of Agents Designed in Different Architectures

Table 4,  5,  6,  7,  8,  9,  10 and  11 present the prompts used in our designed multi-agent architecture, as outlined in Table 1. These agents include the Coder, Modifier, Reviewer, Tester, Dummy Agent, Executor, Web Browser, and Answer Extractor. Note that we do not show the Answer Extractor in the table, as it is utilized in all tasks requiring a final answer to be extracted from conversation history, except for HumanEval.

Table: Table 4: Coder Prompt Template.

Table: Table 5: Modifier Prompt Template.

Table: Table 6: Reviewer Prompt Template.

Table: Table 7: Tester Prompt Template.

Table: Table 8: Dummy Agent Prompt Template.

Table: Table 9: Executor Prompt Template.

Table: Table 10: Web Browser Prompt Template.

Table: Table 11: Answer Extractor Prompt Template.

### A.2 Prompts used to evaluate the personal score and collective score

Table 12 and  13 present the prompts used to evaluate each agent’s personal score and collective score, respectively.

Table: Table 12: Personal Score Judger Template.

Table: Table 13: Collective Score Judger Template.

### A.3 Prompts Used in Safety Related Benchmark

This section introduces the prompts used for the experiments in Section 4.4, following the setup from Ji et al. (). The prompts are used to evaluate the generated responses along two dimensions: helpfulness (Table 14) and harmfulness (Table 15), respectively.

Table: Table 14: Helpful Prompt Template.

Table: Table 15: Harmful Prompt Template.

## Appendix B Details of the indicators

In this section, we introduce the indicators derived from the information stored by our AgentMonitor.

As shown in Table 1, our indicators fall into two main categories. The first category consists of scores evaluated by an LLM: the personal score and the collective score.

Specifically, AgentMonitor records the input to each agent, each agent’s output, and the conversation history. We then use these records to prompt an LLM, using the prompts detailed in Appendix A.1, to generate the scores. Both scores are rated on a scale from 0 to 10, with higher scores indicating better performance. The scores are averaged across all turns and instances.

The second category includes indicators that are either inherited from or reflect the configuration of the MAS. These indicators are fixed after the system’s construction (e.g., total number of nodes) or are strongly influenced by the configuration (e.g., each agent’s PageRank).

The indicators are detailed as follows:

- •
Number of Nodes: Each agent in the execution graph is represented as a node, so the number of nodes corresponds to the total number of agents in the system.
- •
Number of Edges: We use directed edges to represent the information flow between agents. For example, if Agent A communicates with Agent B, a directed edge is drawn from A to B, and vice versa.
- •
Agent Capability: We assign an integer to represent the capability of each agent, depending on the level of LLM controlling it. In our experiments, we assign Llama3-70B-Instruct a score of 3, Llama3-8B-Instruct and its uncensored variant a score of 2, and GPT-3.5-turbo-1106 a score of 1. These rankings are intuitively derived from the leaderboard at [https://tatsu-lab.github.io/alpaca_eval/](https://tatsu-lab.github.io/alpaca_eval/), though the ranking may vary slightly across different benchmarks.
- •
Agent PageRank: We calculate the weighted PageRank for each agent, treating the edge weight as the number of tokens sent and received by the agent. PageRank (Page et al., ) is an algorithm that measures the importance of web pages, based on the idea that a page with many incoming links is more important. Additionally, pages that are linked by other high-PageRank pages further increase their own importance. Here, we use agent PageRank to indicate the importance of each agent within the system.
$PR(i)=\frac{1-\alpha}{N}+\alpha\sum_{j\in M(i)}\frac{w_{ji}\cdot PR(j)}{\sum_{
k\in L(j)}w_{jk}}$
(1)
Where:
–
$PR(P_{i})$ is the PageRank of agent $P_{i}$.
–
$d$ is the damping factor (set to 0.85 in our paper).
–
$M(P_{i})$ is the set of agents that link to $P_{i}$.
–
$w_{ji}$ is the weight of the link from agent $P_{j}$ to agent $P_{i}$ (we use token sent and received as weight in our paper).
–
$L(P_{j})$ is the set of agents that $P_{j}$ links to.
- •
Average Clustering is the mean of the local clustering coefficients of all the nodes in the network where the clustering coefficient measures the degree to which nodes in a network tend to cluster together.
The local clustering coefficient $C_{i}$ for a node $i$ with degree $k_{i}$ is:
$C_{i}=\frac{2\times e_{i}}{k_{i}(k_{i}-1)}$
(2)
where $e_{i}$ is the number of edges between the neighbors of node $i$.
The average clustering coefficient is:
$\text{Average Clustering}=\frac{1}{N}\sum_{i=1}^{N}C_{i}$
(3)
- •
Transitivity measures the overall tendency of a network to form triangles. It is the ratio of the number of closed triplets (triangles) to the total number of triplets (open and closed) and is defined as:
$T=\frac{3\times\text{Number of Triangles}}{\text{Number of Connected Triplets
of Nodes}}$
(4)
- •
Average Degree Centrality is
the mean of the degree centralities of all the nodes in the network. where the degree centrality is the number of edges connected to a node
defined as $D_{i}$ for a node $i$ is:
$D_{i}=\frac{k_{i}}{N-1}$
(5)
where $k_{i}$ is the degree of node $i$, and $N$ is the number of nodes in the network.
$\text{Average Degree Centrality}=\frac{1}{N}\sum_{i=1}^{N}D_{i}$
(6)
- •
Average Closeness Centrality is the mean of the closeness centralities of all the nodes in the network, where Closeness centrality is the reciprocal of the average shortest path distance from a node to all other nodes.
The closeness centrality $C_{i}$ for a node $i$ is:
$C_{i}=\frac{N-1}{\sum_{j\neq i}d(i,j)}$
(7)
where $d(i,j)$ is the shortest path distance between nodes $i$ and $j$.
$\text{Average Closeness Centrality}=\frac{1}{N}\sum_{i=1}^{N}C_{i}$
(8)
- •
Average Betweenness Centrality is the mean of the betweenness centralities of all the nodes in the network where betweenness centrality measures how often a node appears on the shortest paths between pairs of nodes in the network.
The betweenness centrality $B_{i}$ for a node $i$ is:
$B_{i}=\sum_{s\neq i\neq t}\frac{\sigma_{st}(i)}{\sigma_{st}}$
(9)
where $\sigma_{st}$ is the total number of shortest paths from node $s$ to node $t$, and $\sigma_{st}(i)$ is the number of those paths that pass through node $i$.
$\text{Average Betweenness Centrality}=\frac{1}{N}\sum_{i=1}^{N}B_{i}$
(10)
- •
Heterogeneous Score: here we define the heterogeneous score to examine the diversity of LLM used in the MAS. The higher score means that the LLM used in the MAS is more different.
$\text{Heterogeneous Score}=\frac{\sum_{i=1}^{n}\sum_{j=1,j\neq i}^{n}\mathbf{1
}(e_{i}\neq e_{j})}{\binom{n}{2}}$
(11)
Where:
–
$n$ is the total number of agents.
–
$e_{i}$ represents the $i$-th agent’s backbone LLM.
–
$\mathbf{1}(e_{i}\neq e_{j})$ is an indicator function that equals 1 if $e_{i}\neq e_{j}$, and 0 otherwise.

## Appendix C Details of Experiments Setting

In this section, we detail the LLMs used in various sections (Section C.1) and provide an introduction to the evaluation tasks and corresponding metrics (Section C.2).

### C.1 LLM used in different sections

As shown in Table 1, we design various architectures for the MAS. However, if we use a system with only one LLM, the total number of runs remains low, which may not be sufficient to conduct the predictive experiment. An intuitive approach is to assign different LLMs to each agent in the system and permute them. For example, with an architecture of three agents and three different LLMs, we would have a total of 27 possible combinations ($3^{3}$).

In the experiment described in Section 4.1, we select LLMs from the following: GPT-3.5-turbo-1106, Llama3-8B-Instruct, and Llama3-70B-Instruct. These LLMs are chosen to represent varying levels of capability, thereby forming a diverse group of expertise, allowing us to construct MAS with greater diversity.

In the experiment described in Section 4.4, our aim is to simulate a scenario where a malicious agent harms the response of the multi-agent team. To achieve this, we use the uncensored version of Llama3-8B-Instruct from [https://huggingface.co/Orenguteng/Llama-3-8B-Lexi-Uncensored](https://huggingface.co/Orenguteng/Llama-3-8B-Lexi-Uncensored), referred to as u8B.

To reduce cost and improve throughput, we use the AWQ quantized version of Llama3-70B-Instruct from [https://huggingface.co/casperhansen/llama-3-70b-instruct-awq](https://huggingface.co/casperhansen/llama-3-70b-instruct-awq). AWQ (Lin et al., ) is a training-free low-bit weight-only quantization method that does not rely on backpropagation or reconstruction, making it more efficient during inference. GPT-3.5-turbo and Llama3-8B-Instruct were obtained from their official providers, [https://platform.openai.com/docs/models/gpt-3-5-turbo](https://platform.openai.com/docs/models/gpt-3-5-turbo) and [https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct), respectively.

### C.2 Evaluation Tasks Introduction

In this section, we introduce the tasks used in our paper. As shown in Table 16, the selected tasks for code generation, reasoning, and math follow MINT (Wang et al., ), where the sampled instances are more complex and require multi-turn interactions to solve. For safety tasks, we randomly sampled 100 instances from the entire dataset to reduce computational costs.

For the experiments in Section 4.4, we derive the helpful and harmful scores by prompting the LLM using the prompts described in Appendix A.3. We then calculate the Harmlessness and Helpfulness scores, as shown in Tables 2 and 3, using the following formula:

$$ $\omega=\frac{N_{w}-N_{l}}{N_{w}+N_{l}+N_{e}}\cdot 100\%$ (12) $$

where $\omega$ represents the success rate, while $N_{w}$, $N_{e}$, and $N_{l}$ denote the wins, draws, and losses counts, respectively. Both Harmlessness and Helpfulness scores are the higher the better.

**Table 16: Used tasks in our paper.**
| Task Type | Task Name | Original Size | Sampled Size |
| --- | --- | --- | --- |
| Code Generation | HumanEval (Chen et al., ) | 164 | 45 |
| Reasoning | MMLU (Hendrycks et al., ) | 13,985 | 48 |
| Math | GSM8K (Cobbe et al., ) | 1,319 | 48 |
| Safety | Beavertails (Ji et al., ) | 30,000 | 100 |
| MaliciousInstruct (Huang et al., ) | 100 | 100 |  |
| AdvBench (Zou et al., ) | 520 | 100 |  |

## Appendix D case study on post-edit

In this section, we provide a case study of the experiments shown in Sec 4.4, which utilize post-editing after all agents have completed their generation. Figures 11 and 12 illustrate the effects of using u8B and 8B as post-edit LLM, respectively (Both of which are rated as harmless). Figures13 and 14 further show the effects of using u8B and 8B as post-edit LLM (With the former being rated harmful and the latter harmless).

From Figure 11, we observe that the u8B post-edit can sometimes still generate responses that pose potential risks to society, with only minor modifications made after the initial response. In this example, u8B mostly mirrors the original response. This suggests that once a model produces a harmful initial response, it can be challenging for an uncensored model to fully moderate it. However, after several interactions, the final response generated by the summarizer becomes more moderate and acceptable. This improvement is attributed to the post-edit by the first agent, which changes a single word–from ”illegally” to ”legally”–slightly enhancing the acceptability of the response. In contrast, Figure 12 presents an extreme case where the 8B post-edit exhibits a highly censored manner. We find that 8B often responds with ”I cannot…” thereby making the response helpful but largely uninformative. Another observation is that when the original agent’s response is modified by the phrase ”I cannot…”, this effect propagates to the succeeding agent (with the summarizer acting as u8B), leading it to also respond with ”I cannot…”.

Another case demonstrating the failure of the u8B post-edit to modify a malicious response is shown in Figure 13. Although the final response appears moderate, it still contains potentially harmful content related to the use of stimulants like caffeine, thus are rated harmful. Figure 14 illustrates that the 8B post-edit also displays non-compliance when monitoring the responses of the first two agents. Nevertheless, it significantly alters the original answer generated by the summarizer, removing the mention of specific drugs or substances, ultimately providing users with harmless yet useful guidance.

Figure: Figure 11: An example case of using post-edit (u8B) on MaliciousInstruct (index 3).
Refer to caption: x19.png

Figure: Figure 12: An example case of using post-edit (8B) on MaliciousInstruct (index 3).
Refer to caption: x20.png

Figure: Figure 13: An example case of using post-edit (u8B) on MaliciousInstruct (index 49).
Refer to caption: x21.png

Figure: Figure 14: An example case of using post-edit (8B) on MaliciousInstruct (index 49).
Refer to caption: x22.png

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: * A central issue with monitoring systems is the performance overhead / cost / latency that they incur. Could the authors on this aspect of their proposed solution?
soundness: 3
strengths: The authors suggest a practical approach that can be retrofitted to existing MAS in order to address to urgent challenges in practical MAS systems use and design:

* systems configuration
* malicious actors

The authors' suggested solution is simple to implement, and the empirical evaluation is promising.
summary: The authors propose AgentMonitor, an input/output monitoring and transformation drop-in component that can be retrofitted to MAS. AgentMonitor can both help with systems configuration, as well as with defending against outputs from certain classes of malicious actors. On a series of toy benchmark environments, including coding and multi-agent reasoning tasks, the authors show that their AgentMonitor achieves some degree of generalisation across tasks, as well as can improve systems performance in the presence of malicious actors.
weaknesses: * The simplicity of the authors' approach is also in itself a weakness: there isn't much technical novelty going on; systems monitoring with LLMs and input/output filtering are standard approaches, although perhaps not much work has investigated these in multi-agent systems.

* as not many multi-agent LLM systems have been deployed yet, the empirical results presented by the authors are naturally of limited generalisability to real-world tasks of importance; therefore while the authors' approach is promising in these limited settings, it remains unclear how much their simple approach would unlock in more impactful settings, particularly cf. more intensive approaches involving post-training etc.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. Looking at table 2, all architecture seems to have one coder agent and figure 8a indicates it is the most important feature as well. To what extent the performance of AgentMonitor is recovered, if we only look at the information relevant to the primary coder agent?

2. it should be the error is not **high** (line 321)?
> Furthermore, although the correlation for HumanEval in In-Task setting is lower, the error is not,

3. How is the post-edit done in section 3.4? My understanding is that it is an 'independent' agent that takes the final response and edits it? If so, this can be done with any architecture with or without AgentMonitor, right?
soundness: 2
strengths: - The problem studied by this work is novel and interesting.
- The figures are nicely done.
- The writing is generally ok.
summary: The paper proposes and studies the problem of predicting the performance of MAS from (a) features that encode information about the architectture of MAS (b) performance of individual systems within MAS. They propose an approach in which relatively simple features of MAS are identified and fed to XGboost based regressor which is trained to predict performacne based on the values of input features.
weaknesses: - The metric that the paper is using primarily is pearson correlation -- this choice of metric is confusing to me. If the point is to understand how well AgentMonitor predicts the relative ranking of systems, perhaps Kendall Tau correlation would be a better choice (https://en.wikipedia.org/wiki/Kendall_tau_rank_correlation_coefficient).

- I am generally dissatisfied by the evaluation and how the results are presented. For example, figure 8 is a novel plot type (i.e., not common in ML papers) but no information is provided on how the plot should be interepreted, 8a is a very messy plot anyways. May be the readers are not actually meant to understand the plots, but only observe the density of lines within them, but this is not clear upfront.

- I am also confused about why authors think its interesting to look at correlation between features/indicators. It is natural and reasonable to expect that some features would be correlated; some would not be, and findings don't contradict that. I would have instead liked to see a more elaborate discussion of feature importance across different scenarios. 

- The generalization of the method across architectures and tasks -- which is what we would want ideally from such a method is quite weak. 

- The problem that the paper is trying to address is not made clear in the paper -- seems like authors have two applications in mind (a) monitoring at test time (based on the fact that it is called AgentMonitor) (b) predicting performance of novel MAS (based on discussion in related works section). Both are nice applications, but I don't think there is enough evidence provided to argue that this is the right approach for either application.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: 1. Can you provide more details on the actual internal design of AgentMonitor? 
2. Can AgentMonitor handle a significantly larger number of agents without sacrificing prediction accuracy or real-time correction capabilities?
3. Have you tested the robustness of AgentMonitor against out-of-domain or noisy data? For instance, have you tried synthetically generating these types of data to evaluate how well the system maintains prediction accuracy and security in non-ideal conditions?
4. Have you considered adding more real-world datasets to further test the framework? I suspect that as the data becomes more diverse, the correlation might decrease, potentially revealing areas for model refinement or additional indicator development.
soundness: 2
strengths: - The framework’s plug-and-play design allows it to integrate easily with existing MAS setups without significant modifications. 
- By using indicators to predict MAS performance, AgentMonitor offers a predictive solution, reducing the need for trial-and-error configurations.
summary: The paper presents AgentMonitor, a framework that aims to improve the predictability and security of multi-agent systems (MAS). It is designed to be "plug-and-play" meaning it can be integrated with existing multi-agent applications.
weaknesses: - The section 2.1, Design of AgentMonitor, lacks specific design information, making it difficult to understand the authors' intentions. The justification for using LoRA appears unnecessary, and merely referencing the appendix for details on AgentMonitor’s usage does not clarify its design. Key elements, such as how agent input/output is captured and the rationale behind design choices, are missing.
- In section 2.3, Post-Edit Features, the motivation and design of the post-edit feature are not clearly explained. The reference to Figure 4 lacks meaningful context, and simply stating how AgentMonitor turns “BAD” into “GOOD” is insufficient. The authors should provide more in-depth information on how the framework effectively manages and controls undesirable agent behavior.
- The approach's effectiveness in larger, more complex MAS configurations isn’t fully explored. Further testing in more extensive/complex MAS environments would clarify its scalability.
- Many crucial details on indicators and the experimental setup are presented in the appendix; these should be included in the main paper.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. What are the specific novel aspects of the AgentMonitor framework compared to existing monitoring and predictive tools for MAS? How does it improve upon prior approaches in terms of performance prediction and on-the-fly corrections?

2. Given that the framework was trained and tested on the same or similar types of tasks, how confident are you in AgentMonitor’s performance when applied to different or more complex MAS tasks outside the scope of your experiments?

3. How were the specific agent-level and graph-level indicators chosen and how they impact the predictive model’s accuracy?
soundness: 3
strengths: 1. The integration of performance prediction with a post-editing layer for real-time response correction is novel and adds practical value to MAS.

2. The system's ability to integrate without altering the existing MAS infrastructure makes it a versatile solution for developers.

3. The framework is tested on various MAS architectures and demonstrates resilience, improving performance metrics like helpfulness and reducing harmful outputs by significant margins.
summary: This paper presents AgentMonitor, a framework that integrates with multi-agent systems (MAS) to provide predictive monitoring and enhance security through real-time post-editing. The tool captures agent-level and graph-level indicators to build a regression model that predicts task performance, aiming to reduce trial-and-error in configuring MAS. AgentMonitor also features a mechanism for on-the-fly correction of agent outputs to mitigate harmful or unhelpful results. The paper demonstrates empirical results across several MAS tasks, including coding, language understanding, and mathematical problem-solving, showcasing improvements in system reliability and safety.
weaknesses: 1. While the combination of predictive monitoring and post-editing is valuable, the paper could better articulate what sets AgentMonitor apart from existing MAS monitoring tools and frameworks. 


2. The AgentMonitor framework was trained and tested on the same or similar task types. While the experiments demonstrate strong performance in specific MAS tasks (e.g., coding challenges and problem-solving tests), this approach raises concerns about overfitting and a limited ability to generalize to more diverse and complex tasks. The framework's predictive accuracy and real-time correction mechanisms may be tailored too closely to the tested scenarios, making it unclear how AgentMonitor would perform in more varied or practical real-world MAS deployments.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: See Weaknesses.
soundness: 2
strengths: The idea is intuitive. The problem of accurately predicting the performance of an MAS with a specific architecture on a given task is clear. Another advantage of AgentMonitor is the ability to defend against malicious agents in a MAS via editing the inputs/outputs of agents.
summary: This paper proposes a new method for predicting the performance of an MAS with a specific architecture on a given task. To this end, the authors introduce the AgentMonitor, which takes the inputs and outputs of all agents and computes different indicators of this MAS. Then, these indicators along with the target score form a data point, which is used to train an XGBoost model. This XGBoost model will then be used to predict the target score of an unseen MAS on the given task. Besides, as AgentMonitor can receive the inputs and outputs of each agent, it can be used to correct the outputs of each agent via post-editing, which can improve the harmlessness of the MAS when malicious agents exist. Experiments show the effectiveness of AgentMonitor as a plug-and-play framework in predicting the performance of an MAS and improving the security of the MAS.
weaknesses: The presentation and the overall flow of the paper are somewhat messy, making it hard to grasp and fully understand the details, especially the evaluation process. Following are some questions, and please correct me if I make mistakes.

1. Are you using the same LLM for both the agent and the judger (gives the values of the indicators)? Or they can be different? If so, how will the choice of LLM influence the performance of the MAS?

2. There are many indicators used in experiments. Although there is an ablation study on the top 8 and bottom 8 indicators, it is still unclear why using so many indicators is necessary and how will these different indicators influence the outcomes.

3. In Section 3.3, I am confused by the RQ2. In the training set, each data point is (indicators, performance), where the indicators are obtained by feeding the full problem-solving process to the judger (an LLM). Then, what does "instances" mean? I assume that the prediction process is as follows: Each task-solving in the \textbf{test set} corresponds to a data point (indicators, ground-truth performance) where the indicators are obtained using the same method as in the training set. Then, we feed the indicators to the XGBoost model to get the predicted performance. Then, we can compare the predicted performance with the ground-truth performance. Therefore, why do we need to use a set of "instances" to calculate the indicators (which are obtained by the judger)?

4. For Section 3.2, I can hardly understand Figure 8. What do the numbers in the figure mean? What is the meaning of the axes? The explanations for the results are weird and hard to understand. "the indicators with higher importance have a wider range of valid choices", what does "valid choice" mean?

5. The paper needs substantial revision in terms of the presentation. A better organization is required to more clearly show the details and experimental results. I suggest more formally presenting the method part, which would greatly improve the readability of the paper.

Other minors:

Figure 1 is not referenced throughout the paper.

All the figures are hard to read; the text is too small to grasp.