reviews: 
Comments: This paper investigated how to improve the performance of LLMs structured in multi-agent systems in interactive tasks. To this end, it proposed a two-stage strategy: (1) Synthesizing data automatically and training the LLM agents with curriculum learning based on the data; (2) generating reward decomposition across both decision making steps and negotiation step among agents at each decision making step. The proposed approach has been shown to enjoy substantial performance improvement against open-source models and a comparable performance to closed-source models.
Strengths: 1. It is difficult to evaluate the originality of this paper, as it mainly combines multiple existing techniques together to form a strategy for a specific application on LLMs. From the perspective of software engineering and empirical study, this paper lies in the category of original work, but the novelty is limited. It only addressed some evident weaknesses with unsurprising empirical strategies. However, some hypotheses seems insightful, such as the hypothesis to explain the randomly generated edges on communicaton graphs.
2. As an empirical study paper, it focused on proposing some hypothesis and methodology to address a problem, which have been well validated by the experimental results with ablation study to emphasize the importance of each module proposed. For this reason, the overall quality of this paper is good.
3. The motivation of this paper has been well clarified, and the methodology description and experimental setups have been well stated. The experimental analysis is comprehensive and reasonable to justify the importance of the proposed approach.
4. The significance of this paper is not easy to evaluate. Standing from the view of LLM performance improvement, the strategies revealed can make some contribution to the research field (but still waiting for reproducing the results). From the perspective of methodology, I cannot see any novel ideas. Overall, I believe this paper may only attract attention in the domain of LLMs, but I seriously suspect if the result of this paper can give a long-standing impact to the mainstream of ML.
Weaknesses: 1. In the majority voting described in equation (5), how do you process the situation where two or more actions are tied to have the same counts? If the strategy is random selection, could you please show me if this kind of strategy will incur some fluctuation in performance?
2. In line 171-172, the sentence that "The proper size of local memory enhances the diversity of decision making and avoids introducing too long contexts," is not easy to comprehend. I cannot have a clear picture to link the local memory size with the diversity of decision makings. Could you give more explanation about it?
3. I scanned the paper, and have not noticed any discussion on the faithfulness of automatically generated data to the realistic situations. Without this guarantee, I cannot foresee the benefit of automatic generation of data for training LLMs, as the resulting harm to the society could be more severe than the improvement measured in quantitative metrics and paid human labours. How do you guarantee that the generated data is faithful to realistic situations, with no hallucinations?
4. During processing preference knowledge, you proposed to use SFT followed by DPO, but with no reason to explain this strategy. Could you please give more insights into this strategy?
Recommendation: 3
Reviewer confidence: 2
Soundness/correctness: 2
Clarity: 2