Potentially Optimal Joint Actions Recognition for Cooperative Multi-Agent Reinforcement Learning
Authors: Chang Huang
Shatong Zhu
Junqiao Zhao
Hongtu Zhou
Hai Zhang
Di Zhang
Chen Ye
Ziqiao Wang
Guang Chen
Venue: ICLR 2026 Poster (2026)
Keywords: Reinforcement Learning
Value function factorization
Multi-Agent
Primary area: reinforcement learning
TL;DR: Recovers the optimal joint policy by iteratively recognizing potentially optimal joint actions and assigning higher weights to them.

Abstract: Value function factorization is widely used in cooperative multi-agent reinforcement learning (MARL).
Existing approaches often impose monotonicity constraints between the joint action value and individual action values to enable decentralized execution.
However, such constraints limit the expressiveness of value factorization, restricting the range of joint action values that can be represented and hindering the learning of optimal policies.
To address this, we propose Potentially Optimal Joint Actions Weighting (POW), a method that ensures optimal policy recovery where existing approximate weighting strategies may fail.
POW iteratively identifies potentially optimal joint actions and assigns them higher training weights through a theoretically grounded iterative weighted training process. We prove that this mechanism guarantees recovery of the true optimal policy, overcoming the limitations of prior heuristic weighting strategies.
POW is architecture-agnostic and can be seamlessly integrated into existing value factorization algorithms.
Extensive experiments on matrix games, difficulty-enhanced predator-prey tasks, SMAC, SMACv2, and a highway-env intersection scenario show that POW substantially improves stability and consistently surpasses state-of-the-art value-based MARL methods.

Published as a conference paper at ICLR 2026
POTENTIALLY OPTIMAL JOINT ACTIONS RECOGNI- TION FOR COOPERATIVE MULTI-AGENT REINFORCE- MENT LEARNING
Chang Shatong Junqiao Zhao' Hongtu Zhou', Di Zhang', Hai Zhang",
Shatong
Chen Ye', Ziqiao Wang's Guang Chen
School of Computer Science and Technology, Tongji University
2 Stanford University
3 MOE Key Lab of Embedded System and Service Computing, Tongji University, Shanghai, China
The University of Hong Kong
5 Shanghai Innovation Institute
wow.jaanaotttonnngggoo.oo edu.cn cn
ABSTRACT
Value function factorization is widely used in cooperative multi-agent reinforce- ment learning (MARL). Existing approaches often impose monotonicity con- straints between the joint action value and individual action values to enable de- centralized execution. However, such constraints limit the expressiveness of value factorization, restricting the range of joint action values that can be represented and hindering the learning of optimal policies. To address this, we propose Po- tentially Optimal Joint Actions Weighting (POW), method that ensures optimal policy recovery where existing approximate weighting strategies may fail. POW iteratively identifies potentially optimal joint actions and assigns them higher training weights through theoretically grounded iterative weighted training pro- cess. We prove that this mechanism guarantees recovery of the true optimal pol- icy, overcoming the limitations of prior heuristic weighting strategies. POW is architecture-agnosiic and can be seamlessly integrated into existing value factor- ization algorithms. Extensive experiments on matrix games, difficulty-enhaneed predator-pre prey tasks. SMAC, SMACv2, and highway-env intersection scenario show that POW substantially improves stability and consistently surpasses state- of-the-art value- based MARI. methods.
INTRODUCTION
Multi-agent reinforcement learning (MARL) holds great potential for solving cooperative tasks in domains such as swarm roboties (Huang et al. 2020), autonomous driving (Schmidt et al., 2022). and multi-agent games (Terry et 2021). Yet, simultaneous policy learning for multiple agents remains challenging due to non-stationarity and the exponential growth of the joint action space. The centralized training with decentralized execution (CTDE) paradigm has become the standard framework for addressing these challenges, inspiring wide range of policy policy-based methods (e.g., MADDPG (Lowe al. (2017). COMA (Foerster et al.] 2018), FOP (Zhang et al. 2021)) and value- based methods (c.g., VDN (Sunchag et al. 2017), QMIX (Rashid et al. 2020a). QPLEX (Wang al. 2020
Among these, QMIX has achieved strong results on benchmarks such as the StarCraft Multi-Agent Challenge (SMAC) (Samvelyan al., 2019). QMIX factorizes the joint action-value into individual action-values using a monotonic mixing function, thereby ensuring decentralized execution. How- ever, the monotonicity constraint reduces the expressiveness of the value function, limiting its ability to represent many joint action values and often hindering optimal policy recovery.
'The first two authors contributed equally to this work.
1 "oorrsponndig Author

---

Published as a conference paper at ICLR 2026
To address this, WQMIX (Rashid et al. 2020b) proposed weighting joint actions during training, ideally emphasizing optimal ones so that the overall joint action-value function (Qtot) would ap- proximate the optimal target (Q"). However, identifying the truly optimal joint actions requires traversing the entire joint action space, which is intractable in realistic settings. Practical variants such CW-QMIX and OW-QMIX replace this exhaustive search with heuristic approximations. Specifically, CW-QMIX anchors its weighting on the arg max Qtot rather than the arg max Q", thereby avoiding enumeration of the full action space but introducing inaccuracies. OW-QMIX goes further by directly using Qiot values in an optimistic manner, which amplifies errors when judging whether an action is truly optimal. As result, both methods misalign the assigned weights with the actual optimal set: suboptimal actions may still receive large weights, while genuinely optimal ones can be under-emphasized. This creates a persistent gap between WQMIX's theoretical promise and its practical realizations.
We propose the Potentially Optimal Joint Actions Weighting (POW) method, which bridges this gap by introducing a recognition-based weighting scheme with provable convergence guarantees. POW employs a recognition module Q, that explicitly conditions on both state and joint actions, enabling to identify set of potentially optimal joint actions A,. Training weights are then signed adaptively, with higher weights for actions in Ar. Through iterative updates, we prove that A, converges to include the true optimal joint actions, ensuring that Qiot aligns its action pref- erences with those of Q" without requiring exhaustive search or heuristic approximations, This establishes, for the first time, a consistent link between the theoretical guarantces of weighted value decomposition and its practical implementation.
To validate POW, we instantiate on top of QMIX (yielding POW-QMIX) and evaluate across diverse benchmarks: matrix games, predator-prey, highway-env, SMAC, and SMACv2. Results show that POW- POW-QMIX outperforms sfate-of-the-art baselines, particularly in environments with non-monotonic reward structures where existing factorization methods struggle. We further demon- strate that POW can be seamlessly integrated into other value decomposition frameworks, such as VDN and QPLEX, consistently improving their performance. These results highlight both the ver- satility and scalability of our approach.
In summary, contributions are:
We propose POW, recognition-basee joint action weighting framework that provably bridges the gap between the theoretical guarantees of WQMIX and its practical realiza- tions.
We provide rigorous theoretical analysis, proving that the recognition module ensures vergence of the candidate set A, toward the true optimal joint actions, thereby enabling optimal policy recovery.
We conduct extensive experiments across five benchmark families, demonstrating that POW achieves superior performance over strong baselines and generalizes across multi- ple value factorization architectures.
2 PRELIMINARIES
We consider the standard decentralized partially observable Markov decision process (Dec-POMDP) Oliihhokk et al., 2016), defined by tuple (S, A, P, O O, 7), where S is the set of global states, A + A the transition function, r : S x + IR the reward function, O the set of individual observations, O S x {l.....m] n) + O the observation function, n the number of agents, and 7 E (0, 1) the discount factor.
A= -;==,A,, the joint action space, P
At each timestep f, the environment is in state s E S, and agent i selects action a, A, based on its action-observation history т; E (O, x A The joint action is a = (AI,..., an), leading to the next state $141 P(.|st, a. and team reward r(st, a).
A joint policy π = (MI....,,,, R.) defines each agent's policy n.. The objective is to maximize the expected discounted returm:
J(TT) Egg aon
Σ 2 r at)
2

---

Published as a conference paper at ICLR 2026
where p is the initial state distribution.
Centralized Training with Decentralized Execution (CTDE). In CTDE, training can leverage global state information, but execution requires each agent only on its local trajectory тІ. This motivates value function factorization methods, where the joint action-value function Qrae a) decomposed into individual utilities Qi(Ti,a;), A common factorization principle is individ- ual-global-mux (IGM) (Sunehag et al. 2017)
arg max Qtoe a) arg max as) (T), i=1
(2)
QMIX (Rashid et al. 2020a) enforces IGM by using monotonic mixing function:
aQ tot
0, 1...,
(3)
iQ.
Subsequent works such as QPLEX (Wang al. 2020 and QTRAN (Son et al. 2019) relax gen- eralize the decomposition principle, while others (c.g.. WQMIX (Rahidd et al. 2020b). CW-QMIX, OW-QMIX) assign weights to different joint actions during training. Despite progress, existing ap- proaches either suffer from limited expressiveness (e.g., strict monotonicity) or rely on heuristic weighting schemes that may introduce approximation errors. This motivates our proposed POW framework.
3 METHOD
Value decomposition methods under CTDE must factorize the joint action-value function into per- agent utilities. However, with monotonic mixing (e.g., QMIX), agent may still receive in- correct penalty if other agents act suboptimally, making it difficult to assign credit to the optimal joint actions. This motivates weighting schemes such as WQMIX (Rashid al. (020b), which ide- ally give higher training weights to optimal joint actions. Yet in practice, variants like CW-QMIX and OW-QMIX must approximate these weights without knowing the true optimal set, introduc- ing errors. POW addresses this challenge by learning a recognition-guider weighting scheme that provably converges toward the optimal set.
3.1 ARCHITECTURE OVERVIEW
(r,
Q, a) for - v0 20((( (P. arg max (t))
Fiuus Quar (t,
Azot
Q.
Qent
V(+)
W;
MIPP
ML.P
W
MLP
A (r, (T()))
an a Qn an)
Agent
Agent N
(b)
Rackpoopaationn
Bodkgropagation
Figure 1: (a) The Q, Network structure. (b) The overall architecture of POW method. Qtot can be any value function factorization network satisfying IGM.
Fig. Illustratee the POW framework.
Key components are: (1) Q*, an unrestricted joint action value estimator that approximates the true optimal action value function Q" without factorization or monotonic constraints. It provides the bootstrap target shared by all networks during training. (2) Qtot- monotonic mixing network enabling decentralized execution. Its optimality depends on correct weighting of optimal vs. sub- optimal joint actions during leamnin. It be any value factorization network satisfying IGM
3

---

Published as a conference paper at ICLR 2026
(e.g., QMIX, VDN, QPLEX); (3) the potentially optimal joint actions recognition module Qr. It is used identify set of potentially optimal joint actions Ar- Qr is trained approximate Q" (or Q" in theoretical analysis), and its output determines the adaptive training weights applied to each joint action. It takes the joint action input and provides expressive joint action value model unconstrained by monotonicity but conforming to IGM.
The architecture of Qr is shown in Fig. a). The inputs of Q. include: the global state 8, the one-hot encoding of joint action, a and the fixed values of individual advantage functions A,, This joint-action conditioning crucial for distinguishing candidate actions, whereas in QPLEX it pri- marily increases expressiveness of Qtot- By contrast, in POW this design is tied directly to the recognition-weit weighttng mechanism and its convergence properties (see Sec. 3.2). Atot refers to the mixing of the individual agent advantage functions as in QPLEX. The advantage function is defindd by AT(T, 0;) = Qi(Ti, a.) maXa,e (T (T))
Q', Qrot and Qr (detailed in Sec. 3.2 and Sec. 3.3
share the same Q-learning target:
Q- E[((" (T, a) y)
(4a)
LQ E[w(s, a)(Qro((T (T, a) -)
(4b)
Lq- E[(Q,(T, a) v)
(4c)
where
= r (T arg max Qroe(T', , a))
(5)
aE
Together, Q-, Qiot and Q, form mutually reinforcing system: Q, proposes potentially optimal
actions, the weighting guided by Ar shapes the update of Qtor, and Q' ensures consistent bootstrap- ping. We later show that this interaction guarantees the convergence of A, toward the true optimal action set.
3.2 RECOGNITION OF POTENTIALLY OPTIMAL JOINT ACTIONS
We define the recognition module Qr that explicitly takes as input the global state s, individual action-values (T, a;), and the joint action a. Here, conditioning on allows Q, to assess the value of specific joint actions, enabling recognition of candidate set A, of potentially optimal joint actions. Formally, the recognition module is defined as:
Qr(r,a)=
a) = A, - Issss (Qi(t,)))) max Qi(Tr,a)) + V(s), a.EA;
(6)
where )t(s, a) are scaling factors. The subtraction term centers each agent's action-value by its best individual choice, while (s)) captures state-dependent value shared across agents. Intuitively. this form highlights whether a joint action sacrifices individual agent optimality, while allowing Q- to adaptively weight such trade-offs.
Importantly, this construction also guarantees the IGM property. Since the contribution of each agent i is maximized exactly when is its individually optimal action (the centered term becomes zero and all other actions are negative), maximizing Qr, a) over a is achieved by maximizing each Q, o;) independently. Thus ensuring IGM without enforcing any monotonicity constraint on the underlying Q;-
The training objective of Q, is to approximate the optimal joint action value function Q* in theory:
fo. = (E((((((Ta) Q" ())
During training, updates are applied to the parameters of the mixing function, leaving the parameters of the individual action value functions unchanged. The scales Xr(s, a) are computed by hypemet- work, where the global state and the joint action a are used as inputs to obtain the neural network weights Wi and W2. We take the absolute values of W, and Wa ensure that Ar a) ' 0.
4

---

Published as a conference paper at ICLR 2026
Definition 1 (Potentially optimal joint action set A,). We define Aigm := fa E A vi as arg max Qi(Ti, a:)) be the set joint action obtained by greedy individual choices. Let Aigms a,EA, then the potentially optimal joint action set is:
A, == {a E A Qr(s, a) 2 Qr(s, -C},
(8)
where C' 2 O is a small tolerance constant for stability.
This definition ensures Ar always contains at least the joint greedy action and potentially other promising joint actions.
Theorem (Containment of optimal joint actions). Let Atgm = {a E A a
arg max Q" (s, })} denote the set of truly optimal joint actions. If Qr converges to Q", then aEA
Argm C Ap.
Thus Ar is guaranteed not to exclude optimal actions, which is critical for policy improvement. Proofs are given Appendix
3.3 RECOGNITION- WEIGHTING FUNCTION
We now define the POW weighting function:
w(s, a) = 11, a a Ar,
(9)
aa, a a A a [0,
where a down-weights actions outside A,. all our experiments, we set a = 0, so only actions in A, contribute updates, aligning theory with practice. The training objective for the factorized value network Qoo is then:
LQ.. = E[w(s, (Q - -
(10)
with target
y V= arg max aeA
Qtorls' a)),
(11)
where Q" is an unrestricted joint value estimator used to approximate Q" Theorem (Convergence of weighted training). Under the weighting scheme in Eqn. 9 A, verges to contain only optimal joint actions, then Qtot recovers the optimal policy.
Qrat can recover the joint action with the maximal value of Q-, that is, when arg max Qroe = arg max (T' Q' becomes the optimal joint action value function acA
a
Q' according to the Bellman equations indicated by Eqn. and Eqn. Thus Qiot can learn optimal policy.
Detailed proofs are given in Appendix
3.4 ITERATIVE WEIGHTED TRAINING
POW proceeds iteratively: (1) update Qr toward approximating Q using supervised targets, (2) update Qtot using the weighting w(s, a) defined by the current A,, and (3) update Q based the updated Qzor- This recognition- weighting loop continues throughout training. Unlike heuristic approximations in CW-QMIX or OW-QMIX, this iterative scheme ensures that A, progressively contracts toward the true optimal set, closing the gap between theoretical guarantees and practical implementation. The pseudocode is provided in Alg. T
EXPERIMENTS
In this section, instantiate framework with QMI and propose the POW-QMIX algorithm. We first evaluate on matrix games and a defficulty-ehhaneee predator-pre prey task, both of which ex- hibit strong non-monotonicity that challenges monotonic value factorization. We then test the
5

---

end for Published as a conference paper at ICLR 2026
Algorithm POW Training Procedure
Require: Replay buffer D; value networks Qua, Qr, Q"; tolerance C
1: Initialize all network parameters
2: for each training iteration do
3: Sample a batch of episodes B = (.00:r,A1:::::::))) from D
4: Input observations into agent networks generate histories:
5: H. = {(T1, a1. : Tr, ar, TT, T where T/ = T++1
6:
for each time step and episode in batch do
7:
Compute greedy next action under factorized critic by arg maxa Qua(T',))
8:
Compute TD target y shared by crities by Eqn. 11
9:
Update recognition network Q. by minimizing Eqn. 4c
10:
Determine greedy action for recognition module under IGM: arg maxa (r(T,
11:
Compute training weight based Eqn. and Eqn.
12:
Update factorized critic Qwo by Eqn. 10
13:
Update unconstrained value estimator Q" by Eqn. 4a
end for
15:
return Qiot
SMAC, a widely used but relatively monotonic benchmark. Finally, we extend the evaluation to SMACv2 and highway-env intersection scenario.
We also conduct ablations examine (i) the applicability POW other value decomposition methods, and (ii) the effect of increased network size. All experiments are implemented using PyMARL2 (Hu et al., 202Tb. Hyperparameters such as optimizer type and replay buffer size are tuned for each method. Further details are provided in Appendix [F] All results are averaged over five independent runs with different random seeds and are reported with means and 95% 55% confidence intervals.
4.1
MATRIX GAME
A
0.069(A)
.0.169(B)
-0,045(C)
Q.
0.060(A)
-0.160(3)
-0.045(C)
0.041(A)
7.96
0.041(A)
-12
-0,550000
7.95
7,90
7.93
-0.150000
-12,00
0.00
0.00
-12
7.9
0051[[]]
7.98
7.52
7.95
0.051(C)
-12.00
0.00
7.90
(a) Payoll Matrix
(b) POW-QMIX Q, Q2 Qzo
[c] POW-QMIX: Q, Qu.
4
22.90(A)
-0.12(5))
Q2
081114
0.133(B)
0.912(C)
Q2
0.060(A)
-0.160(3)
-0045(2)
-23.29(A)
-8.11
18.10
-8.11
0.835(A)
16.27
12.67
16.70
0.041(A)
8.00
7.95
7.97
.0.141(8)
.8.10
.0.33
0.15
0.12013)
13.21
9.62
13.63
.0.150[8)
7.95
7:90
7.93
0.091(()
0.906(C)
00051CC)
(d) QMIX:
Qui
(c) DW QWW.OMMX:OO
Qut
DW LW-QMIX Qu Qz Qws
0.319(A)
-1.205(3)
0.004(()
0.159(A)
-.367(8)
0.143(())
0.15%(A)
-0.367(3)
0.243(C)
-0.31(4)
0.68
-12.77
-14.52
0.155(A)
8.04
6,99
8.11
0.155(A)
0.04
-11.58
-11.42
-1.10000)
-0.332(00
..006(()
-10.64
.0.38
9.69
0.10KC]
7:99
6.96
7.97
0150(C)
-11.12
0.23
7.94
(5) QPLEX Qu. Qu. Quot
(b) ResQ: Qu, Qu. Qut
(0 ResQ Qu. Qz Qu
Figure 2: Payoff matrix of one-step matrix game and reconstructed joint and individual values. Boldface indicates greedy actions. Blue denotes the true optimal joint action, red denotes suboptimal joint actions.
We begin with a coordination matrix game exhibiting strong non non-mononee monotonicity. following the setting 1 in e-greedy, producing
of ResQ. To remove effects of exploration randomness,

---

Published as a conference paper at ICLR 2026
uniform data distribution. After convergence, record the learned joint and individual values (including Qr for POW-QMIX and Q5 for ResQ), shown in Fig.
POW-QMIX, CW-QMIX, and ResQ successfully recover the optimal policy. In POW-QMIX, the Qr module precisely estimates the values of all joint actions, enabling accurate recognition of the optimal set A, for weighting. In contrast, QMIX converges to locally optimal solution due to monotonicity. OW-QMIX also fails, reflecting its approximation limitations. QPLEX shows partial overestimation of the optimal joint action while remaining accurate elsewhere-an an observation that inspired our Q, design.
4.2 PREDATOR-PREY
FOW CMIX
(b) pa-4
(c) pw-5
Retum
20
36
-10
251
-23
0N
-10
Timesteps
Timestops
Timestess
Figure 3: Test return in Predator- with three different mis-capture penalties.
In this task, predators must cooperate to capture prey. If an agent attempts capture without coordi- nation, all agents receive penalty p. Higher [p] increases non-monotonicity and encourages passive strategies.
Fig. shows test returns under three penalties. POW-QMIX is the only method that consistently learns the optimal cooperative strategy across all settings. This highlights its ability to resolve non- monotonic structures where baselines fail.
4.3
SMAC
POW-QMIX (ours)
QMIX
OW-QMIX
CW-OMIX
OPLEX
ResQ
(a) 3652 (eapy)
[b) 3s vs. 52 (hard)
(c) 6h vs. 82 (supen hard)
1.0
2.5
1.0
7est Vin Rutte
0.0
0.6
0.4
0.4
0.0
0.0
5M
OM
0M
Timesteps
Timesteps
corridor (superhard)
MMM2 (superhard)
3552
3062
1.0
Test Wi2 Rate
0.0
0.0
0.6
Test Win
0.6
6.0
0M 1M 2M 3M 4M 5M 6M 7M 8M 9M 10M
0M 3M 2M 3M 4M 5M 6M 7M 8M 3M 10M
0M
1M
2M 3M 4M 5M BM 7M BM 9M 10M
Timesteps
Timesteps
Timesteos
Figure 4: Test win rate on SMAC maps.
We evaluate six SMAC maps: one easy, one hard, and four super-hard. Fig. 4 shows that POW- OMIX achieves strong performance across maps. Although SMAC is mostly monotonic (Hu et al. 2021), POW-QMIX still matches outperforms baselines. CW-QMIX, while successful in matrix games, struggles to scale here. QPLEX exhibits instability due to its dueling architecture. OW. QMIX performs well in SMAC but lacks theoretical guarantees, as shown in Fig.
7

---

Published as a conference paper at ICLR 2026
4.4 EVALUATION ON HIGHWAY-ENT ENV INTERSECTION AND SMACV2
We include experiments on the highway-eny intersection task Leurret 2018 and on SMACv2 (Ellis al., 2023). These benchmarks introduce safety-critical decision making and generalization challenges, complementing the main results.
PONACMMI (ound)
DRIX
DN--OMIX
CIANNIX
Reel
Trat Starrees Rass
Text Pasuun Meknn
Tool Rutu Moon
Tosl 500000
3M
Timesteps
Timestape
Figure 5: Test return in the highway-env intersection scenario.
As shown in Fig. POW-QMIX achieves the best overall performance, successfully balancing safety and efficiency. In comparison, CW-QMIX converges to overly conservative policies, QPLEX suffers from training instability, and QMIX learns much more slowly. These results highlight POW- QMIX's superior ability to handle strongly non-monotonic environments.
GMIX
(5) piotoss__ sunt
(bI Icrran 5, unts
(0) zorp.rrp F. unis
Text Ritumm
0M
5M 10M
4M Timeslops
CM
BM
Timasta00
Timesiace
Figure Test return the SMACv2 benchmarks.
SMACv2 presents more subtle challenges: as win rates of different algorithms often saturate and appear indistinguishable, we adopt average test return as the evaluation metric. Fig. 6 shows that POW-QMIX consistently achieves strong performance across most tasks. While QPLEX outper- forms POW-QMIX -W-II in the protoss scenario, it collapses in the zerg scenario. Notably, the ablation results in Sec. [.5.TT that POW-QPLEX successfully stabilizes QPLEX, confirming that POW's benefits extend beyond QMIX.
4.5
ABLATION STUDIES
4.5.1
APPLYING POW TO VDN AND QPLEX
We integrate POW into VDN and QPLEX, producing POW-VDN and POW-QPLEX. Tab, summa- rizes results across all emvironments, with detailed learning curves in Appendix E In predator prey, baseline networks fail to learn, but their POW variants converge quickly to optimal policies. In the highway-env crossroad task, POW greatly improves stability and success rates. On SMAC, POW
8

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison environments. Crossroads and SMAC values win rates; Predator-Prey and SMACv2 values are returns. 1 indicates improvement over the baseline. Bold indicates best performance.
Algorithm Predator-Pree
SMAC
SMACv2
Crossroads
38 As.VS.. 3s.v
corridor
MMM2
protoss
terran
zerg
QMIX
0.28
0.69
18.3
17.1
17.6
VDN
0.73
0,97
0.87
0.81
17.5
17.0
15.5
QPLEX
0.26
0.96
0.30
0.00
19.2
17.3
ResQ
0.88
1.00
0.00
0.90
17.7
16.3
17.5
CW-QMIX
16
0.43
0.99
0.79
0.92
17.2
16.0
15.7
OW-QMIX
0.88
1.00
0.70
0.98
18.4
16.3
16.9
POW-QMIX
40
40
0.92
1.00
0.95
0.98
18.8 f
19.0
18,4
POW-VDN
40
40
0.81
0.96
0.87
0.90
17.9 T
17.0
16.8
POW-QPLEX
40
40
0.93
1.00
0.94
0.93 f
19.9
19.4
18.1
reduces QPLEX instability, while in SMACv2, all POW variants consistently improve average turns.
4.5.2
ENLARGING THE NETWORK SIZE
POW QNIX B00K
OMIX 800K
OW B00K
CW QNIX 800k
OPLEX BOOK
20
100
10
0.4
0.2
-00
-10
5.0
GH
Timeslagss
Tineslaps
Figure 7: Ablation: effect of network size. (a.b) Predator Prey with p = -4, -5. (c) SMAC map.
To test whether performance gains come simply from added capacity. we enlarge baseline networks to match POW's parameter count. Fig. 7 shows that larger networks improve some baselines (CW- QMIX, OW-QMIX) in predator-prey but hurt in SMAC. Enhanced QMIX improves in SMAC but still fails under non mon-tonot.n, QPLEX performs poorly regardless of size. Thus, gains of POW stem from its recognition weitthinn design, not from parameter count. While Q, adds moderate complexity (roughly 15-20% training time overhead), POW achieves stronger performance across all tasks. We therefore describe POW effective trade-off between computational cost and policy quality.
RELATED WORK
Value decomposition is the predominant paradigm under CTDE. VDN (Sunehag et al. 2017) assumes additivity, whereas QMIX (Rashid et al. 2020a) introduces monotonic mixing network. QPLEX (Wang al., 2020 improves expressiveness via a dueling structure and advantage-bases mixing.
Value decomposition in MARL.
WQMIX (Rashid al., 2020b) reveals the limitation of uniform weighting and proposes an ide- objective. However, practical variants (CW-QMIX, OW-QMIX) must approximate the optimal action set and therefore cannot guarantee correctness the assigned weights. Our method is most closely related to WQMIX but differs in two key respects: (1) POW replaces heuristic weighting with a recognition weighting mechanism that provably converges to- ward the optimal joint action (2) our recognition module Q, explicitly conditions on the joint action a, enabling reliable discrimination among actions, unlike QPLEX where joint-action inputs primarily to enhance representational capacity. Thus, POW resolves the gap between theoreti-
alized hatpmmaallattoowieeeeeee

---

Published as a conference paper at ICLR 2026
cal guarantees and practical realizations that neither WQMIX nor QPLEX addresses. More detailed comparisons are provided in Appendix A
Beyond these classic methods, CIA (Liu et al. 2023) introduces contrastive identity-aware represen- tation learning to improve credit assignment, and VDT (Zhao et al.) 2025) leverages transformers to exploit temporal structure in multi-agent trajectories. Although effective, these methods are orthog- onal to our focus: they enhance representation quality or temporal modeling rather than addressing the cheoretical-practicll mismatch in weighted value decomposition. Therefore, they do not directly evaluate the specific problem POW aims to solve.
Approaches such REMIX (Mei al. 2023) and concaveQ (Li et al. 2023 introduce alternative structural assumptions (c.g.. concavity or regularization). Our method differs by maintaining gener- ality and instead rethinking how potentially optimal joint actions can be recognized and up-weighted during training.
Approximation error in value-based MARI.. Several works highlight approximation error as a central challenge. ResQ (Shen et al. 2022) mitigates representational bias by injecting joint-action terms, while CW-QMIX and OW-QMIX approximate weighted learning heuristically and lack op- timality guarantees. POW shares the motivation of reducing approximation error but introduces new mechanism-joon joint--oonttiiionoo action conditioning via Q, and iterative recognition-gue guided weighting that ensures optimal actions retained without requiring exhaustive search.
Beyond value decomposition. Actor-critic MARL. methods such as MADDPG (Lowe et al., 2017) and MAPPO (Yu al., (022) do not rely on value factorization but instead employ joint crit- ics or attention-baa based critics to stabilize training. These methods differ fundamentally from value decomposition and excel in continuous-action or competitive settings. Our focus is on cooperative discrete-action tasks, where value decomposition remains the most effective and widely used ap- proach. Nevertheless, POW can be viewed complementary to actor critic MARL, both aim to identify joint action structures that improve stability and performance.
6 CONCLUSIONS AND LIMITATIONS
We introduced Potentially Optimal Joint Actions Weighting (POW), an iterative weighted train- ing framework for cooperative multi-agent reinforcement learning. POW leverages a recognition module Q, to identify potentially optimal joint actions and guides training by adaptively weighting them. We formally proved that under this scheme, the recognized set converges to the true optimal joint actions, ensuring that Qiot recovers the optimal policy. Extensive experiments across matrix games, predator-prey, SMAC, SMACv2, and highway-env confirm that POW not only matches its theoretical guarantees but also achieves superior empirical performance over strong baselines.
Despite these advantages, POW introduces additional modules to address non-monotonicity. which increase training complexity in large-scale environments. Moreover, our current study is limited to cooperative settings with discrete action spaces under CTDE. Extending POW to policy-gradient or actor-critic frameworks (e.g., MAPPO) would broaden its applicability to continuous control and mixed cooperative-competitiy domains.
REFERENCES
Yixin Huang, Shufan Wu, Zhongeheng Mu, Xiangyu Long, Sunhao Chu, and Guohong Zhao. A multi-agent reinforcement learning method for swarm robots in space collaborative exploration. In 2020 6th international conference on control, automation and robotics (ICCAR), pages 139- 144. IEEE, 2020.
Lukas M Schmidt, Johanna Brosig, Axel Plinge, Bjoemn M Eskofier, and Christopher Mutschler. An introduction to multi-agent reinforcement learning and review of its application to autonomous mobility. In 2022 IEEE 25th International Conference Intelligent Transportation Systems (ITSC), pages 1342-1349. IEEE, 2022.
J Terry, Benjamin Black, Nathaniel Grammel, Mario Jayakumar, Ananth Hari, Ryan Sullivan, Luis Santos, Clemens Dieffendahl, Caroline Horsch, Rodrigo Perez- Vieez-Vicente, et al. Pettingzoo: Gym
10

---

Published as a conference paper at ICLR 2026
for multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 34: 1503,,1503,, 2021.
Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAl Picter Abbeel, and Igor Mordatch. Multi- agent actor-critic for mixed coperaatie-coompettive environments. Advances in neural informa- tion processing systems, 30, 2017.
Jakob Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. Counterfactual multi-agent policy gradients. In Proceedings of the AAAI conference artificial intelligence, volume 32, 2018.
Tianhao Zhang. Yucheng Li, Chen Wang, Guangming Xie, and Zongqing Lu. Fop: Factorizing optimal joint policy of maximum-entropy multi-agent reinforcement learning. In International Conference on Machine Learning, pages 12491-12500. PMLR, 2021.
Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojeiech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Mare Lanctot, Nicolas Sonnerat, Joel Z Leibo, Karl Tuyls, et al. calo--decomposition networks for cooperative multi-agent learning. arXiv preprint arXiv:/706.055:
2017.
Tabish Rashid, Mikayel Samvelyan, Christian Schroeder De Witt, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. Monotonic value function factorisation for deep multi-agent reinforcement learning. The Journal of Machine Learning Reseurch, 21 21(1):7234-2284, 2020a.
Jianhao Wang, Zhizhou Ren, Terry Liu, Yang Yu, and Chongjie Zhang. Qplex: Duplex ducling
multi-agent q-learning. arXiv preprint arXiv:2008. 01062, 2020.
Mikayel Samvelyan, Tabish Rashid, Christian Schroeder De Witt, Gregory Farquhar, Nantas Nardelli, Tim GJ Rudner, Chia-Man Hung, Philip HS Torr, Jakob Foerster, and Shimon Whiteson. The starcraft multi-agent challenge. arXiv preprint IrrXiv::902.000 04043, 2019.
Tabish Rashid, Gregory Farquhar, Bei Peng, and Shimon Whiteson. Weighted qmix: Expanding monotonic value function factorisation for deep multi-agen reinforcement learning, Advances in neural information processing systems, 33:10199-10210, 2020b.
Frans A Oliehoek, Christopher Amato, al. A concise introduction decentralized POMDPs, volume 1. Springer, 2016.
Kyunghwan Son, Daewoo Kim, Wan Ju Kang, David Earl Hostallero, and Yung Yi. Qtran: Learning to factorize with transformation for cooperative multi-agent reinforcement learning. In Interna- tional conference on machine learning, pages 5887-5896. PMLR, 2019.
Jian Hu, Siyang Jiang, Seth Austin Harding, Haibin Wu, and Shih-wei Liao. Rethinking the imple- mentation tricks and monotonicity constraint in cooperative multi-agent reinforcement learning. arXiv preprint arXiv:2 2021.
Edouard Leurent. An environment for autonomous driving decision decision-making https://githin
com/ eleurent
eleurent/aiggwwyeeen /wwigwawwhwwwwyeeeee 2018
Benjamin Ellis, Jonathan Cook, Skander Moalla, Mikayel Samvelyan, Mingfei Sun, Anuj Mahajan, Jakob Nicolaus Foerster, and Shimon Whiteson. SMACv2: An improved benchmark for coop- erative multi-agent reinforcement learning. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
Shunyu Liu, Yihe Zhou, Jie Song, Tongya Zheng, Kaixuan Chen, Tongtian Zhu, Zunlei Feng, and Mingli Song. Contrastive identity-aware learning for multi-agent value decomposition. Pro- ceedings of the AAAI Conference Artificial Intelligence, 37(10): Jun. 2023. doi: (0.1609//aaa..3311022223 article/view/2551
aaai, URL httpp:://oo...a....ooo///.n wwwp/d/ww.pppppppapeaepaa AAAI//
Zhitong Zhao, Ya Zhang, Wenyu Chen, Fan Zhang, Siying Wang, and Yang Zhou. Sequence value decomposition transformer for cooperative multi-agent reinforcement learning. Infor-
mation Sciences, 720:122514, 2025. ISSN 0020-0255. doi: http:///oi.org/110001//// 4025.122514. URL https: htttp:://ww..e.. wttp://wwwwiiiiiieeeee.......... http://wwww.eeeeeeeee//eeeeeeee/ 5062002552200044

---

Published as a conference paper at ICLR 2026
Yongsheng Mei, Hanhan Zhou, and Tian Lan. Remix: Regret minimization for monotonic value function factorization in multiagent reinforcement learning. arXiv preprint arXiv:2302.05593, 2023.
Huiqun Li, Hanhan Zhou, Yifei Zou, Dongxiao Yu, and Tian Lan. ConcaveQ: Non-Monotonic Value Function Factorization via Concave Representations in Deep Multi-Agent Reinforcement Learning, December 2023.
Siqi Shen, Mengwei Qiu, Jun Liu, Weiquan Liu, Yongquan Fu, Xinwang Liu, and Cheng Wang- Resq: A residual q function-based approach for multi-agent reinforcement learning value factor- ization. Advances in Neural Information Processing Systems, 35:5471-5483, 2022.
Chao Yu, Akash Velu, Eugene Vinitsky. Yu Wang, Alexandre Bayen, and Yi Wu. The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games, 2022. URL http:ttp:/// htttt:tttp::/iaie /ttp:::////xxvvor///
0as/2103.01955
Chang Huang, Junqiao Zhao, Hongtu Zhou, Hai Zhang, Xiao Zhang, and Chen Ye. Multi-agent decision-man making at unsignalized intersections with reinforcement learning from demonstrations. In 2023 IEEE Intelligent Vehicles Symposium (IV), pages 1-6, 2023. doi: 10.1109///55555.22222. 10186792.
12

---

Published as a conference paper at ICLR 2026
A RELATIONSHIP TO RELATED WORKS
This section clarifies the relationship of POW to prior value decomposition MARL methods. Al- though POW draws inspiration from both WQMIX and QPLEX, its design addresses their key lim- itations and provides distinct contributions.
POW WQMIX. WQMIX (Rashid et al., 2020b) proposes idealized scheme in which op- timal joint actions are assigned higher training weights. In principle, this enables recovery of the optimal value function. However, its practical implementations (CW-QMIX, OW-QMIX) must ap- proximate the optimal actions using heuristic strategies, which introduces unavoidable approxima- tion error and prevents strong guarantees. POW differs fundamentally: we introduce a recognition module Q, that explicitly incorporates joint actions inputs and adaptively identifies set of po- tentially optimal joint actions A,. The weighting function then prioritizes actions in A,,. We provide theoretical analysis (Theorem and 2) showing that A, converges toward containing only optimal actions, thereby eliminating the approximation gap present in WQMIX. Thus, POW achieves the theoretical guarantee envisioned by WQMIX without resorting to exhaustive search or heuristic ap- proximations.
POW vs. QPLEX. QPLEX (Wang al. 2020) enhances representational capacity through dueling-basee decomposition that incorporates joint-acttion-deeeedden advantage terms. Although this also conditions on joint actions, the primary goal to increase expressiveness of the value function rather than to guide training dynamics. By contrast, POW leverages joint action inputs within Q- for a fundamentally different purpose: recognizing potentially optimal joint actions and using them to drive a principled weighting scheme. Ablation results (Tab. and Appendix confirm that POW's performance gains cannot be explained merely by including joint action information. Instead, they arise from the recognifion-weighting mechanism, which explicitly aligns the training process with the optimal joint value function and provides theoretical guarantees absent in QPLEX.
POW vs. ResQ. ResQ (Shen et al. 2022) introduces auxiliary joint-action value term to reduce representational bias. However, its objective remains to approximate Qtot without targeted weight- ing of potentially optimal joint actions. POW differs by explicitly reweighting the learning process toward recognized optimal actions, offering a more direct mechanism to recover optimal policies. As shown in our experiments (Fig. Tab. 1 POW outperforms ResQ in environments with strong non-monotonicity.
B PROOF OF THEOREMS
In this section we provide detailed proofs of the main theoretical results. Compared with the original WQMIX analysis, our derivations clarify why the proposed recognition module Q avoids approx- imation errors, and how the recighition-weighting mechanism guarantees recovery of the optimal policy. We explicitly restate all assumptions to avoid ambiguity.
C PROOF THEOREMS
C.I
LEMMA 1
For any 7 and joint action a & Aigm. if Qr has converged, it holds that
Qr(T, a) min(Qr(T, (T,
Proof. By the definition of Q,, for any & Aigm we have
@r( ())
The local loss for each joint action is
Lq. (Qr(T, - a))
Consider cases:
13

---

Published as a conference paper at ICLR 2026
(f 2 Q+(T.)): a), then
(Qr(T.a) - (T, - ( a))
Minimizing Le. (T, a) requires Q,(T, a) to be as large as possible under the constraint
Qr(T,a) - (T(T, a), yielding
Qr(T, Q)(T,a) mi,(Q.(T, ("
(T. Q+(T,)). the loss is minimized when
Qr(T, a) = (T. a) = (T)
Combining both cases completes the proof.
C.2 LEMMA 2
Let Qr have converged. Then it holds that
Qr(T, =Q(Taa))
where a" E Atgm is any truly optimal joint action.
Proof. Suppose, for contradiction, that
@r(T,))> Q²)),a").
From Lemma 1, for any a é Aign have
Qr, min ((). Q²
Construct new function Ø based on Q,:
(,(T,)) = (Q" a E Aigms
(Q) & Aigm.
The corresponding loss for Q is
Lo: Σ
-
Σ (((,,)))
Aiam
agA.gmm
Σ at A
(H(T,,)-Q''(T +
Σ ((((Taa)-Q'(T..)
aEAigm |A
Σ
(Q)(T,)) - -T'()))))
Σ
(Q,(T,a)- a))
AA
aE aEAigm W4
LQ..
Since Q, is assumed have fully converged, the loss cannot be decreased further. But Lo: fo- under the assumption Qr(T, â) > (" which is contradiction.
Hence, we must have
and Lemma 2 holds.
C.3 THBOREM CONTAINMENT OPTIMAL JOINT ACTIONS
For any 7 and joint action a, let Q have converged. Then we have
Armm Ar,
i.e. all truly optimal joint actions are contained in the potentially optimal set A,.
Proof. Consider any a" A Atgm-
14

---

Published as a conference paper at ICLR 2026
If a Aigms then by definition Aigm S A,, and thus a" Ap.
If a* Aigms from Lemma 1 we have
Qr(T,a") (i, ((,,a)) -r(
By Lemma 2, Q+(T, (T,
Qr(T,a") Q)
and therefore a' E A,.
Since every a' E Argm is included in A,, we conclude that
Algm S Ar.
This completes the proof.
C.4 LEMMA 3
When Qr has converged:
If Aigm Aigms then
If Aigm & Atgms then
min A
(
Proof.
If Aigm Atgm. then setting Q)( achieves Lq. = 0. Any other value
leads to LQ. 0, so the minimum is achieved exactly when Q, à) = (T"T'a"')
If Aigm & Aigms split the loss Lo into
L1 =
Σ
(Q(TT,a)-Q"(T,)0)
aEAigm Argi
C2
aT)
ag AgAigmUAAgp
By Lemmas and 2, for a E Aigm U A Argm: Qr(T,)) Q,(T, so
L1
Σ
(Qr(T,a)- Q²(
aEAigmUA Utgm
Consider L1 as a quadratic function of Qr(T, a). Its minimum m satisfies
min aEAigm
Q²(T,a) < m Q²(T,a)).
For L2, since Qr(T,a) a) S a)(T,a) by Lemma 1, is monotonically decreas- ing for Qr(T, à) < max Aigm U (T, and constant for Q,(T,ā) 2 maxagAigm aT)
Combining L1 and Lz, the global minimum of Lq. L1 La occurs value min Q²(T,a) < Q+(rra) (T)',,), ac Aigm
establishing the second case.
This completes the proof.
15

---

Published as a conference paper at ICLR 2026
C.5 THEOREM 2 (CONVERGENCE OF WEIGHTED TRAINING)
Assuming Qto satisfies IGM and has a unique maximal joint action à, there exists a 0 such that Quat converges with Atgm and A, Azgm.
Proof.
We consider the weighted loss for Qiot:
Luue (Qrata)(Q aT)
Partition joint actions in the method section:
a à a E a 2 ( (T. E a, a, (T, a) Qroe (T. a き A, (weighted by a).
When a = the last is zero. Then we exclude the third term and get lower bound:
Liue 2 (Qror(T, à) Q" (T) +
E
(Quor(T, Q" (T, a))
aya Q' (T ((a
Similarly, after Qr converges, its loss takes the same form:
La. = (Qr(T, a) (²( +
Σ a aFa (T..))22.(r.a))
- (Q((T,a)-Q((1))))
Define Qr(T, a) m the minimum of Lq.- And for joint actions that satisfy A,,a + à, (T, a) 2 Qior(T a), Qroe(T, a) S Qroe(TT a). Qtoe (T, a) should be as large as possible and finaly equal (To a). Therefore, the minimum values of LQ.. and LQ, actually the same. We can then construct valid Qior satisfying all consumptions:
{
m + 6,
a == à
Qiar (T, a)
(m.
a + à
where ensures unique maximal joint action.
Two cases arise:
("",(tr)))Q"(r__(eeeee (TTr,a),, and Qiot has converged. If à & Atgm. Lemma gives ("( < m < <T'(T,
If à E A Aigm: then
Construct
a a=a" (Q'(T, a") (T ( m,
a + a':
which satisfies for.. LQuer ensuring iterative training that moves toward Atgm-
Thus, with iterative training and = 0. Qtot converges such that à E Agmm and A, Aigm- The result also holds for Qust satisfying IGM without strict monotonicity, e.g., QPLEX.
C.6 REMARK ON THE UNIQUE MAXIMAL JOINT ACTION ASSUMPTION
In Theorem 2, assume that Qtos has unique maximal joint action à for simplicity of analysis. In practice, this assumption can relaxed:
Even if multiple joint actions achieve the same maximal value, the weighted training proce- dure will assign higher emphasis to those in Atgms guiding the learning dynamics toward the set of potentially optimal joint actions.
16

---

Published as a conference paper at ICLR 2026
The uniqueness can also be enforced with arbitrarily small perturbation added to break ties, which does not affect policy performance but ensures theoretical convergence of the proof.
Empirically, in stochastic environments or with function approximation, exact ties are rare, this assumption reasonable for most practical multi-agent RL tasks.
Thus, the assumption mainly simplifies the theoretical exposition without restricting the practical applicability of the method.
D DISCUSSION
This section provides accessible discussion of the motivation and design rationale behind POW, clarifying the innovations and avoiding potential confusion with existing value factorization meth- ods.
D.1 CORE INNOVATIONS
The novelty of POW is reflected in two key aspects: (1) eliminates the need to traverse the entire exponentially large joint action space when recognizing optimal joint actions; (2) It provides theo- retical guarantee of convergence to the global optimum, without introducing approximation error in practice.
These advantages directly address the limitations of prior approaches such as WQMIX and QPLEX.
D.2 PROBLEM CONTEXT
Within the CTDE framework, the IGM condition requires training on a centralized Qtot function. Due to the monotonicity constraints imposed by mixing networks (e.g.. QMIX), joint action may be incorrectly undervalued when some agents take suboptimal actions. This prevents accurate esti- mation of globally optimal joint actions.
WQMIX mitigates this by reweighting potentially optimal joint actions more heavily during training Rashid al., (020b). However, identifying these actions requires unrestricted value function over the full joint action space. Since the joint action space grows exponentially with the number of agents, WQMIX resorts to approximations that inevitably introduce error, limiting its practical applicability.
D.3 DESIGN RATIONALE OF Qr
POW avoids the drawbacks of WQMIX by introducing recognition module, Qr. that directly iden- tifies a superset of potentially optimal joint actions, denoted A,. Instead of exhaustively searching over all joint actions, POW uses arg max, QT, a) as reference and recognizes A, without approximation. This set is then weighted more strongly during training of Qiot. ensuring accurate estimation of globally optimal policies.
To achieve this, Q is designed with three essential properties:
Independence joint action values. that QT(T, a) is independently parameterized for each action. This avoids the monotonic coupling between joint actions present QMIX's mixing structure, enabling Q, to recover the true Q" (T, a) values without interference.
Q, explicitly takes the joint action as input, ensuring
Satisfaction of IGM. Although free from monotonicity, Qr still satisfies the IGM condition, ie.. arg max, (T(T, a) à. This allows á to serve a baseline for identifying all potentially optimal joint actions in A,.
3. Accurate recovery of Q". Qr is trained against the true joint action values Q*, rather than surrogate targets. Thanks to its independence property, Q, can precisely match Q* (T, a) for each action, ensuring that A, can be recognized by simple comparison with Q, á.)
17

---

Published as a conference paper at ICLR 2026
D.4 How Or ENABLES POW
These three properties guarantee that Q, recovers the set A, without approximation, and that A, gradually contracts to Aigm as training proceeds. This mechanism enables POW retain the strengths of WQMIX (emphasizing potentially optimal joint actions) while avoiding its reliance on approximations. Unlike QPLEX, which assigns equal weight all joint actions and often suffers from instability, POW selectively emphasizes A,, ensuring both stability and convergence guaran- tees.
Fig. 8 provides an intuitive visualization: Q, establishes baseline plane QT @), above which potentially optimal actions are recognized. As training proceeds, this plane rises until aligns with the true global optimum, at which point A, Atgm and the optimal policy is recovered.
AvOM
moc (r.m))
(r.
4
This figure illustrates the Q -aluue landscape, where the height each column represents the -a associated with particular joint action. (The exact heights are not critical for the concepts discussed herein.) The current convergence state of the Q, network resembles Stage 2 in The red area represents å. The yellow area highlights Ar, which is determined via à and represents the subset of actions on which POW focuses weighted training efforts. The green area denotes the global optimal joint actions. For Qr, the Q-values beneath the conceptual plane already learned, while the Q-values within A, are set at the plane's 's level. As the training progresses, the plane is expected to rise incrementally. identifying increasingly higher Q" ---uuus
Figure 8:
Fig. 2
D.5 ILLUSTRATIVE EXAMPLE
Stage
Stage
Stage III
c
37%
2311
X.1
387
385
4/6
K41
5:0
7.18
363
3,51
1/2
Qcot
Quot
Qoot
7.50
-0.11
-003
-1206
-1206
11.91
I12
336
C
-11.8
I12
7.90
-11.6
-010
7:00
Q
9:
9.
-0.10
-0.0
831
-000
-021
648
0.57
-0.99
0.6
Q
Q
Q
Figure 9: Three stages of the matrix game. The potentially optimal joint action is highlighted with a yellow border.
18

---

Published as a conference paper at ICLR 2026
To provide intuition for how POW-QMIX overcomes non-monotomicity, illustrate its behavior in one-step matrix game. The joint action space is {A, B, C) In such settings, Q" and Q' equivalent to the ground-truth reward function, allowing us to directly track the evolution of joint action values during training.
The training process can be divided into three stages (I-IIf) as depicted in Fig.
Stage I-II. At the beginning of training, based the values estimated by the Qr module, identify (A, A) and (C, C) as potentially optimal joint actions, with weights set to 1, while all other weight. During Stage II, the Qtor value for (C, C) already matches Q', so its gradient vanishes. In contrast, for (A, A), Qrot < Q", meaning the gradient update increases Qtor and propagates improvements to the corresponding individual utilities Qi (T and Q2 A).
joint actions receive
Stage III. As training proceeds, (A, A) becomes the only remaining potentially optimal joint ac- tion. This action coincides with the true global optimum, enabling POW-QMIX to escape the local optimum (with value 7.9) and converge to the correct solution.
E ADDITIONAL RESULTS OF ABLATION STUDIES
We test the generality of POW by applying it other value decomposition baselines, yielding POW-VDN and POW-OPLEX. These experiments demonstrate that POW is not tied to specific base algorithm but provides general mechanism for improving non con-monotonicity handling.
POW--MMX
QMX
POW-VON
POWA-GPLEX
GPLEX
Tos Robum Moin
3M
5M4
0M
2M4 Tinmslepa
Timestapa
Temustaga
Figure 10: Application of POW to Predator-Prey with three levels of mis-capture penalty.
POWN-MIX
GMMX
POWA-IN
POWN-PPIXX
OPLEX
(8) conticor
MNV22
3.6
2.5
0.8
To2. Won Ratc
5M
Figure 11: Application of POW to SMAC benchmarks.
The results across Predator-Pres (Fig. I0). SMAC (Fig. L.. highway-env intersection (Fig, 12p. and SMACv2 (Fig. [33 consistently show that adding POW substantially improves performance and stability. Importantly, POW-QPLEX alleviates the instability issues commonly observed in QPLEX, and POW-VDN provides noticeable gains despite VDN's limited expressiveness. These findings support the general applicability of POW as a plug-in improvement to value decomposition methods.
19

---

Published as a conference paper at ICLR 2026
PONONIX
DNXX
POW-MTN
VON
Test Success Rasa
Test Paeuun Maan
Toll Su00se Rab
Too Rotum
09
Timestops
Timestaps
Figure 12: Application of POW to the highway-env intersection scenario.
PON-OMIX
GMIX
POWAVON
VON
POW--PPLEX
OPLEX
(b)
Took Roturn Mon
OM 1M. 2M 3M 4M BM BM 7M SM 5M 10M
0M 1M 2M 3M
4M 5M BM 7M 3M 9M 10M
0M
1M
2M
3M
4M SM BM TM
BM BM 10M
Timeslage
Timerapee
Timmsdages
Figure 13: Application of POW to SMACv2 benchmarks.
F EXPERIMENTAL SETUP
Note: In Sec. 4 we set the weight for potentially optimal joint actions to and for all other joint actions [0. 1), following the weighting function Equation (10) of the WQMIX paper, which is commonly used in the WQMIX methodology.
We emphasize that Theorem holds strictly when a As stated after the definition, both our theoretical analysis and experimental implementation consistently adopt 0 to ensure alignment between theory and practice. practice, setting a avoids introducing approximation errors from down- down-weighting suboptimal actions, ensuring that only the recognized potentially optimal set contributes to training.
F.1 PARAMETER SETTINGS FOR BASELINE ALGORITHMS
We conducted experiments using the PYMARI2 framework, an enhanced version of the origi- nal PyMARL, specifically optimized for the StarCraft Multi-Agent Challenge (SMAC). PyMARL2 incorporates several implementation refinements and hyperparameter adjustments to improve per- formance across various scenarios. There are many code-level tricks in PyMARL2, such as the of the Adam optimizer, the batch size, the replay buffer size, the rollout processes, the c-greedy exploration strategy, and the 7 (A)) parameter. These hyperparameters across all algorithms, including ours, to ensure a fair comparison.
set the same values
The hyperparameters listed in Tab. Tab. Tab. 4 Tab. and Tab.
20

---

Published as a conference paper at ICLR 2026
Table 2: Common Hyperparameters in Pymar12
HYPERPARAMETER
VALUE
TRAINING MODE
PARALLEL
ROLLOUT PROCESSES
REPLAY BUFFER SIZE
5000
BATCH SIZE (TRAINING)
128
ACTION SELECTION
6-GREEDY
1.0
FINISH
0.05
ANNEAL STEPS
500k
OPTIMIZER
ADAM
LEARNING RATE
0.001
TARGET NETWORK UPDATE INTERVAL
200
TD(A)
0.6
LAYER NORMALIZATION
FALSE
ORTHOGONAL INITIALIZATION
FALSE
ORTHOGONAL
0.01
PRIORITY EXPERIENCE REPLAY (PER)
FALSE
PER a
0.6
PER 8
0.4
RETURN-BASER PRIORITY
FALSE
MIXING EMBEDDING DIMENSION
HYPERNETWORK EMBEDDING
HYPERNETWORK LAYERS
Table QMIX-Specific Hyperparameters
HYPERPARAMETER
VALUUE
AGENT ARCHITECTURE
RNN
QMIX LOSS WEIGHT
1.0
Table 4: CQMMI-Speciffe Hyperparameters
HYPERPARAMETER
VALUE
WEIGHTS FOR OPTIMAL JOINT ACTIONS
WEIGHTS OTHER JOINT ACTIONS
0.1
Table 5: QPLEX-Specifi Hyperparameters
HYPERPARAMETER
VALUE
DOUBLE Q-LEARNING
TRUE
ADVANTAGE HYPERNETWORK LAYERS
2
ADVANTAGE HYPERNETWORK EMBEDDING
64
NUMBER OF KERNELS
MINUS-ONE TRANSFORMATION
TRUE
WEIGHTED HEAD
TRUE
ADVANTAGE ATTENTION
TRUE
GRADIENT MECHANISM
TRUE
F.2 MATRIX GAME
In matrix game environment, two agents independently select actions, forming joint action to receive an immediate reward. This reward directly reflects the true value of the joint action. This type of environment is characterized by simple and unique state space, eliminating the need to consider complex state transitions. Simultaneously, the reward directly equivalent to the true value, requiring no additional modeling. Furthermore, the reward structure can be flexibly designed,
21

---

Published as a conference paper at ICLR 2026
Table 6: ResQ-Specific Hyperparameters
HYPERPARAMETER
VALUE
LEARNER
RESQ CENTRAL LEARNER
DOUBLE Q-LEARNING
TRUE
MIXING NETWORK
QMIX
HYSTERETIC QMIX HEW/OW-QMIX,
FALSE
CENTRAL MIXING EMBEDDING
128
CENTRAL ACTION EMBEDDING
CENTRAL MAC
BASIC CENTRAL MAC
CENTRAL AGENT
CENTRAL RNN
CENTRAL RNN HIDDEN DIMENSION
64
CENTRAL MIXER
FEEDFORWARD
RESQ VERSION
CENTRAL LOSS WEIGHT
1.0
NO-OPT LOSS WEIGHT
1.0
QMIX LOSS WEIGHT
1.0
CONSTRAINT LOSS TYPE
MSE
CONSTRAINT LOSS DELTA
0.001
MAX SECOND GAP
CONSTRAINT METHOD
MAX ACTION TRUE
RESIDUAL Q-VALUE ABSOLUTE
facilitating the construction of test scenarios with different characteristics. Lastly, the results are intuitive and easy to analyze and visualize. It is precisely because of these characteristics that matrix games have become an ideal testbed for studying the theoretical performance of value decomposition algorithms.
We set = 1 throughout the experiments on matrix game to achieve uniform data distribution and set ideal weights for the purpose theoretical analysis. The weights for potentially optimal joint actions and other joint actions in POW-QMIX are and 0. The weights for optimal joint actions and other joint actions in CW-QMIX and OW-QMIX are and The constant C used in Eqn. iis set to 0.05.
F.3 PREDATOR-PRES
+10
-p
-p
Figure 14: Stag Hunt Game
The "Stag Hunt" game game theory classic scenario that profoundly reveals the inherent conflict between individual rationality and collective rationality, as well as potential coordination mechanisms, while also highlighting the crucial role trust fostering cooperation. This tension between individual and collective rationality precisely constitutes the core of the non-monotonicity problem explored this paper. In the Stag Hunt scenario, agents face two strategic choices: one high-risk cooperative strategy, which yields the highest payoff when all participants choose this strategy, but only a single agent attempts to cooperate while other agents choose not to, that agent will suffer severe losses penalties; the other is low-risk safe strategy, where agent
22

---

Published as a conference paper at ICLR 2026
adopting this strategy can obtain stable but relatively low payoff, regardless of the choices of other
agents.
Table 7: Predator-Prey Experiment Payoff Matrix
A1
A2
Move Up
Move Down
Move Left
Move Right
Stay Still
Capture
Move Up
0
0
0
Move Down
P
Move Left
0
0
0
0
0
-p
Move Right
-p
Stay Still
0
0 -p
0 0 0
P
Capture
-P
-P
-P
-P 10
The predator-prey environment adopted in this paper is an extension of the Stag Hunt concept within complex Markov Decision Process. This environment retains the core characteristics of the Stag Hunt game while introducing a richer strategy space and dynamic interactions. In this environment, multiple agents acting as predators need effectively cooperate successfully capture the prey. All units (including agents and prey) move and interact in discrete grid world.
The detailed settings of this environment are as follows: We construct state space, where each grid cell can contain: empty space, an agent, or the prey. Considering the limitations real-world perception, we limit the observation range an agent to a grid area centered on itself, allowing it to only perceive the types of units within this range, thus forming Partially Observable Markov Decision Process (POMDP). The action space an agent includes six discrete choices: moving in the four cardinal directions (up, down, left, right), staying in place, and performing capture action.
10 grid world as the
The reward mechanism is designed reflect the necessity of cooperation: Only when at least two agents simultaneously perform a capture action in positions adjacent to the prey can the capture be successful, whereupon all agents receive positive reward of +10. Conversely, if only one agent attempts to perform capture action in isolation, not only will the capture action fail, but that agent will also incur a penalty of -p. This design directly maps to the risk-reward trade-off in the Stag Hunt game.
As the absolute value the mis-capture penalty parameter p increases, the non-monotonic charac- teristics of the environment become more prominent. A stricter penalty mechanism reinforces the non-monotomicity of the reward structure, prompting agents vative strategies- trrateges-coomllttee completely avoiding the risk of performing capture action action-thereby potentially missing out on high-payoff cooperative opportunities. This phenomenon provides an ideal test sce- nario for our research on how algorithms can overcome non-monotonicity limitations.
more inclined adopt conser-
The default experimental settings are consistent with those the PyMARL2 framework. The con- stant C' used in Eqn. g is set to 1.
F.4 SMAC
In the PyMARL2 framework, certain parameters such as hidden size and TD(X) have been specifi- cally fine-tuned for the 6h_vs_8z and $s52_vs_3s62 maps. However, for the sake of a fair comparison, we set all algorithms to use default parameters across all maps. The constant C' used in Eqn. 8 set to 0.05.
F.5 SMACV2
The default experimental settings are consistent with those the PyMARL3 framework. The con- stant C used in Eqn. & is set 0.05.
F.6 INTERSECTION SCENARIO IN HIGHWAY-ENV
Highway-env Leurent 2018 is a collection environments specifically designed for autonomous driving decision-making tasks. Its intersection scenario simulates complex traffic environment,
23

---

Published as a conference paper at ICLR 2026
example of which is shown in Fig. 15 providing ideal platform for us evaluate the performance of algorithms on non-mono non-oonotynicity problems.
Figure 15: Example of the intersection scenario environment.
In this scenario, multiple vehicles approach an unsignalized intersection from different directions, with each vehicle controlled by independent agent policy. These vehicles follow pre-planned routes, and the primary task of the agents is to control their vehicle's speed to ensure safe and efficient passage through the intersection. The reward mechanism is intricately designed: positive reward is given only when all vehicles safely pass through the intersection and reach their respective destinations; conversely, any collision occurs, all agents not only receive a severe negative penalty, but the current episode also terminates immediately.
This design leads the environment exhibiting strong non-monotonic characteristics. Due the significant penalty associated with collisions, agents can easily learn extremely conservative strate- gies-such such as stopping completely and waiting outside the intersection to avoid any potential colli- sion risk. However, while such conservative strategies can avoid penalties, they fail to achieve the positive reward successfully navigating the intersection, leading to poor overall performance. Therefore, agents need to learn to find a balance between safety and efficiency, making this an ideal scenario for testing an algorithm's ability to handle non monotonot challenges.
We adopted the same scenario and reward settings in Huang al. (2023). The value is set to 0.1 to ensure the same data distribution for all algorithms. The constant used Eqn. 8 is set to 0.1.
THE USE LLMS
We thank ChatGPT-5 for its assistance polishing the writing and proofreading of this paper. The authors are responsible for the content and presentation.
24


Reviews:
[{"review_id": "sgTE9r979E", "reviewer": "Reviewer_ZhCF", "summary": "The paper proposes POW, a value decomposition based MARL method under CTDE for optimal joint policy recovery using recognition weighting design. Empirically, the method demonstrates improved performance over baselines on multiple benchmarks", "questions": "- I believe in multiple equations $ \\hat{Q^\\star}$ and  ${Q^\\star}$ and mixed and used against the verbal description in paper, making it hard to understand if it's typo, approximation or error.\n\n- Q_r(s,a) and Q_r (\\tau, a) are mixed and used; According to verbal description it should be Q_r(s,a)?\n\n- I wonder the C in eq5 can be discussed in terms of its sensitivity\n\n- Is Q^* and Q_tot network the same as in WQMIX? Also I wonder the \\alpha setting used for WQMIX in the experimental setting", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- This paper is generally well written and easy to follow; With a constrained Q_r to over include promising joint actions and focus learning signal on them and bridges the gap of WQMIX in ideal weighting and practical approximation.\n\n- There are multiple benchmarks demonstrating the effectiveness empirically.", "weaknesses": "- Some of the claims can be too strong. For theorem 1, it assumes the condition of Q_r can converge to  ${Q^\\star}$, however  ${Q^\\star}$ is also unknown and to be learned. E.g in eq.4 it is not using \\hat{Q^\\star} but ${Q^\\star}$, yet also claims Q_r is optimizized to $\\hat{Q^\\star}$. The gurantee in theorem 2 also requires A_r to converge to only optimal a; These are the actual challenging points in practice and cannot be just assumed; This setting is not beneficial to the actual experiment or pracitcal settings.\n\n- Also as above, Q_r and Q_tot rely on $ \\hat{Q^\\star}$ for bootstrapping, however how the bias in  $\\hat{Q^*}$ will effect A_r and Q_r is not discussed. Under extrem case they will not converge at all.\n\n- Some of the SOTA works are missing. This paper only compares with some of the classic Cooperative MARL works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2DQ2cR88I1", "reviewer": "Reviewer_P6qW", "summary": "This paper proposes a new method named Potentially Optimal Joint Actions Weighting (POW) to address the problem of representativeness of a wide range of value factorization functions. This is achieved by assigning higher weights to actions that are found with potential to be optimal.", "questions": "1. in lines (168-169): \" In all our experiments, we set $\\alpha=0$, so only actions in $A_r$ contribute to updates, aligning theory with practice.\" - so what is the pointof proposing this $\\alpha$ weight?\n2. i wonder what happens if $\\alpha$ is not 0? has that been tested?\n3. could the authors elaborate on what is the loss \"$L_{Q^*}$\" in figure 1?\n4. in eq 4, is the optimal value function $Q_*$ also learned? or is it known a priori? from the notation it seems almost like an on policy approach being described, which makes the link between notation and practice a bit unclear\n5. in figure 3, the performances across the 3 different levels of penalties seem a bit inconsistent; for example, could the authors elaborate on why the proposed method performs better in p=-4 and p=-5 but not in p=-3?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This work good theoretical groundings and a comprehensive range of experiments across different MARL environments. The matrix games provided are also useful to demonstrate the representational properties of the method and support the claims made. Theorem 1 and definition 1 are sound.", "weaknesses": "A number of different methods for value function factorization has been explored recently, and some of them can theoretically guarantee the factorization of any family of the environments (such as QTRAN). I feel the motivations for the proposed method are hence not sufficient, simply by saying that their object is to try to improve the expressiveness of the range of factorisation functions that can be represented.\n\nIn section 4.5.2, WQMIX seems to be better than POW in Figure 7(a) and 7(b); it is unclear to me how these results show that the performance of POW \"stems from its recognition-weighting design, not from parameter count\" (line 400); what it shows is that other methods such as WQMIX improve when the network size increases and QPLEX stays the same; it is unclear how it says something about POW's parameter count and in fact it shows that WQMIX performs better in 7(a) and 7(b) when the number of parameters is the same.\n\nSome notations could also be made more clear; for instance, in figure 1, it is unclear to me the meaning of $A_{tot}$.\n\nPlease find below some more questions that reflect other concerns.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "M1dpfYNGt5", "reviewer": "Reviewer_Nw5q", "summary": "This paper proposes POW (Potentially Optimal Joint Actions Weighting), a value-decomposition method for cooperative MARL. It introduces a recognition module $Q_r$ that identifies potentially optimal joint actions and up-weights them during training. The authors prove convergence guarantees and evaluate POW on matrix games, predator–prey, SMAC, and other benchmarks, demonstrating improved performance over QMIX, WQMIX variants, and other MARL algorithms.", "questions": "Questions about the current manuscript:\n1. Will $Q_r$ and iterative loops increase implementation difficulty compared to QMIX or QPLEX? If so, how can this be mitigated?\n2. May not be strictly required, but I wonder if any possible comparisons with CTDE actor-critic MARL methods like MAPPO?\n3. Can we include the algorithm pseudo-code for clarity?\n4. [Minor]: Line 423: the cited work REMIX seems not to match the given reference. Please check.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper addresses the gap between WQMIX theory and its heuristic approximations.\n2. The authors provide proof that the recognized action set converges to optimal actions.\n3. The method was tested on extensive benchmarks, showing performance gain across both monotonic and non-monotonic tasks.", "weaknesses": "See the questions section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XcZhCsl9Ym", "reviewer": "Reviewer_LRKB", "summary": "This paper proposes a new way to identify the optimal joint actions to weight the learning of value-decomposition methods. An additional recognition $Q$-function is learned, which induces a set of actions that may potentially be optimal. While its learning proceeds toward approximating the optimal values, this set gets further refined, until only the optimal actions are present. Experimental results on a wide set of different problems show the better performances achieved by this method, as well as how it can be combined with general value-decomposition algorithms and improve over their vanilla versions.", "questions": "- You claim that, being trained against the optimal $Q^\\*$, the maximizing actions of $Q_r$ are going to be the same as those of $Q^\\*$ itself, and thus the set $A_r$ will include them. But the training of both $Q_r$ and $Q^\\*$ is done simultaneously, and thus we are not guaranteed that their interplay is accurate before convergence occurred. This would probably lead to similar problems to those you highlighted for CW-QMIX and OW-QMIX (i.e., inaccuracies due to learning) no? Am I missing something here?\n\n- I struggle to understand how the additional $Q_r$ is bringing any benefit over the use of $Q^\\*$ directly to identify the optimal joint actions: at convergence, these will both represent the same action-value function, no? And $Q_r$ is indeed trained to chase the optimal unrestricted $Q^\\*$. So why adding an additional structure rather than simply restructuring $Q^\\*$ itself to be formulated as Equation (3)?\n\n- When computing $w(s,\\mathbf{a})$, how do we check if $\\mathbf{a}\\in A_r$? If we need to explicitly construct the set of recognized optimal actions $A_r$, then this may be a quite expensive operation. Such an aspect should be eventually stated clearly.\n\n- It would be good to explicitly state the loss function for training $Q^\\*$ in the paper, as currently it can only be found in Figure 1.\n\n- In Figure 2, why are the values for $Q_1$ and $Q_2$ of ResQ different between sub-figure (h) and (i)?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The issue of overcoming the representation limitations of existing value-decomposition methods, for example through mean of weighting their learning updates as done by this paper, is a key direction to deliver better MARL algorithms. The idea of doing so by identifying the optimal joint actions in an effective way is a viable path for this, and already proved capable of achieving improvements in driving existing methods, as with WQMIX. The proposed set of experiments is very wide and varied, and the reported empirical results are strong. The paper is in general quite clear and easy to follow.", "weaknesses": "I struggle to understand some of the implementation choices made here: for example, the use of a separate $Q_r$ over the already existing $Q^\\*$ is not entirely justified to me. Also, it is not completely clear how the proposed method should overcome the limitations identified in the existing CW-QMIX and OW-QMIX algorithms. Finally, the cost (in terms of computational time) of the proposed method seems a bit hidden and not sufficiently and clearly highlighted, making it difficult to actually assess the trade-off one has to make when choosing the proposed method over other ones. Please see the Questions below for a more in-details explanation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)