High-order Interactions Modeling for Interpretable Multi-Agent Q-Learning
Authors: Qinyu Xu
Yuanyang Zhu
Xuefei Wu
Chunlin Chen
Venue: NeurIPS 2025 poster ()
Keywords: Continued fraction Q-learning
high-order interaction
interpretability
multi-agent reinforcement learning
Primary area: reinforcement_learning
TL;DR: A novel value decomposition framework of Continued Fraction Q-Learning (QCoFr) is proposed to model rich cooperation for multi-agent reinforcement learning without combinatorial explosion.

Abstract: The ability to model interactions among agents is crucial for effective coordination and understanding their cooperation mechanisms in multi-agent reinforcement learning (MARL). 
However, previous efforts to model high-order interactions have been primarily hindered by the combinatorial explosion or the opaque nature of their black-box network structures.
In this paper, we propose a novel value decomposition framework, called Continued Fraction Q-Learning (QCoFr), which can flexibly capture arbitrary-order agent interactions with only linear complexity $\mathcal{O}\left({n}\right)$ in the number of agents, thus avoiding the combinatorial explosion when modeling rich cooperation. 
Furthermore, we introduce the variational information bottleneck to extract latent information for estimating credits.
This latent information helps agents filter out noisy interactions, thereby significantly enhancing both cooperation and interpretability.
Extensive experiments demonstrate that QCoFr not only consistently achieves better performance but also provides interpretability that aligns with our theoretical analysis.

High-order Interactions Modeling for Interpretable Multi-Agent Q-Learning
Qinyu Xu
Yuanyang Zhu
School of Management and Engineering
School of Information Management
Nanjing University
Nanjing University
qinyuxu@smail www.. edu.cnn
yuanyangzhušn ju. edu.
Xuefei Wul
Chunlin Chen
School of Management and Engineering
School of Robotics and Automation
Nanjing University
Nanjing University clchenonju. edu. cn
xuefeiwu@amail. wwj.::ammmmmii.......... edu.
Abstract
The ability to model interactions among agents crucial for effective coordination and understanding their cooperation mechanisms in multi-agent reinforcement learning (MARL). However, previous efforts to model high-order interactions have been primarily hindered by the combinatorial explosion the opaque nature of their black-box network structures. In this paper, we propose a novel value de- composition framework, called Continued Fraction Q-Learning (QCoFr), which can flexibly capture urbitrary-order agent interactions with only linear complexity O (n) in the number of agents, thus avoiding the combinatorial explosion when modeling rich cooperation. Furthermore, we introduce the variational information bottleneck extract latent information for estimating credits. This latent informa- tion helps agents filter out noisy interactions, thereby significantly enhancing both cooperation and interpretability. Extensive experiments demonstrate that QCoFr not only consistently achieves better performance but also provides interpretability that aligns with our theoretical analysis.
Introduction
Cooperative multi-agent reinforcement learning (MARL) has recently drawn much attention [1], such autonomous vehicles [2, 3, 4], robotics [5, 6), and autonomous warehouses [7, 8J. Success in these domains generally depends on a tapestry of local interactions: agents influence one another not only through explicit pairwise effects but also through higher-order dependencies that shape emergent behaviour [9, 10], which is key factor to make coordination efficient. How to model those interactions among agents and do so in way humans understand remains challenge.
Value decomposition MARL has emerged as a powerful framework with the large capacity of deep neural networks and the centralized training and decentralized execution (CTDE) paradigm, which factorises a team return into individual utilities. Leading value decomposition models-such as the monotonic mixer in QMIX [i]]-capture interaction structure only implicitly, leaving their decisions opaque. Post-hoc explainers (e.g.. Shapley value attribution [12] and masked-attention visualisation [13]) offer limited insight and could not recover the underlying temporal or relational dynamics. Another inherent interpretable technique, like decision tree [14, 15, 16] or Shapley- based [12], handles only low-order interactions; higher-order terms still explode combinatorially.
CCoreespondigg author: Yuanyang Zhu and Xuefei Wu.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Here, we argue that modeling high-order collaboration patterns among agents is still crucial for promoting coordination, which is still missing. Continued fractions [17, 18] provide natural remedy: their recursive form captures interactions of any order, and simple truncation yields finite and interpretable approximation without combinatorial blow-up.
Building on these ideas, we propose Continued Fraction Q-Learning (QCoFr), a novel framework that combines expressive value decomposition with intrinsic interpretability. QCoFr represents the joint action-value function as a weighted sum of continued fraction that explicitly approximate arbitrary-orde interactions among agents with linear complexity ((n) in the number of agents. To strengthen credit assignment, QCoFr incorporates a variational information bottleneck (VIB) [19] to capture mesk-relevant latent information to bridge local histories to the optimal joint action u. The assistive information will be used to estimate the credits together with global state s for assisted value factorization, which relieves the spurious correlation between state and Qot We also give a rigorous proof that QCoFr takes linear combination of continued fractions and has the property of universal approximation, even each with finite depth and linear layers. We demonstrate through extensive experiments on LBF, SMAC, and SMACv2 benchmarks that QCoFr not only generally achieves better performances on solving all tasks but also provides interpretability lacking in the state-of-the-art baselines.
structures
2 Background
Dec-POMDP. A fully cooperative multi-agent task can be modeled as Decentralized Partially Observable Markov Decision Process (Dec-POMDP) [20]. A Dec-POMDP can be defined as a tuple (N, S, U, r, Z, O, n, 7). where N denotes set of n agents and s S is the global state of the environment. At each time step, agent selects an action u, E U, forming joint This results
a the next state s' according to the state transition function ( 8, u): S Ux - [0, 1]. All agents obtain the same joint reward r(s. u): +SXU* + I and y E (0 is the discount factor. Due to partial observability, each agent E N receives individual observation 2, E Z from observation function O E O(s, i). Each agent maintains an action- observation history T2 E T, conditioned by its policy n. The overall objective is to find an optimal joint policy π (T ( TT) to maximize the joint value function Q" (st, u,) E [Rt, st, u.].
action u, E U= U
Value Decomposition. Value decomposition has emerged as a dominant paradigm in multi-agent reinforcement learning (MARL) under the centralized training and decentralized execution (CTDE) paradigm. It seeks to approximate the joint action-value function Qior by decomposing it into individual utility functions Q; where each utility depends solely on the agent's local trajectory т;. To ensure that decentralized action selection remains consistent with the globally optimal joint action, the decomposition must satisfy the Individual-Global-Me
(IGM) principle [11]:
arg max Qtor T, u) = {arg max,; Q1 (TI, uI), ... arg max.... Qn (Ta, Ux))
(I)
Among representative methods, VDN [21] implements a simple additive decomposition, treating all agents' contributions equally by summing their utilitics. To capture more complex coordination while preserving IGM, QMIX [11] introduces a mixing network that combines utilities into the joint value through monotonic function. During training, the mixing network and agent networks are jointly optimized by minimizing the temporal-difference (TD) loss of Qiot, while during execution, agents act independently using their local policies derived from the learned utility functions. The introduction of representative related works for the above formulation can be referred to in Appendix A.
3 High-order Interactions Modeling for Decomposition
In large multi-agent task, agents are usually decomposed into several coalitions, each consisting of multiple agents that cooperate to accomplish the common goal. Besides, each agent could belong to different coalitions at different time steps. This coalition organization is general and can characterize most coordination patterns among agents. Thus, it is essential to model complex interactions among agents and estimate their credits for understanding their coordination patterns. In this section, provide an informal comparative analysis of using the continued fraction network (CFN) architecture against widely adopted value decomposition methods: VDN [21], QMIX [11], and N [22].
From the view of value factorization, VDN provides simple yet highly interpretable approach by representing the joint Q-value simply as the sum of individual agent Q-values, Quo((T, u)
2

---

EIEN Qi (Te, u.). VDN inherently models only first-order interactions, which could limit its ability to capture richer and group-level coordination. QMIX enriches the functional class of factori- sation than that of VDN by introducing a state-conditioned monotonic mixer Qtor (8. T; u)
SQMIX (s. [Qi (Ti, u1) ,...., Qn (Tn, un)]), where fQMIX constrained to be monotonic in each argument. This implicit higher-order modeling increases representational capacity, but at the expense of pnverpeetbbllty-Sqaae
behaves as a black box, obscuring which interactions drive performance. NA²Q tries to expand the joint value via Taylor-like decomposition across all agents up order n.
Qio fo E ai (Q
ERED, Ok JK (Qx) +
O1...n fi... (Q1, Qn.
order-1
order h
order-n
where fü (fi..... non-empty subsets h E {I..... with order-h interactions. While this yields interpretability. low-order interaction terms (e.g., 2-order), the number of terms grows exponentially, O(2"), making full expansion impractical for large teams.
f..... 3 is modeled with neural additive model [23] and D/, is the set of all
To overcome the limitations of existing value decomposition meth- ods discussed above, we enrich with the value factorization via a continued fraction neural network (CFN) inspired by CoFrNets [18], whose recursive structure and linear composition form a function class with universal approximation capability [24]. Following the terminology of CoFrNets, we call the function depicted in Fig. I "ladder", its pictorial representation resembles a rail-and-rung structure that sequentially propagates inputs through the nodes. The joint action-valus function can be established as weighted summa- tion of individual values:
Q1 Q2 ... Qn
Qio = - an f (Q):
(3) Figure I: diagram of CFN
where f denotes a continued fraction structure uan Q+ wig Q +
composed of different depth ladders that take individual values Q inputs and out- put temporal values Q with d- order interactions.
with linear functions wk of the input Q (Q...... Qn3 given ladders, and ag are learnable weights used to estimate the contri- bution of individual agents and coalitions of agents. With a power series expansion, such recursive structure can be rewritten as
Wyy 4+ w2Q+... Ep Spi
(4)
where the coefficients Cp........ and the weight parameters ue in one-to-one correspondence [17, 18], and each Cp........ used to derive interactions among agents.
CFN generally approximates interactions up to finite depth d, which constitutes rational approx- imation Ra(Q) of the joint utility function up to d-order interactions among agents. It shows that truncating the continued fraction provides practical rational approximation that explicitly models agent interactions up to order d:
Q we Q+ wa
n Pe - Cpi....PP. Pr. II Q.
(5)
waQ
Definition (Padé Approximant [25, 26)). Let C(z) Ex=o ox2 be a formal power series in the variable z, then the Padé approximant of order [L/M] is a rational function of the form:
(L) = [AL.M (BL,)/)))
(6)
where (z) and BI, (2) are polynomials of degrees at most I, and M, respectively, chosen such that
B (L)M(2) (L) -((2())))(() O22++++1),
(7)
where notation Ox*²) denotes some power series of the form Em Znz". This approximation minimizes the difference between the rational function and the power series up to the order M.
We give formal proof that such truncation naturally satisfies the Padé approximation condition as defined in Definition 1, i.e., f(Q) Ra(Q) Ó(Qá+1), where ʃ(Q) denotes the complete representation of agent interactions based on their individual value functions. This implies that the
3

---

Overall Architecture
Agent Network VIB Modale
Mixing Network
VIB Loss
VIB
Reparameterizztion
Qut
T, M
VIB
0.
Qet
fo-a
yya
fl 4..
Mixing Network
0g
2 F =
f4,
CFN
Eyye
Q Loss
Encoder
Decoder
min ((()
Qi
Q%
w
Agent
h
Agent
Agent n
(ol,
Q1(%
(%, mz)
(oj.
[ol "0 1)
A
Qi (11, Qn (Tas Rnp)
Figure 2: The overall architecture of QCoFr. A CFN-based mixing network models high-order interactions among agents by expressing Que as linear combination of ladders over individual values Q. The VIB module encodes the hidden state h into assistive information m, which is used to deduce the credits of each agent of coalitions together with the global state s.
depth-d truncation of the continued fraction yields an exact approximation of ((Q) up to the d-th order of Q. The detailed proof is provided in Appendix B.2.
Morcover, we can easily control the capacity of the interaction order among agents with the acceptable complexity (n) by adjusting the depth of CFN. Since linear combinations of CF modules constitute function class with universal approximation power [24], they can approximate any continuous mapping between finite-dimentn dimensional spaces. Its recursive and interpretable structure reveals the contributions of individual agents and coalitions of agents, enabling precise credit assignment while faithfully modeling high-order interactions among agents. We also study two CFN variants, CFN-C and CFN-D (see Fig. 10 and Appendix C.6). CFN-C combines single-feature and full-input CF ladders, trading higher expressivity for additional computational cost. CFN-D retains only single- feature ladders, yielding purely additive mixer that cannot represent higher-order interactions. Empirical comparisons are provided in Section 5.2.
Methods
We next present interpretable value decomposition framework that explicitly models arbitrary agent interactions without incurring a combinatorial cxplosion. The overall architecture QCoFr is illustrated in Fīg. 2, which consists of three components as follows: (1) an individual action-value function for each agent, (2) an assistive information generation module that encodes the hidden state h via variational information bottleneck (VIB) [19] to produce the assistive information to promote the credit assignment, and (3) a joint action-value function Q.o that composes individual action-value functions into a joint action-value function using CFN architecture, where the assistive information m and the global state 8 are fed into the mixing network to estimate credits. During the centralized training, the whole network learned in an end-to-end fashion minimize the sum of TD and VIB loss, and each agent selects actions using its Q-function based on local cotionnobservattin history a decentralized fashion.
Individual Action-value Function is computed by a recurrent Q-network with gated recurrent unit (GRU) for each agent é, which takes the local observation of. historical information 4/-1, and previous action un 11 as inputs and outputs local QT u.).
Assistive Information Generation Module. Previous works usually use all available information for estimating the credit assignment ag,, which could be quite inefficient [27]. The global state is an unobserved confounder as the common cause factor of the global state and the joint value function [28, 22]. Thus, it will be necessary to introduce additive information to cut off the confounder. To this end, we generate an assistive latent representation m from the hidden state h of agents to predict the optimal action selection u², which forms the Markov dependency h + m -> u². Here, we utilize VIB to capture the task-relevant information that can provide more accurate estimation of the optimal joint action selection from the hidden state h, which can promote the credit assignment. The goal is to learn encoding that contains as little redundant information as possible while

---

providing maximal information about the prediction u that affects environmental information or its private properties, which can be written
JI [(m, u² (m,, h: Φ),
(8)
where the Lagrange multiplier 3 2 0 controls the tradeoff. Here, we adopt variational approximations
to simplify the computation of the mutual information / in Eq. 8. The following theorems provide tractable bounds for each term.
Theorem 1 (Lower Bound for [(m, v); Let the representation be reparameterized random variable drawn from multivariate Gaussian distribution m, NN (mag dmn) T), where fm is an encoder parameterized by Φm. h; denotes the hidden state of agent and I is the identity covariance matrix. Then, the mutual information between the assistive information m and the optimal joint action u² is lower-bounded as:
I(m, ;²) 2 Be-pte-- q(u;) [h; c)),
(9)
where q(uţ mm;) is variational distribution approximating the true posterior p(uß ma), and my = =f(h,, denotes a deterministic funciiooo of h2 and the Gaussian random variable e.
This bound follows from the variational representation of mutual information. By introducing a variational approximation qu, m,) and applying the reparameterization trick [29] to write p(m; hy)dm; = p(c de, the intractable posterior is replaced with a more manageable form, enabling efficient optimization via gradient-base based methods.
Proof sketch:
Theorem 2 (Upper Bound for (m, h; Φ)). Let q(m) denote variational approximation of the marginal distribution p(m).. Then, the mutual information between the representation m and the hidden state h admits the following upper bound:
I(m, h; +b) S KI(p(m hx) []
(10)
Proof sketch: This result exploits the non-negativity of the KI. divergence. By approximating the marginal distribution p(m) with variational approximation q(m), the mutual information can be bounded from above via the definition of the KI. divergence.
Combining the above conclusions,
arrive at the following variational objective function:
N
LVIB
Σ
Benple) [- log q (ut [ <)) + SKL [p(m) hh).g(m))
(11)
The detailed derivation and rigorous proofs for these bounds are provided in Appendix B.I.
Mixing Network. We introduce the CFN-based mixing network consisting of ( ladders (Fig. 2) with maximum depth which are designed model d-order interactions among agents. Each ladder takes the local value functions Q as input and produces an output Q defined in Eq. 5. To satisfy the IGM principle [30], each CFN layer adopts strictly non-negative activation function of the form max{[=];35 where z denotes the sum of the current weighted input and the reciprocal of the previous layer's ' output, and 8 a small positive constant to prevent poles caused by near-zero denominators. Since the universal approximation theorem applies any linear combination of continued fractions and does not rely on non-negative weights, an extension to non-IGM mixers is possible, with details discussed in the Appendix E.2.
Effective coordination hinges on precisely deducing the contributions cach individual agent or coalition to the overall success. To this end, we leverage assistive information m together with the global state 8 to estimate the credits. Specifically, the credit ag for each ladder is computed as
exp (wmm) ReLU (w,s)
ag
(12)
Ek exp (w.,, ReLU (ur,s)
where the weights wm, and w are the learnable parameters, and ReLU the activation function. When the credits 0g are then used to aggregate the outputs Q₂ of the ladders, the joint action-vall value function Qtot can be represented
Qiot = El=rah Qx = Ek 0% Ed Ep.... pn= Sp Spt......pp Pn Ih Q₂¹ = Eph =0= E S Pn 1= O
(13)
5

---

where 2y ...... denotes the final coefficient of each interaction term. For notational simplicity when discussing interaction patterns, we denote the coefficient of a specific agent interaction (e.g.. agent and j) as Bij. where the subscript indicates the agent indices involved.
Overall Learning Objective. To sum up, train QCoFr end-to-end with two terms of loss functions. The first one is naturally the original mean-squared temporal-diffrrent (TD) error, which enables each agent learn its individual agent policy by optimizing the joint-action value of the mixing network module. The last one is the VIB loss LvrB that is encouraged to produce assistive information. Thus, the overall loss function is formulated follows:
((0) = Σ (Ota (8, u) y:)² + LVIB,
(14)
where yi - Quu (s', arg max Elt Qwi (s', u with denotes the parameters of the target network. We summarize the full pseudo-code of QCoFr in Appendix C.1.
5 Experiments
In this section, we conduct experiments to evaluate QCoFr on three challenging benchmarks over the Level Based foraging
(LBF) [31], the StarCraft Multi-Agent Challenge (SMAC) [32] and SMACv2 [33]. The details of the environment can be found in Appendix C. We compare our method against nine prominent baselines, including VDN [21], QMIX [11], QPLEX [34], Centrally-Weighted Weighted QMIX (CW-QMIX) [35], CDS [36]. SHAQ [37], GoMARL [38], ReBorn [39], and NA [22]. We carry out experiments with 5 random seeds, and all performance results are plotted using mean + std. Furthermore, perform interpretability analyses of QCoFr to provide empirical evidence for the contributions individual agents and coalitions of agents.
5.1 Performance Comparison
Performance on LBF, We first evaluate QCoFr on two con- structed LBF tasks: 3 agents with 3 food items (lbf-3-3) and 4 agents with 2 food items (lif- 4-2). LBF is a grid-based multi- agent benchmark that supports configurable levels of coopera- tion and observability [40]. Each
QMIX
CW-QMIX
SHAQ
NAPQ
QCofr (Ours)
Ibf-3-3
Ibf-442
100
100
Test Test Return Mean
Test Return Mean
80
80
60
60
40
40
20
20
agent navigates a 10 x 10 grid
world and observes
2 sub-
0.50
1.00
0.25
0.50
0.75
1.00
grid centered around it, where a group of adjacent agents can suc- cessfully collect a food item if the sum of their levels is greater than or equal to the item's level. As shown in Fig. 3, QCoFr achieves competitive performance compared to state-of ofte-of-the-asd art methods. However, the simplified task design inherently constrains the model's capacity to fully leverage higher-order interaction mechanisms, whose potential advantages may emerge in complex multi-agent coordination scenarios. The failure of the CDS may due to an overemphasis on behavioral diversity without considering effective coordination among agents. QMIX has a slow convergence speed due to its inability to exploit latent information, which restricts its representational capacity. GoMARL and ReBorn also require more steps to discover effective agent groupings and allocate neuron weights. While CW-QMIX, NA?Q, and SHAQ demonstrate competitive performance by introducing additional modules to enhance value decomposition, their inability to model higher-order interactions or reliance complex computation may hinder scalability in complex multi-agent coordination tasks.
T (mil)
T (mil)
Figure 3: Performance results on LBF benchmark.
Performance on SMAC. We next compare QCoFr with baselines on the more challenging SMAC benchmark, where agents make decisions based on local observations while cooperating to defeat Al-controlled enemies. We show the performance comparison one easy map: 2s3z, three hard maps: 2c_vx_ 2r_vs6642g, 2x_vs_5z, 5m_ $m_vs__0_ vs_6m, and two super-hard maps: 3s52_ vs 3s6z, 6h vs 8z. As shown in Fig. 4, QCoFr achieves superior performance across almost all
six different scenarios, including

---

VON
QPLEX
CDS
GOMARL NA'Q
ReBorn
QMIX
CW-QMIX
SHAQ
QCoFr (Ours)
2s3z
2c_ 64zg
3s $s_vs_52
100
100
100
Test Win Rate of
80
Test Win Rate 86
80
9
80
60
60
60
40
40
50
100
150
0.50
1.00
1.50
2,00
0.50
1.00
1.50
2.00
(mil)
(mil)
5m 6m
3s52 Vs. 3s62
6h vs 8z
100
Test Win Rate *9
80
Test Win Rate %
Test Win Rate %
60
60
40
40
20
0.50
1.00
1.50
2.00
1.00
2.00 (mil)
3.00
4,00
5.00
1.00
2.00
3,00
4.00
5.00
T (mil)
T (mil)
Figure 4: Performance results on SMAC benchmark.
VDN
OPLEX
CDS SHAQ
GoMARL NAPQ
ReBom QCoFr (Ours)
QMIX
CW-QMIX
10gen_protoen
10gen_terran terran
10gen_zen zerg
80
80
80
60
Test Win Rate %
60
Test Win Rate %
60
Win Rate
40
40
40
@20
20
20
0
20
40
60 (mll)
80
100
20
40
60
80
100
20
40
60
80
100
(mil)
(mil)
Figure 5: Performance results on SMACv2 benchmark.
scenarios, especially on the super hard tasks. CW-QMIX and QPLEX do not achieve satisfactory performance, which may be due to excessive approximation and the relaxed constraints introduced during training. CDS exhibits slower convergence rates, which may be due to requiring more training steps to capture diverse individualized behaviors. Although GoMARL and NA can model high- order interactions to yield notable performance complex scenarios by leveraging grouping and enumeration, GoMARL requires extended training durations learn effective groupings, and NA²Q considers low-order interaction terms avoid combinatorial cxplosion.
Performance on SMACv2. We evaluate QCoFr performance on three scenarios from SMACv2, including zerg zerg________ $___________ vs_5, protoss 5 vs_5, and terran_5 1erram___vv____ 5. SMACv2 introduces randomly generated and positioned for unit, increasing environmental stochasticity compared SMAC. The performance of QCoFr is significantly better than other algorithms across all scenarios. In contrast, GoMARL performs the worst, which due to its dynamic grouping structure, leading slow convergence. While SHAQ demonstrates marginally superior performance, its inability model higher-order agent interactions limits its adaptability all maps. Compared to NA QMIX, and CDS, QCoFr achieves better performance, which should benefit from the assistive information for estimating credit assignment, especially for the high-order interaction patterns among agents.
5.2
Ablation Studies
To discuss the influence of each component, we conduct ablation studies about (a) the number of interaction orders among agents, (b) the CFN structure, and assistive information performance.
7

---

vs. 642g
35 VS 5z
6h Vs 82
100
100
100
oCofred -1
OCoFrid id
Test Win Rate %
80
Test Win Rate 86
80
OCofred 2)
OC
Testt Win Rate S6
80
Ocutr
Qcafrid
OMIX
QCotr B1
60
60
GPRX
40
40
40
ocetr (2 -1
20
oc 2)
0C0f-15 40
20
20
OMX
0.50
1.00
1.50
2.00
0
0.50
1.00
1.50
2.00
1.00
2.00
3.00
4.00
5.00
T (mil)
T (mil)
(mill
Figure 6: Comparison of different numbers of interaction orders.
QCoFr (Ours)
QCoFr w/o
QCoFn.C w/o VIB
QCoFrD we VIB
QMIX
2c vs 64zg
3s vs 5z
6h vs 8z
100
100
80
Test Win Rate %
80
66
80
Test Win Rate %
60
Test Win Rate
60
40
40
22
0
0
0.50
1.00
1.50
2.00
0
0.50
1.00
1.50
2.00
1.00
2.00
3,00
4.00
5.00
T (mil)
(mil)
(mil)
Figure 7: Performance with and without VIB and comparison of different CFN structures.
The Number of Interaction Orders. We conduct ablation experiments by varying the depth d of CFN, which governs the highest order of inter-agent interactions modeled by the framework. On 2c 2c_vs_ose pa_64rg and 3s vs _s_5z maps, QCoFr (d 2) yields significant performance improvements compared to QCoFr (d = 1), which underscores the importance of explicitly modeling higher-order inter-agent effects as shown in Fig. 6. However, beyond the optimum, performance could degrade, likely due to overfitting to spurious higher-order correlations. The same trend emerges in the 6h_ th_v___ & scenario, where the model achieves its best performance with depth d 6 before degrading with further depth increases. These findings demonstrate that modeling higher-order interactions yields performance improvements for complex coordination tasks, but only within bounded regime that aligns with the task's intrinsic interaction complexity. Notably, our method using a single-layer CFN still outperforms QMIX, which confirms that the assistive information can better help deduce the contribution of agents for their team.
Impact of CFN Structure and Assistive Information Module. To validate the effectiveness of the CFN structure, we compare QCoFr w/o VIB against two CFN-based variants- CCoFr-C w/o VIB and QCoFr-D w/o VIB, under a unified setting without the VIB module. As shown in Fig. 7, QCoFr w/o VIB achieves comparable performance to QCoFr-C w/o VIB, while offering a simpler structure and lower computational cost. In contrast, QCoFr-D w/o VIB exhibits significantly degraded performance, especially on the 6h vs 8z scenario, which underscores the necessity of explicit inter- agent interaction modeling for complex tasks. To summarize, its ability to capture higher-order interactions with a simple, efficient design. By integrating a VIB-based assistive information generation module, QCoFr achieves significant improvements in both convergence speed and final performance, demonstrating its efficacy and practical utility. More additional ablations in Appendix E.1 detail the respective contributions of the VIB and CFN modules.
adopt CFN as the backbone for
5.3 Interpretability
To intuitively demonstrate the interpretability of QCoFr, we demonstrate QCoFr (d 2) and QCoFr (d 4) and display some key frames on 5m $n_vs__ 6m scenario. As shown in Fig. 8(b), the pairwise coalitions (agent 3, agent 4) and (agent 3, agent 5) emerge coordinated coalitions that focus fire on the same enemy unit. Here, we find that they have higher coalition contributions of 3.884 and 3.339 than the others. In contrast, agent disengages from combat when it has low health value to avoid early elimination, which indicates that it does not contribute to the team and obtains lower contribution of 0.552. These observations highlight QCoFr's ability to facilitate diverse role
8

---

134T
VDN
(b) QCoFr (d 2)
(c)
8.0g
8.00
3331
Paralag 51
5-21
3u0 Am
80
(d) QMIX
(e) QCoFr (d=
(1)
Figure 8: Visualization of evaluation results for QCoFr and baselines on 5m ps [0_____óm0_ 6m map. (b) and (e)
illustrate the behaviors of QCoFr with CFN depths of 2 and specific time step. (c) visualizes
the weights of individual agents and pairwise coalitions corresponding to the behavior shown in (b), while (f) presents the top five hegghst coalitions extracted from (e), due to the increased
number possible interactions. (a) and (d) show the behaviors of VDN and QMIX for comparison.
Similarly(VDN))
Similarity(QMII:
Similarntylle SimilaritlOCaa))
1.000
2.50
0.375
2.05
0.93
8:59
0.95
3.44
0.57
0.95
5.97
0.9
0.300
0.89
0.84
9.
0:66
0.30
0.22
3.55
295
0.200
-.000
0.375
-3775
0.96
0.84
0.97
0.80
0.47
0.95
0.93
0.72
0.00
0.01
2:95
-3.94
0.250
-3.10
0.225
(b)
Figure 9: Visualization of agent diversity via QCoFr and baselines. Cosine similarities of agent Q-values under the same observation indicate that QCoFr yields more diverse agent behaviors.
specializations and demonstrate the advantage of modeling agent interactions, which helps deduce the contribution of individual agents and coalitions within their team.
Further, we show the behavior the agents when 4 for QCoFr to demonstrate that the model captures complex agent cooperation as shown in Fig. 8(e). As shown in Fig. 8(c), the weights predominantly concentrated on agent and agent 4, as well as their interactions with other agents, consistent with their coordinated attacks. While in deeper QCoFr, top-ranked terms predominantly higher-order, such as interactions among agents 3, and 4, which yield the highest contributions, demonstrating the model's capacity encode intricate cooperative dynamics. In contrast, QMIX and VDN produce less differentiated policies, with individual Q-values remaining close, making their decision logic harder interpret. Furthermore, as shown in Fig. 8(a) and Fig. 8(d), agents under VDN and QMIX frequently attack multiple enemy units simultaneously, leading to prolonged numerical disadvantage and increased failure risk.
To evaluate whether QCoFr facilitates more diverse agent behaviors during training, we compute the cosine similarity of individual Q-values under the same observation across different methods, as shown in Fig. 9. QCoFr yields consistently lower similarity, which indicates more diverse and thus more specialized agent preferences, in line with the qualitative findings in Fïg. 8. In contrast, VDN and QMIX maintain similarity close to 1, which reflects homogenized preferences that hinder the emergence of complex cooperative strategies and obscure individual decision-making
6 Conclusion
In this paper, we introduce QCoFr, an interpretable value-based MARL framework grounded in the expressive and compact structure continued fractions. By leveraging continued fraction neural networks and variational information bottleneck over agent histories, QCoFr explicitly models agent

---

interactions of arbitrary order while maintaining low model complexity and inherent interpretability. Extensive experiments show that QCoFr matches or surpasses strong ralpoddecomposition baselines and yields clearer attributions to individuals and coalitions. We believe QCoFr presents a promising a
direction for designing MARI. algorithms with mathematically grounded, interpretable structures and highlights the importance of modeling higher-order coordination. Future work will explore adaptive mechanisms to dynamically adjust the depth of interaction modeling response to task complexity.
Limitations. In the current implementation, the CFN depth is fixed per task, which may be suboptimal or wasteful. A promising direction is to adapt depth during training (or per state) to balance computation with representational power.
7
Acknowledgements
This work was supported in part by the China Postdoctoral Science Foundation under Grant Number 2025T180877, the Jiangsu Funding Program for Excellent Postdoctoral Talent 20257B267, the National Key Research and Development Program of China under Grant 2023YFD2001003 Major Science and Technology Project of Jiangsu Province under Grant BG2024041, and the Fundamental Research Funds for the Central Universities under Grant 0814380004..
References
[1] Afshin Oroojlooy and Davood Hajinezhad. A review of cooperative multi-agent deep reinforce- ment learning. Applied Intelligence,
2023.
[2] Songyang Han, He Wang, Sanbao Su, Yuanyuan Shi, and Fei Miao. Stable and efficient Shapley value-based reward reallocation for multi-agent reinforcement learning of autonomous vehicles. International Conference on Robotics and Automation, pages 8765-8771, 2022.
[3] Zhili Zhang, Songyang Han, Jiangwei Wang. and Fei Miao. Spatial-temporrl-awwaaa -ware safe multi- agent reinforcement learning connected autonomous vehicles in challenging scenarios. In International Conference Robotics and Automation, pages 5574 55800, 2023.
[4] Joris Dinneweth, Abderrahmane Boubezoul, René Mandiau, and Stéphane Espié. Multi-agent reinforcement learning for autonomous vehicles: A survey. Autonomous Intelligent Systems, 2(1):27, 2022.
[5] Yixin Huang, Shufan Wu, Zhongcheng Mu, Xiangyu Long, Sunhao Chu, and Guohong Zhao. multi-agent reinforcement learning method for swarm robots in space collaborative exploration. In International Conference on Control, Automation and Robotics, pages 139-144, 2020.
[6] Weizheng Wang, Le Mao, Ruiqi Wang, and Byung- Cheool Min. Multi-robot cooperative socially- aware navigation using multi-agent reinforcement learning. International Conference on Robotics and Automation, pages 12353-12360. 2024.
[7] Tong Zhou, Dunbing Tang. Haihua Zhu, and Zequn Zhang. Multi-agent reinforcement learning for online scheduling in smart factories. Robotics and Computer-Integratte Manufacturing, 72:102202, 2021.
[8] Aleksandar Kmjaic, Raul D Steleac, Jonathan D Thomas, Georgios Papoudakis, Lukas Schäfer, Andrew Wing Keung To, Kuan-Ho Lao, Murat Cubuktepe, Matthew Haley, Peter Börsting, et al. Scalable multi-agent reinforcement learning for warehouse logistics with robotic and human co-workers. In International Conference on Intelligent Robots and Systems, pages 677-684, 2024.
[9] Yaodong Yang, Rui Luo, Minne Li, Ming Zhou, Weinan Zhang, and Jun Wang. Mean field multi-agent reinforcement learning. In International Conference on Machine Learning, pages 5571-5580, 2018.
[10] Daan Bloembergen, Karl Tuyls, Daniel Hennes, and Michael Kaisers. Evolutionary dynamics of multi-agent learning: A survey. Journal of Artificial Intelligence Research, 53:659-699 667, 2015.
10

---

[11] Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. QMIX: Monotonic value function factorisation for deep multi agent reinforcement learning. In International Conference on Machine Learning, pages 29944434, 2018.
[12] Jianhong Wang, Yuan Zhang, Kae-Kyun Kim, and Yunjie Gu. Shapley Q-value A local reward approach to solve global reward games. In AAAI Conference on Artificial Intelligence, pages 7285-7292, 2020.
[13] Tsuyoshi Goto, Hidenori Ita Tsubasa Hirakawa, Takayoshi Yamashita, and Hironobu Fu- jiyoshi. Solving the deadlock problem with deep reinforcement learning using information from multiple vehicles. In 2022 IEEE Intelligent Vehicles Symposium, pages 1026-1032, 2022.
[14] Stephanie Milani, Zhicheng Zhang, Nicholay Topin, Zheyuan Ryan Shi, Charles Kamhoua, Evangelos E Papalexakis, and Fei Fang. MAVIPER: Learning decision tree policies for in- terpretable multi-agent reinforcement learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 251-266, 2022.
[15] Zichuan Liu, Yuanyang Zhu, Zhi Wang, Yang Gao, and Chunlin Chen. MIXRTs: Toward interpretable multi-agent reinforcement learning via mixing recurrent soft decision trees. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(5):4090-441 4107, 2025.
[16] Daniël Vos and Sicco Verwer. Optimizing interpretable decision tree policies for reinforcement learning. arXiv preprint arXiv:2408. /632, 2024.
[17] Carl M Bender and Kimball A Milton. Continued fraction discrete nonlinear transform. Journal of Mathematical Physics, 35(1))364-36) 1994.
[18] Isha Puri, Amit Dhurandhar, Tejaswini Pedapati, Karthikeyan Shanmugam, Dennis Wei, and Kush R Varshney. CoFrNets: interpretable neural architecture inspired by continued fractions. In Advances in Neural Information Processing Systems, pages 21668 06660, 2021.
[19] Alexander Alemi, Ian Fischer, Joshua Dillon, and Kevin Murphy. Deep variational information bottleneck. In International Conference on Learning Representations, 2017.
[20] Frans A Oliehock and Christopher Amato. concise introduction to decentralized POMDPs, volume 1. Springer, 2016.
[21] Peter Sunchag, Guy Lever, Audrunas Gruslys. Wojeiech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Mare Lanetot, Nicolas Sonnerat, Joel Leibo, Karl Tuyls, et al. Value- decomposition networks for cooperative multi-agent learning based on team reward. In In- ternational Conference on Autonomous Agents and Multi-Agent Systems, pages 2085-2087, 2018.
[22] Zichuan Liu, Yuanyang Zhu, and Chunlin Chen. NA Neural attention additive model for interpretable multi-agent Q-learning. In International Conference on Machine Learning, pages 22539-22555 2023.
[23] Rishabh Agarwal, Levi Melnick, Nicholas Frosst, Xuezhou Zhang, Ben Lengerich, Rich Caruana, and Geoffrey E Hinton. Neural additive models: Interpretable machine learning with neural nets. In Advances in Neural Information Processing Systems, pages 4699-4711, 2021.
[24] Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks universal approximators. Neural Networks, 255):359-366, 1989.
[25] George A Baker Jr and John L Gammel. The padé approximant. Journal of Mathematical Analysis and Applications, 2(1):21-30, 1961.
[26] Lisa Lorentzen. Padé approximation and continued fractions. Applied Numerical Mathematics, 60(12):1364-1370, 2010.
[27] Weixun Wang, Tianpei Yang. Yong Liu, Jianye Hao, Xiaotian Hao, Yujing Hu, Yingfeng Chen, Changjie Fan, and Yang Gao. Action semantics network: Considering the effects of actions in multiagent systems. In International Conference Learning Representations, 2020.
11

---

[28] Jiahui Li, Kun Kuang, Baoxiang Wang, Furui Liu, Long Chen, Changjie Fan, Fei Wu, and Jun Xiao. Deconfounded value decomposition for multi-agent reinforcement learning. In International Conference on Machine Learning, pages 12843-12856, 2022.
[29] Diederik P Kingma and Max Welling. Auto-encoding variational Bayes. In International Conference on Leurning Representations, 2014.
[30] Kyunghwan Son, Daewoo Kim, Wan Ju Kang, David Earl Hostallero, and Yung Yi. QTRAN: Learning to factorize with transformation for cooperative multi-agent reinforcement learning. In International Conference on Machine Learning, pages 5887-5896, 2019.
[31] Filippos Christianos, Lukas Schäfer, and Stefano V Albrecht. Shared experience actor-critic for multi-agent reinforcement learning. In Advances in Neural Information Processing Systems, pages 10707-10717, 2020.
[32] Mikayel Samvelyan, Tabish Rashid, Christian Schroeder De Witt, Gregory Farquhar, Nantas Nardelli, Tim GJ Rudner, Chia-Man Hung, Philip HS Torr, Jakob Foerster, and Shimon White- son. The StarCraft multi-agent challenge. In International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2019.
[33] Benjamin Ellis, Jonathan Cook, Skander Moalla, Mikayel Samvelyan, Mingfei Sun, Anuj Mahajan, Jakob Foerster, and Shimon Whiteson. SMACv2: An improved benchmark for cooperative multi-agent reinforcement learning. In Advances in Neural Information Processing Systems, pages 37567-37593, 2023.
[34] Jianhao Wang, Zhizhou Ren, Terry Liu, Yang Yu, and Chongjie Zhang. QPLEX: Duplex dueling multi-agent Q-learning In International Conference Learning Representations, 2020.
[35] Tabish Rashid, Gregory Farquhar, Bei Peng, and Shimon Whiteson. Weighted QMIX: Expand- ing monotonic value function factorisation for deep multi-agent reinforcement learning. In Advances in Neural Information Processing Systems, pages 10199-10210, 2020.
[36] Chenghao Li, Tonghan Wang, Chengjie Wu, Qianchuan Zhao, Jun Yang. and Chongjie Zhang. Celebrating diversity in shared multi-agent reinforcement learning. In Advances in Neural Information Processing Systems, pages 3991-4002, 2021.
[37] Jianhong Wang, Yuan Zhang, Yunjie Gu, and Tae-Kyun Kim. SHAQ: Incorporating Shapley value theory into multi-agent Q-learning. In Advances in Neural Information Processing Systems, pages 5941-5954, 2022.
[38] Yifan Zang. Jinmin He, Kai Li, Haobo Fu, Qiang Fu, Junliang Xing, and Jian Cheng. Automatic grouping for efficient cooperative multi-agent reinforcement learning. In Advances in Neural Information Processing Systems, pages 46 46105-46121, 2023.
[39] Haoyuan Qin, Chennan Ma, Deng Deng, Zhengzhu Liu, Songzhu Mei, Xinwang Liu, Cheng Wang, and Siqi Shen. The dormant neuron phenomenon in multi-agent reinforcement learning value factorization. In Advances in Neural Information Processing Systems, pages 35727-35759, 2024.
[40] Georgios Papoudakis, Filippos Christianos, Lukas Schäfer, and Stefano V. Albrecht. Bench- marking aulti-agent deep reinforcement learning algorithms in cooperative tasks. In Neural Information Processing Systems Track on Datasets and Benchmarks, 2021.
[41] Frans A Oliehoek, Matthijs TJ Spaan, and Nikos Vlassis. Optimal and approximate Q-value functions for decentralized pomdps. Journal of Artificial Intelligence Research, 32:289-353: 2008.
[42] Landon Kraemer and Bikramjit Banerjee. Multi-agent reinforcement learning as a rehearsal for decentralized planning. Neurocomputing, 190:82-94, 2016.
[43] Sarit Kraus, Amos Azaria, Jelena Fiosina, Maike Greve, Noam Hazon, Lutz Kolbe, Tim- Benjamin Lembeke, Jorg P Muller, Soren Schleibaum, and Mark Vollrath. Al for explaining decisions in multi-agent environments. In AAAI Conference on Artificial Intelligence, pages 13534-13538, 2020.
12

---

[44] Shariq Iqbal and Fei Sha. Actor-attention-ee for multi-agent reinforcement learning. In International Conference on Machine Learning, pages 2961-2970, 2019.
[45] Jiachen Yang, Igor Borovikov, and Hongyuan Zha. Hierarchical cooperative multi-agent reinforcement learning with skill discovery. In International Conference on Autonomous Agents and Auett- Systems, 2020.
[46] Lingxiang Yun, Di Wang, and Lin Li. Explainable multi-agent deep reinforcement learning for real-time demand response towards sustainable manufacturing. Applied Energy, 347:121324, 2023.
[47] Trevor Hastic and Robert Tibshirani. Generalized additive models, volume 43. CRC Press, 1990.
[48] Zhiwei Xu, Bin Zhang, Guangchong Zhou, Zeren Zhang, Guoliang Fan, et al. Dual self- awareness value decomposition framework without individual global max for cooperative MARL. In Advances Neural Information Processing Systems, pages 73898-73918, 2023.
13

---

A Related Work
Value Decomposition. Centralized training with decentralized execution (CTDE) has emerged as a powerful paradigm in MARL [41, 42], where global information can be accessed during centralized training and learned policies are executed with only local information a decentralized way. Under the CTDE paradigm, value decomposition methods show their strength in expressing the joint value function conditioned on individual value functions. VDN [21] introduces a linear decomposition, representing the joint Q-value as sum of agent-wise Q-values. However, its additive nature ignores inter-agent interactions, limiting its scalability to complex coordination tasks. QMIX [11] improves representational capacity by employing nonlinear monotonic mixing network parameterized via hypernetworks, but the imposed monotonicity constraint hinders its flcxibility. To overcome this, QTRAN [30] introduces relaxed transformation-bast based decomposition to bypass monotonicity, while WQMIX [35] incorporates a weighted projection to enhance approximation quality. QPLEX [34] further refines the decomposition by adopting duplex dueling architecture that satisfies the Individual- Global-Max (IGM) principle via an advantage-bare based formulation. Despite their improvements in expressiveness, these methods primarily focus functional accuracy and provide little insight into the underlying coordination structure. This lack of interpretability becomes particularly problematic in partially observable and interactioniintensive environments, where understanding agent dependencies is crucial for robust credit assignment. To address this, we propose a novel interpretable value decomposition framework that explicitly encodes high-order interactions, offering both performance and transparency.
Interpretable MARL. Recent advances in interpretable MARL can be broadly categorized into two paradigms, focusing either on (i) intrinsic interpretability or (ii) post-hoc explanation [43]. Intrinsic interpretabillity requires the learned model to be self- ueeeredddsttt by nature, which is achieved by using a transparent class of models, whereas post-hoc explanation entails learning a second model to explain already-trained black-box model. Post-hoc methods provide auxiliary insights without modifying the underlying learning process. For instance, SQDDPG [12] estimates individual agent contributions via Shapley Q-values, while Goto et al. [13] use masked-attention to identify salient observation regions in multi-vehicle coordination tasks. Although informative, these techniques lack robustness guarantees and struggle to recover the relational or temporal structure intrinsic to multi-agent cooperation. In contrast, intrinsically interpretable approaches seek to construct models whose decision logic is understandable by design. Tree-based architectures such as MIXRT [15] and MAVIPER [14] represent agent policies using soft or symbolic decision trees, providing explicit reasoning paths. DTPO [16] advances this line by directly optimizing tree structures via policy gradients. combining transparency with performance. Attention-base models, such as MAAC [44], further enhance interpretability by dynamically identifying inter-agent dependencies, while other methods promote explainability through latent skill inference [45] or constrained policy spaces that encode global objectives [46].
Within the value decomposition framework, central to cooperative MARL, several works also try understand how agents cooperate via agent-level contributions. VDN [21] factorizes the team reward additively, assuming agent independence. SHAQ [37] adopts Shapley value theory to quantify marginal contributions across coalitions. More recently, NA [22] expands the joint value function via a Taylor-like decomposition to capture high-order interactions. However, such expansions scale exponentially with the number of agents, leading to substantial computational and interpretability challenges. These limitations highlight the need for more principled and scalable formulation that can compactly model high-order agent interactions without sacrificing transparency. To this end, we propose a novel approach that integrates continued fraction networks into the value decomposition framework. By leveraging the recursive structure of continued fractions, our method enables compact and interpretable representations of arbitrary-order interactions while maintaining linear complexity with respect to the number of agents. This formulation provides a powerful alternative to polynomial expansion-bar -based methods, offering both expressive capacity and interpretability in large-scale cooperative MARL settings.
14

---

B Proof
B.1 Objective Functions for Variational Information Bottlenecks
Considering the Markov chain u² ++ h <> m, which means the assistive information cannot depend directly on the u*. So we have p(m h, x²) p(m h).
As in the IB, the objective can be written as:
(nn(Ø) [(m, u² :6) - (I h; Φ).
(15)
The 3 is to realize the trade-off between succinct representation and inferencing ability.
Theorem (Lower Bound for [m, u" Φ)). Let the representation m, be reparameterized as random variable drawn from multivariate Gaussian distribution m, [f (hg; Qm I). where
fm is an encoder parameterized by Φms h; denotes the hidden state of agent i, and is the identity covariance matrix. Then, the mutual information between the assistive information and the optimal joint action u² is lower-bounded as:
I(m, u² x) Ei logg [(hi; e))],
(16)
where q(ui mi) is a variational distribution approximating the true posterior p(u, ma), and m, = f(h,,)) denotes a deterministic funemiititiiicuniionn of h and the Gaussian random variable e.
Proof.
f dm,dui p (m,, ut) log p (mi, ut)
I(m, ==:
=
p (m;) (u))
i dm,du p (m, ut) log p (uî m))
(u))
where p (ut m:)) is fully defined by our encoder and Markov Chain as follows:
/ dhap(h;, u,] m.)
p(u, m,;)
/ dhap(ut hh)p(h; m.)
/ dhg plut ho)p(m ho)p(hs)
p(ma)
Since this is intractable in our case, let q(u, m;; be variational approximation to p(uţ m;).
where this is our decoder which we will take another neural network with its own of parameters. Using the fact that Kullback Leibler divergence is always positive, we have
KLip(ut m4), q(u;) m.)] 2 0
-
du, put mx) logp(u; m:) 2 / puup m;) log q(u. m:),
and hence
T(m, u² u ;)) 2 dmap((. m.) log q(u5 mi)
p(u{)
/ du p(u,, m.) log (uu) m²) - dut logp((()
f du;; dm u. m.) log q m:) H(u))
f dh,du, dm p (h pppp hh)p(mm h4) loug mz) + H(u))
f dmap (ma Ti) log m.) + H(u)).
15

---

Notice that the entropy of our labels (u;) ) is independent of our optimization procedure and so can be ignored. And we can rewrite p(m; h:)dm; p(e)de, m, = f(h,, e). So we have
N
1(m, ;
E Ee) |- log quu c(h,,
N
Theorem 2 (Upper Bound for 7(m. h; Φ)). Let ((m) denote variational approximation of the marginal distribution p(m)). Then, the mutual information between the representation m and the hidden state h admits the following upper bound:
I(m, h; Φ) S KL(p(m h4) (m)))
(17)
Proof.
/ dm,dh:p(h;; m.) log p(m,)
p(m= hi)
I(m, h: Φ)
dm,dh:p(hu m;) log p(m; h2) - dm..(mm)) log p(m:).
Let ((m;) be the variational approximation to the marginal distribution p(m:) = I dhap(m; | h;) h.)p( Since KL(p(m:), q(m:)] 2 0 - dmm.((m;) log logp(m.)) 2 dm::(mms) log @(m:), we
have
1(m, h; Φ) / dh.dm.p((h))pmmm h.) log p(m; h4)
((ms)
Σ p (m= hs) log p(m: | hj)
@(m=)
KL ip (m g(())
Combining Theorem I and Theorem we have the objective functions for variational information bottlenecks, which is to minimize
LVIB Σ
Ee-n(o) log (uf If (h,, c))] + BKL (m ((m))
(18)
B.2 Correspondence between Continued Fraction Depth and the Order of Agent Interactions
In this section, we establish one-to-one correspondence between the depth of the continued fraction network and the order of agent interactions. This property allows the d-th order continued fraction to accurately represent the d-th order approximation the agent's behavior.
Specifically, a continued fraction network of depth d, u w2f+
Q can be reformulated as
f(Q) Ta( Q) + O++1(11 (Q), where (Q) is degree-d polynomial of Q, and Oa+1(Q) denotes terms of order + 1 or higher in Q.
By setting a the continued fraction
can be transformed into wsQ+
w2Q
x
2
K(z)
(19)
W1 W2 Wa +...'
since these approximants are arranged along the "staircase diagonals" of the Padé table.
Theorem 3. For the d-th order truncation of the continued fraction Ry(2) (,(2) the following holds:
As(a)
d+1 1
Pa
qd
(20)
2
where pa deg(Aa),
qd deg (Ba).
16

---

Proof. The k-th order asymptotic function A+(z) satisfies the recursive relations: (z)
Ax(z)
= wk Ak (z) + -
Bx(2)
wi BB (2) 2 Bk 2 2
with
A-1 Ao
0
B_1
Bo
0 1
Assume that for all S n, the following holds:
k + 1
deg(Bx) k
deg(As)
2
Base Cases:
1:
1+1 1
A1(2) wi Ao(z) =2)1(2) = deg(A1) =1,
2
B1(z) un Bo(2) =B_1(22) wi deg( B1) 0,
0.
2:
A²(z) = W2 A1 (z) + =Ao(z) = w2 deg(Az) =
2 = 1.
(²(z) = (1) + (=) = w1 w2 + deg((B)) = 1,
= 1.
Hence, the base cases hold.
Then assume the statements hold for and k n and prove that they also hold for
k = n.
Degree of the Numerator An(z):
From the recurrence:
An(z) WnAnni(x)) #An-2(z),
and by the induction hypothesis:
deg(An-1)) dog(An-2)
Then:
deg( Wn An
71
deg(2 An = 1
n1
2
Case 1: even, let 2m:
n1
m
= m 1.
2
2
Then
n +
deg(wnAn-1) m, deg(zAn-2) = (m 1) m + deg(An) m
Case 2: n odd, let n 2m + 1:
1
m,
2
2
Then
1
deg deg(w,, An-1) = m,
deg(zAn-2) m = m + 1 deg(An) = m +
2
17

---

Degree of the Denominator B,,
From the recurrence:
Ba(z) = Wn Bn 12)) + z zBn BBn=((2)
and using the induction hypothesis:
n2 1
deg( (eg(Bn-2) 2
deg( Bn-1)
2
Then:
deg( wn Bn Bn-1)
11
deg( 2 (n =1
n
2
Case 1: n 2m (even):
n
n
= m 1,
m
2
2
Then
deg wn Bn-1) = m 1, deg(=Bn-2) = m + deg(Br) = m
n
Case 2: n 2m + (odd):
71
n
= m,
= m 1.
2
2
Then
n
dog( Ww, Bn- = m, dog(=Bn = m = (n,) = m
2
By mathematical induction, we conclude that:
deg(An) = n + 1
deg(B,)) = 12
2
Therefore, when the truncation order is = d, we have
+
Pa
qd
2
Theorem 4. The d-th order truncation of the continued fraction Ra(z) the conditions for Padé approximant, specifically:
Aa(z) B.(z)
naturally satisfies
f(z) Ra(z) O22
(21)
which means that its Taylor expansion coincides with the first d terms of the original function [z).
Proof.
Definition (Padé Approximant [25, 26]). Let C(z) = -E be a formal power series in the
variable z, then the Pudé approximant of order [L/M] rational function of the form:
(1) [Ar,M ( (B)))BBr,,()()
(22)
where AL, (2) and BL.M (2) are polynomials of degrees most L and M, respectively, chosen such that
(z) Ar,M( (
(23)
where notation O²2²) denotes some power series of the form In= This approximation minimizes the difference between the rational function and the power series up to the order M.
Ae(x) Since Ra(z) Ba(z)*
we have
As(z) Ba(z)
f(z)
(zp++outt))
(24)
which implies
f())Bo(z) As(z) O (ZpotRutl) Ba(x) =0 (2netuatl)
(25)
18

---

Lemma 1. For all & 2 0, there exists polynomial Sx(z) such that:
[z)) (x(()))))))))))))))) = (-1)* - Sx(z)
(26)
and the constant term (0) #
Proof. We can prove this lemma by mathematical induction.
Base Case:
k= == 0:
(()))o())-AAp) - = - f(z).
By the definition of continued fractions, f(z) "ait.., 80: 80:
1
f(z) = 2 (analytic function) =So(z)) So(0) + 0.
Inductive Hypothesis (k- 1 and A: - hold):
B1 - Ak-1(z) = (-1)a-12* Sk-1
[(z)Bx-2(z) - 2x= = (-1) 1
Sk 2())
for k:
Substituting the recurrence relations:
[(=)Ba(z) Ax(z) [(z) (x) ((z - =A 2()))
= wh (z)) (4 (f(2) A 2z))(z))
S (2) + -mx(-1)t-~~55-
2-)
(-1)* (2) + 2z))
= (-1)*==~~(((())
where (z) is the polynomial obtained from ww Sz-1 + Sa 2z=2(z), divided by z.
From the lemma, we have:
Aa(z)
f(z)Ba(z) Aa(z)
(-1)d=441Sa(2)
f(z)
Ba(z)
Ba(z)
Ba(z)
Since Ba(0) = Wh W2 ... wd + (assuming we + 0) and Sa(0) * 0, follows that:
f(z) Ra(z) ((z d11).
According to Theorem 3, the degree of the numerator As(2) is pd = d+1 and the degree of the denominator Bz(z) is qu = -gg. When is odd, we have = Qd qd d-1 In both cases, it follows that pa + qd = d. Therefore,
2
d. 2. when d is even, 2
d+1
f(z) Ra(z) = o (2 O (zpatect)),
which satisfies the condition of Padé approximant.
In conclusion, the depth-d continued fraction network represents the d-th order truncation of the continued fraction:
w Q+ w2Q+
w
which forms a [pa, 4a] Padé approximant with pa + qa d. This enables accurate representation of the first d-th order interactions among agents.
19

---

C Experimental Details
C.1 Algorithmic Description
Algorithm Continued Fraction Q-Learning
Initialize environment, agent network Qi (Ta, u4; u4:0). 0), target network Qi (T's u/; u/;0). mixing net-
work Qior. and VIB module Gg(Es, D&.) Initialize replay buffer D
2:
3: repeat
4: Obtain the initial global state s
5: for 0 T do
6:
For each agent i, get action-observation history T
7:
Calculate individual value function Q;
8:
Get the hidden state li
9:
Select action of via value function with probability exploration
10:
Execute joint action u', receive reward r', next state s'+1
11:
end for
12:
Store the episode trajectory in D
13:
Sample a mitc-batch B size b from D
14:
for f = 0 to T 1 do
15:
Calculate J4, o=Eq, (ht)
16:
Generate assistive information
17:
Get the attention weight og by the intervention function in Eq. 12 Calculate the joint value function Qtot
18:
19:
end for
Calculate loss C(0) LQ... + LVIB via Eq. 11 and Eq. 14. Update e and 0 by minimizing the above loss
21:
22:
Periodically update +- 0
23:
until Qi (Ti, uy; 0) converges or maximum steps reached
C.2 LBF Description and Hyperparameters Settings
Level-Based Foraging (LBF) [31] is mixed cooperative- competitive MARL benchmark, where each agent navi- gates a x 10 grid world. Agents and food items are randomly placed in 2D grid, and each one is assigned a level. A food item can only be collected when the com- bined levels of all participating agents equal exceed its level. The environment induces a spectrum of col- laborative behaviors through its level-dependent reward structure: while low-level food items permit independent collection, higher-level resources necessitate coalition for- mation. Furthermore, we set the penalty reward for move- ment -0.002, and the detailed hyperparameter settings of LBF are shown in Table 1.
Table 1:
The configurations LBF.
Hyperparameter
Value
Max Episode Length
50
Batch Size
32
Test Interval
10000
Test Episodes
32
Replay Batch Size
5000
Discount Factor
0.99
Start Exploration Rate
1.0
End Exploration Rate
0.05
Anneal Steps
50000
Steps
1M
Target Update Interval
200
Observation Space. Each agent observes a x 2 square grid centered on its own position. Within this range, the agent receives structured array containing the (z, y) coordinates and levels of all visible food items and other agents. This observation provides both spatial and attribute-leve information to support localized decision-making.
Action Space. The discrete action space for each agent consists none, move [direction]. and load food. Each agent only moves into one unoccupied grid. If multiple agents attempt to move into the same grid, collisions are resolved by canceling the conflicting moves, leaving the agents in their original positions.
20

---

Rewards. This reward depends on the food's level, which is distributed among the participating agents in proportion based on their levels. The rewards are normalized maintain a unit sum across all agents. This design ensures contribution-based fairness in reward distribution while enhancing cooperative efficiency among agents.
C.3 StarCraft II Description and Hyperparameters Settings
All implementations of algorithms are conducted on Star- Craft II unit marrommmnn tasks (SC2.4.10). We evaluate performance in combat scenarios where enemy units are controlled by the built-in AI with the difficulty=7 setting, and each allied unit is controlled by the decen- tralized agents with reinforcement learning algorithms. During battles, the agents seek to maximize the damage dealt to enemy units while minimizing damage received, requiring the coordination of diverse lactical skills. We assess our method across variety of challenging scenar- ios that differ in terms of symmetry, agent composition, and unit count (as shown in Table 3). For clarity, we also outline the core settings of the StarCraft Multi-Agent Chal- lenge (SMAC) [32], including observation, state, action, and reward configurations. The detailed hyperparameter settings SMAC are shown Table 2.
Table 2: The configurations of SMAC.
Hyperparameter
Value
Difficulty
7
Batch Size
32
Test Interval
10000
Test Episodes
32
Replay Batch Size
5000
Discount Factor
0.99
Start Exploration Rate
1.0
End Exploration Rate
0.05
Target Update Interval
200
Optimizer
RMSprop
Learning Rate
0.0005
Observations and States. At each time step, each agent receives local observation of units within its field view. The observation includes the following features for both allied and enemy units: distance, relative X and Y positions, health, shield, and unit type. Note that the agents can only observe the others if they are alive and within their line of sight range, which is set to 9. When unit (ally enemy) becomes invisible is eliminated, its feature vector is reset to all zeros, indicating either death or being outside the field of view. The global state is only available during centralized training, which contains information about all units the map. Finally, all features, including the global state and the observation the agent, are normalized by their maximum values.
Action Space. Each unit takes an action from the discrete action set: no-op, stop, move [direction], and attack [enemy id]. Agents are allowed to move with a fixed movement amount in four directions: north, south, east, and west, where the unit is allowed to take the attack [enemy id] action only when the enemy within its shooting range.
The target goal is to maximize the win rate for each battle scenario. At each time step, the agents receive a shaped reward based on the hit- damage dealt and enemy units killed, as well as a special bonus for winning the battle. Additionally, agents obtain a 10 positive bonus after killing each enemy and a 200 bonus when killing all enemies, which consistent with the default reward function of the SMAC.
Rewards.
Table 3: The StarCraft Multi-Agent Challenge benchmark.
Map
Ally Units
Enemy Units
Difficulty
Steps
Anneal Steps
d
2s3z
Stalkers, Zealots 2 Stalkers, 3 Zealots
Eazy
1.5M
50000
2
2c vs_64zg
2 Colossus
64 Zerglings
Hard
2M
50000
2
3s_vs__2
Stalkers
Zealots
Hard
2M
50000
2
5m_vs_6n
Marines
6 Marines
Hard
2M
50000
4
3s52, vs. 3s62
Stalkers, Zealots
3 Stalkers, 6 Zealots
Super Hard
5M
200000
6
6h vs_ 8z.
6 Hydralissks
Zealots
Super Hard
5M
200000
21

---

C.4 SMACv2 Description and Hyperparameters Settings
SMACv2 [33] an enhanced benchmark for cooperative multi-agent reinforcement learning built on top of StarCraft II. It preserves the original SMAC API while introducing three procedural innovations increase scenario diversity and challenge contemporary MARL algorithms: randomising start positions, randomising unit types, and changing the unit sight and attack ranges.
Randomized Start Positions.
Allied and enemy units are spawned either in a "surround" configuration, where enemies encircle the allies, via "reflect" scheme that mirrors allied positions across the map center. This ensures that agents cannot overfit to fixed spawn patterns.
Table 4: The configurations of SMACv2.
Race
Unit
probability
Stalker
0.45
Protoss
Zealot
0.45
Randomized Unit Types. Each battle can feature mixed unit compositions rather than uniform rosters. For Terran, Protoss, and Zerg, three unit types are sampled with config- urable probabilities through the team_gee _een distribution (as shown in Table 4), promoting adaptable strategies under varied team makeups.
Colossus
0.1
Marine
0.45
Terran
Marauder
0.45
Medivac
0.1
Zergling
0.45
Zerg
Hydralisk
0.45
Unit Sight and Attack Ranges. Unit vision cones and attack radii are aligned with their true in-game values, increasing realism and preventing agents from exploiting the simplified ranges used in SMAC.
Baneling
0.1
C.5 Implementation Details
We compare our method against nine value-based baselines, including VDN [21], QMIX [11], QPLEX [34], Centrally-Weighe QMIX (CW-QMIX) [35], CDS [36], SHAQ [37], GoMARL [38], ReBorn [39], and N [22]. To ensure fairness, we implement all experiments within the PyMARL framework All hyperparameters of baselines are set identically to method compare algorithms fairly. Please refer to PyMARL's open-source implementation for further training details and fair comparison settings. The depth d of CFN is determined based on the scale of agents and the complexity of each task.
All scenarios are trained on system equipped with an NVIDIA RTX 3080TI GPU and an Intel 19-12900k CPU, with training time ranging from 16 hours per scenario, depending on the task complexity and episode length.
"The source code of implementations from https://githubb com/oxxhir1lppmmrr...
22

---

Q1
Q2
Qu
Q Q2... Qn
(a) CFN-C
(b) CFN-D
(c) CFN
Figure 10: Three variants of the CFN architecture. (a) CFN-C integrates both single-feature ladders,
where each ladder processes single input dimension Q and full ladders, which take the complete set of individual values employs only the full ladders with increasing depth.
input. (b) CFN-D utilizes only the single-feature ladders. (c) CFN
C.6 Detailed Description of CFN Structure
As illustrated in Fig. 10, the CFN framework includes two structural variants addition to the main architecture. Fig. 10(a) presents CFN-C, composite architecture inspired by CoFrNet [18], which combines two types of ladders: single-feature ladders, each processing an individual agent utility Q, and full-input ladders, which receive the complete utility vector Q at every layer. Each ladder yields partial joint value QA. and the aggregation of all ladders constitutes the final joint Q-value.
The number of single-feature ladders equals the number agents, enabling additive modeling of individual effects. In contrast, full-input ladders are deeper and designed to capture complex joint dependencies among agents by recursively combining all inputs, thereby facilitating high-order interaction modeling.
Fig. 10(b) and Fig. 10(c) depict two simplified variants: 1) CFN-D, which retains only the single- feature ladders, thereby modeling additive effects with strong transparency [47] but lacking the capacity to express inter-agent interactions; 2) CFN, which retains only the full-input ladders, striking balance between modeling power and computational efficiency.
In our QCoFr algorithm, we adopt the CFN structure with full-input ladders as the default architecture. Compared to CFN-C, this version significantly reduces parameter overhead while preserving the ability capture arbitrary-order interactions. We further include CFN-D as an ablation baseline isolate the contribution of high-order modeling: while CFN-D offers interpretability due to its decom- posable additive form, its inability encode dependencies across agents limits its expressiveness in cooperative settings.
Finally, key advantage of the CFN structure is its linear scalability: the number of parameters grows as ((n) with the number of agents, making particularly suited to large-scale MARL scenarios where modeling expressive joint behavior is critical without incurring prohibitive computational costs.
23

---

no-op
stop
north
south
east
west
attack (enemy id)
2
3
4
5
6
7
8
10
10
Agents
6
10
20
30
40
50
60
70
80
90
100
110
Training Steps
(a) Actions of each agent.
O,
00
0.236
PL
0.94
1,213
L172
1.316
8408
8,672
0,075
(b) t=
(c) t
P.0.
0.991
2.609
1.129
0,568
1.500
1.401
1.397
1.834
1.465
1.195
Aafa
0.317
0.922
0.854
1,190
B.821
0.145
(d) t- 85
(c) t 100
Figure 11: Visualization of evaluation results for QCoFr on 3s vs__5_52 map. Agents demonstrate coordinated riee-and-focus-efe strategy: agent 2 initially kites four enemies alone, while agent and agent 3 eliminate another. Agent 3 then draws away two of the remaining enemies, enabling agent I and agent to dispatch the others. Finally, agents regroup to defeat the last enemies.
D Extended Interpretability Analysis
Fig. 11 illustrates the interpretability of QCoFr on 3s_vs_5e scenario. At the beginning of the episode, agent 2 independently kites four enemies, creating numerical advantage that enables agent I and agent 3 to quickly eliminate an isolated opponent. As result, agent receives the highest individual contribution score (1.337), while the strongest pairwise contribution observed between agents and due to their effective coordination. As the engagement progresses, agent 3 draws two enemies away, allowing agent and agent 2 to jointly take down the remaining targets. During this phase, the coalition contribution of agents and 2 increases, and agent 3's individual contribution also rises as delays the enemy. After these enemies are defeated, all three agents regroup to focus fire on the remaining units, resulting in more balanced distribution of credit across agents. This case study demonstrates that the agents have learned iite-and-focuss observed behaviors and quantitative contribution values confirms the interpretability of QCoFr, which faithfully attributes both individual and coalition-level contributions with high-order interactions in executing complex cooperative tacties.
strategy. The alignment between
24

---

VDN
QMIX
Qof w/o VIB
QCofr (Ours)
8m_ vs_ 9m
corridor
3$52_ ___2__s_______ 3962
100
100
100
Test Win Rate $8
Test: Win Rate
Ama
40
40
20
20
20
0.50
1.50
2.00
1.00
0
3.00
5.00
T (mil)
(mil)
T (mil)
Figure 12: Performance with and without VIB on three extra scenarios of the SMAC benchmark.
VDN
QMIX
NAOQ
NAOO VIB
QCofr (Ours)
corridor
3s5z
3s6z
6h vs 8z
100
100
80
Test Win Rate *
80
Test Win Rate %
80
Test Wie Rate %
60
60
60
40
40
220
0
1.00
2.00
3.00
4.00
5.00
0
1.00
2.00
3.00
4.00
5.00
1.00
2.00
3,00
4.00
5.00
(mil)
Figure 13: Performance comparison of NA Q with the VIB module and our method.
E
Additional Experiments SMAC
E.1
Additional Ablation Experiments
The Role of the VIB Module. We ablate the VIB component on three additional SMAC scenarios (Fig. /2), comparing QCoFr with and without VIB under identical settings. With VIB, QCoFr consistently accelerates early learning and achieves higher test win rates, confirming that task-relevant assistive information improves credit assignment and coordination.
The Role of CFN. Since N struggles to model higher-order interactions, we equip it with the same VIB module and evaluate on three super-hard SMAC maps, comparing against QCoFr (Fig. 13). This isolates the effect of interaction modeling from that of assistive information. While N QQ+VIB outperforms the original N clear gap remains to QCoFr. The results indicate that, complex tasks, explicitly modeling higher-orde dependencies enables more refined cooperative strategies, which highlights the effectiveness of the CFN module beyond what VIB alone can provide.
VON
QMIX
QCofr w/o IGM
QCofr (Ours)
2c vs. 64zg
3s52, V 3s62
6h vs 8z
100
100
80
3e 80
3 80
Test Win Rate %
60
Test Win Rate
Test Win Rate
60
60
40
20
0.50
1.00
1.50
2.00
1.00
2,00
3.00
4.00
5.00
1.00
2.00
3.00
4.00
5.00
(mil)
(mil)
(mil)
Figure 14: Performance comparison of QCoFr with and without IGM constraint.
25

---

E.2 Discussion on the IGM Constraint
To isolate the effect of the continued-fractione mixing paradigm from mon-monotonic joint-action search, we enforce the Individual SetviddullGGlbal-Meee Global-Max (IGM) constraint in our framework. Notably, the universal approximation theorem applies to any linear combination of continued fractions and does not require non-negative weights [18], suggesting that the approach can be extended to non-IGM mixers. Integrating CFN with fully IGM-free methods such DAVE [48] is therefore natural direction. DAVE emphasizes that most value decomposition methods operate under IGM, which couples the optimal joint action with the optimal individual actions. Relaxing this constraint requires agents to explicitly search for the globally optimal joint action at execution time, often via an auxiliary network. To probe this possibility, we conduct experiments under relaxed IGM assumptions on three SMAC scenarios. As shown in Fig. 14, QCoFr achieves comparable or even slightly improved performance without IGM, indicating that our architecture can still recover high-quality joint actions.
26

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA]-
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked While "[Yes] " is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] provvdded a proper justification is given (e.g., "error bars expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
the checklist one of the factors in their evaluation.
reported because it would be too computationally
"[No] " or "[NA] is not grounds for rejection. While the questions are phrased acknowledge that the true answer often more nuanced, so please just use your best, judgment and write justification elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope. The method can be found in Section C.6 and 4. Experimental results are illustrated in Section The detailed proofs and experimental settings can be found in the Appendix.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
27

---

Answer: [Yes]
Justification:
We include the limitations in our conclusion. We leave this part as our future
work.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We have provided the full set of assumptions and corresponding complete proofs under each proposed Theorem with detailed derivations included in the Appendix B.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
28

---

Justification: In addition to the description in the paper, we included more detailed hyperpa- rameters settings in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: The model architecture and hyperparameter settings are all included in this work. We believe we have provided enough details. We will make the code available in the near future.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips) cc/
Bublic/guides/Cee Setiddds/CooeSSumisiiiiioo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: //nips. wttppcccpppcccccccccccccccc.c.ll httpp/ppplgguudss///epppppp httpl////guuiee//edeuubaaaaeeees Soooeesubmissioooolliiiii
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
29

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The details included in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification; For every experiment introduced, run with multiple random seeds and reported both mean and std averaged over these multiple random seeds.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars are reported in tables or plots. The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
30

---

Justification: As mentioned in Appendix, our model runs on an NVIDIA RTX 3080TI GPU and an Intel 19-12900k CPU.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https : http://twrrrcccccce//. httpppp....ppppccccccccceeeeeeee
Answer: [Yes]
Justification:
We are strictly following NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our paper proposes a novel method for MARL, and we do not think there would be any direct social impact of it.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
31

---

Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cited all the creators original owners of assets mentioned in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcee thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
32

---

14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy (https: / ctt.iiissscc/cooffeeeeeeee for what should or should not be described.
33


Reviews:
[{"review_id": "fjpdYW6XN0", "reviewer": "Reviewer_PxnY", "summary": "This paper proposes a novel MARL value-decomposition paradigm. By introducing continued fraction modules, the authors achieve two key improvements: reducing computational complexity for higher-order terms and realizing interpretable value decomposition. Furthermore, the authors construct a practical value decomposition approach named QCoFr by integrating Continued Fraction Network (CFN) and Variational Information Bottleneck (VIB) modules. Experimental results demonstrate the superiority of QCoFr compared to existing value-based methods, supported by ablation studies and interpretability experiments.", "questions": "In addition to the concern mentioned above, I would like the authors to address the following questions:\n\n1.Linking value decomposition with agent interactions feels somewhat counterintuitive. Do the authors have any deeper insights on this?\n\n2.Why was Qatten not mentioned in the method discussion, given that it also implements a Taylor-like decomposition among agents? This is also an interpretable algorithm. I suggest the authors analyze the complexity of the terms in this algorithm and conduct a thorough comparison (beyond experiments) with it.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**: \n\n(Major) From an innovation perspective, the authors propose a novel, interpretable value decomposition method and provide a universal approximation theorem proof under the MARL framework. \n\n(Major) In terms of writing, the paper is well-written with a clear structure. The illustrations are precise and facilitate comprehension and reading. \n\n(Minor) The comparative experiments are comprehensive, with thorough baseline and benchmark selections.\n\n---\n\n**Weaknesses**: I will consider recommending acceptance if the authors satisfactorily address my concerns:\n\n(Major) I am particularly perplexed by the role of the VIB model in the overall methodology and paper. From the perspective of proposing a novel value decomposition paradigm, the CFN model appears sufficient. The VIB model seems unnecessary in this paper, contributing neither to interpretability nor computational complexity reduction, and potentially disrupting the reading experience. Moreover, the ablation studies suggest that the VIB model does not decisively improve performance. Consequently, I reasonably suspect the VIB model might be artificially introduced to inflate innovation claims.\n\n(Major) Why does the Mixing method still adhere to the IGM principle (is this a regression)? Numerous studies have already explored value decomposition without IGM constraints, demonstrating superior performance on complex tasks compared to QMIX and VDN. Theoretically, removing IGM constraints should enhance representational capacity. I recommend the authors address either: a. the necessity of maintaining IGM (e.g., theoretical integration with CFN) or b. the applicability of the proposed method under non-IGM conditions.\n\n(Major) While the initial sections of the paper sparked my interest in the method's interpretability, the experimental results fail to adequately demonstrate its superiority in this dimension. What is the authors' definition of interpretability? The experiments could be more comprehensive, such as discussing order-related influences or providing more extensive visualizations. Overall, the interpretability experiments are insufficient. (Note: I personally believe that the visualization analysis of diversity is not strongly related to the interpretability of the proposed method. This analysis only demonstrates that the proposed method can learn diverse strategies more quickly in certain scenarios, but the method itself is not inherently related to diversity.)", "was_revised": true, "final_justification": "Most of my concerns addressed.", "rebuttal": "Dear Reviewer,\n\nWe appreciate your insightful comments and provide our response as follows.\n\n1. **The role of the VIB.**\n\n   **Response:** \n   We would like to emphasize that VIB is an important module in our method by promoting credit assignment in cooperative MARL. Credit assignment aims to attribute global team success to individual agent contributions. Due to partial observation, there exists a potential backdoor path $\\boldsymbol{s} \\leftarrow E \\rightarrow \\tau \\rightarrow Q_{tot}$ between the global state $\\boldsymbol{s}$ and $Q_{tot}$, which can lead to a spurious correlation that causes confounding bias for estimating credit assignment. Following prior works [R1, R2], we address this by intervening on the global state $\\boldsymbol{s}$ using local history $\\tau_i$ via VIB. To this end, we encode the agent's history $\\boldsymbol{h}$ into a latent representation $\\boldsymbol{m}$, forming a Markov chain $\\boldsymbol{h} \\rightarrow \\boldsymbol{m} \\rightarrow \\boldsymbol{u}^*$, where $\\boldsymbol{m}$ is optimized to be maximally informative for predicting the optimal joint action. This can encourage $\\boldsymbol{m}$ to memorize $\\boldsymbol{u}^\\*$ with meaningful information. This latent $\\boldsymbol{m}$ acts as assistive information during value factorization to reduce confounding and promote policy learning.\n\n   Empirically, while the gains from VIB appear modest on simple tasks, it accelerates the early stage and becomes critical in harder scenarios. On the super-hard map 6h_vs_8z, removing VIB results in a significant performance drop, indicating the necessity of additional latent information for complex coordination. As shown in Table 1, across one hard and two super-hard tasks, QCoFr with VIB achieves better results, which confirms the effectiveness of leveraging additional information.\n       \n    ### Table 1. Additional comparison on 3 extra scenarios.\n    \n    | Method        |     8m_vs_9m      |       corridor    |  3s5z_vs_3s6z     |     \n    | :------------:| :---------------: | :---------------: | :---------------: | \n    | VDN           |   0.80$\\pm$0.11   |   0.46$\\pm$0.36   |   0.64$\\pm$0.33   |   \n    | QMIX          |   0.75$\\pm$0.11   |   0.01$\\pm$0.01   |   0.61$\\pm$0.19   |  \n    | QCoFr w/o VIB |   0.82$\\pm$0.07   |   0.60$\\pm$0.32   |   0.65$\\pm$0.25   |   \n    | QCoFr (Ours)  |   **0.93$\\pm$0.02**   |   **0.79$\\pm$0.03**   |   **0.87$\\pm$0.07**   |  \n\n\n\n   [R1] Jiahui Li, et. al., Deconfounded value decomposition for multi-agent reinforcement learning, ICML, 2022.\n\n   [R2] Zichuan Liu, et. al., NA$^2$Q : Neural attention additive model for interpretable multi-agent Q-learning. ICML, 2023.\n\n2. **IGM Constraint.**\n\n   **Response:** Our goal in this work is to introduce a novel high‑order value decomposition paradigm via continued fractions. Retaining the IGM constraint allows us to isolate the benefits of the continued fraction structure without conflating them with non‑monotonic search mechanisms. The universal approximation theorem we prove applies to any linear combination of continued fractions; it does not rely on non‑negative weights, so the theory could extend to non‑IGM mixers. Integrating CFN with fully IGM‑free methods like DAVE is an interesting direction [R3], but it is outside the scope of this submission. DAVE points out that value decomposition methods typically operate under IGM, which links the optimal joint action to the optimal individual action. Removing IGM requires agents to search for the global best action at execution time and often entails an additional network.\n\n   To probe this, we conducted additional experiments relaxing the IGM constraint. Results in Table 2 show QCoFr achieves comparable or slightly improved performance without IGM, indicating that our architecture can still recover high-quality joint actions. We will make this discussion explicit in the revised paper.\n\n    ### Table 2. Comparison with and without IGM constraint.\n    \n    | Method        |     2c_vs_64zg    |     3s5z_vs_3s6z  |      6h_vs_8z     |     \n    | :------------:| :---------------: | :---------------: | :---------------: | \n    | VDN           |   0.88$\\pm$0.05   |   0.64$\\pm$0.33   |   0.00$\\pm$0.00   |   \n    | QMIX          |   0.96$\\pm$0.05   |   0.61$\\pm$0.19   |   0.11$\\pm$0.19   |  \n    | QCoFr w/o IGM |   0.96$\\pm$0.04   |   **0.89$\\pm$0.07**   |   **0.67$\\pm$0.07**   |   \n    | QCoFr (Ours)  |   **0.96$\\pm$0.03**   |   0.87$\\pm$0.07   |   0.58$\\pm$0.12   |  \n    \n\n   [R3] Zhiwei Xu, et. al., Dual self-awareness value decomposition framework without individual global max for cooperative MARL. NeurIPS, 2023.\n\n3. **Additional interpretability experiments.**\n\n   **Response:** Thank you for prompting further clarification on interpretability. Our motivation is to design an interpretable model where agent interactions can be easily understood. The continued fraction structure of CFN enables symbolic representation of the Q-function, which can be analytically expanded into a multivariate power series using tools such as *Mathematica*. This expansion yields exact attributions for both individual agent terms and higher-order interactions.\n   On the 5m_vs_6m scenario (in Figure 8), a CFN with a shallow depth $d=2$ produces the symbolic expression:\n\n   $$ \\alpha_1 \\cdot \\frac{1}{0.2455 Q_1 + 0.3093 Q_2 + 0.0015 Q_3 + 0.2326 Q_4 + 0.0459 Q_5 - 1.1213 + \\frac{1}{0.0977 Q_1 +         0.2005 Q_2 + 0.2859 Q_3 + 0.2732 Q_4 + 0.1135 Q_5 + 0.4600}} + \\alpha_2 \\cdot \\frac{1}{0.2277 Q_1 + 0.3581 Q_2 + 0.0329 Q_3 + 0.1803 Q_4 + 0.0367 Q_5 + 1.4818 - \\frac{1}{0.2311}} +   f_0,$$\n\n   where $\\alpha_k$ denotes the attention weight assigned to each ladder varing over time. Expanding this at $t=10$ into a power series up to 2-order yields:\n\n   $$Q_{tot}= 0.08Q_1+0.26Q_2+0.60Q_3+0.46Q_4+0.22Q_5-0.06Q_1Q_2-0.01Q_1 Q_3-0.05Q_1Q_4-0.01Q_1Q_5+0.21Q_2Q_3+0.07Q_2Q_4+0.06Q_2Q_5+0.50Q_3Q_4+0.27Q_3Q_5+0.16Q_4Q_5+\\cdots.$$\n\n   We observe that the terms with the largest coefficients, such as $Q_3Q_5$,$Q_3Q_4$,$Q_2Q_3$, align well with the most critical agent combinations shown in Figure 8. \n   For a deeper CFN with $d=4$, we omit the full expression due to space constraints\n\n    $$\\frac{1}{0.2804 Q_1 +  \\cdots + 0.4022 Q_5 - 2.0528+\\frac{1}{0.0896 Q_1 +  \\cdots + 0.0590 Q_5 - 0.1697 + \\frac{1}{0.3713 Q_1 + \\cdots + 0.2525 Q_5 - 0.2219+\\frac{1}{0.2575 Q_1 + \\cdots + 0.3108 Q_5 + 0.3811}}}}\\cdots .$$\n\n   At $t=10$, we extract the top-10 terms (by coefficient magnitude):\n\n    | term     | coefficients | term | coefficients |\n    | --------- | --------------- | ----- | ------ |\n    |  $Q_2 Q_3$ | 0.579 | $Q_1 Q_2 Q_3 Q_5$ | 0.449 |\n    |  $Q_2 Q_3 Q_4$ | 0.579 | $Q_3 Q_4 Q_5$ | 0.447|\n    |  $Q_1 Q_2 Q_4$ | 0.567 | $Q_1  Q_3 Q_5$ | 0.375 |\n    |  $Q_2 Q_4 Q_5$ | 0.543 | $Q_1 Q_4 Q_5$ | 0.123 |\n    |  $Q_1 Q_3 Q_4$ | 0.469| $Q_1 Q_2 Q_5$ | 0.098 |\n\n   This result explicitly quantifies the contribution of higher-order interaction terms to their team, most of which align with the top-ranked agent combinations highlighted in Figure 8. In contrast, prior methods limited to unary or pairwise terms fail to capture these complex yet key patterns of cooperation with more than 3-order interactions.\n\n4. **The connection between value decomposition and agent interactions.**\n\n   **Response:** Thank you for highlighting this point. We believe that treating value decomposition as a structure of interaction orders is both theoretically justified and practically advantageous. In cooperative MARL, the joint Q-function encodes not only the individual contribution of each agent but also the surplus value generated when agents cooperate. It resembles Shapley value and synergy decomposition: first‑order terms capture individual effects, while higher‑order terms quantify the effects of coalitions [R4]. Such an interaction modeling helps avoid over‑ or under‑estimating any agent's contribution and provides a clear credit assignment estimation—interaction terms reflect cooperation patterns. VDN considers 1-order interaction and takes a sum of individual values, which may limit its capacity to ignore higher-order interactions. QMIX theoretically can represent any order interaction relationship among agents with two non-linear networks. QCoFr theoretically not only can own the same capacity as QMIX but also explicitly model all possible higher-order interactions with linear‑time complexity.\n\n   [R4] Jianhong Wang, et al., SHAQ: Incorporating Shapley Value theory into multi-agent Q-learning. NeurIPS, 2022.\n\n\n5. **Discussion of Qatten.**\n\n   **Response:** We agree that Qatten is a related baseline. We will add it in the revision and summarize the key differences here:\n\n   • Complexity:  Qatten relies on multi‑head attention, where each head computes a full similarity matrix over all agents, yielding $O(n^2)$ computational cost. In contrast, our CFN recursively aggregates individual utilities, leading to $O(n d)$ complexity where $d$ is the depth.\n\n   • Interpretability:  In Qatten, attention weights indicate agent importance, but there is no direct correspondence between heads and interaction orders. QCoFr explicit models of higher‑order interactions avoid the combinatorial explosion.\n\n   • Performance: We have conducted extended experiments on SMAC, where QCoFr achieves higher gains than Qatten on a range of tasks, which indicates the superiority of considering high-order interactions.\n\n    ### Table 3. Performance Comparison with Qatten on 3  maps.\n    | Method        |     5m_vs_6m      |    3s5z_vs_3s6z    |    6h_vs_8z   |     \n    | :------------:| :---------------: | :---------------: | :---------------: | \n    | VDN           |   0.64 ± 0.06   |   0.64 ± 0.33   |   0.00 ± 0.00   |   \n    | QMIX          |   0.66 ± 0.16   |   0.61 ± 0.19   |   0.11 ± 0.19   |  \n    | Qatten        |   0.64 ± 0.13   |   0.04 ± 0.04   |   0.01 ± 0.03   |   \n    | QCoFr (Ours)  |   **0.78 ± 0.09**   |   **0.87 ± 0.07**   |   **0.58 ± 0.12**  |"}, {"review_id": "I6nOAANZIK", "reviewer": "Reviewer_CAhf", "summary": "This paper proposes a new method named QCoFr that uses higher order interactions inspired by Taylor series to improve the decomposition of joint Q functions in a manner that is more interpretable.", "questions": "Please find some questions below.\n1. the architecture of the proposed method seems quite close to NA2Q's architecture; could the authors elaborate better how their method is different? this could be made more clear in the paper\n2. what is the intuition behind predicting the joint optimal action from the history of observations (as shown in figure 2)? what information can that give to learn a good value for $m$?\n3. there are several claims regarding better credit assignment across the paper; however, from my understanding there is not clear evidence that the credit is being correctly assigned as a product of the proposed method; i can find the contribution levels illustrated, but are these reflected in the credit given to the agents?\n4. since the proposed method is closely aligned with one of the related methods Na^2Q, it would be interesting to see how they compare in terms of interpretability, since the authors also mention that this method also enables interpretability (lines 89-92); has this been tried?\n5. in figure 9, the q-values similarities between QMIX and QCoFr are quite close; while when comparing to VDN the difference is clear, with QMIX we could argue that QCoFr is not necessarily producing that much diverse behaviours when compared to QMIX; could the authors elaborate on this? \n6. in section 5.2, in the interaction orders ablation: \"These findings demonstrate that modelling higher-order interactions yields performance improvements for complex coordination tasks, but only within a bounded regime that aligns with the task’s intrinsic interaction complexity\" (lines 270-273) - this is an interesting observation but it shows that for different environments the best depth level can be different; have the authors further explored these findings? is it possible to theoretically define this bounding that can create a balance between performance improvements and scalability?", "limitations": "These have been discussed in the conclusion and are appropriate. Please also refer to my comments above for other potential limitations.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Overall, this paper is well written and generally well structured. The proposed method shows very strong performances in different tasks, outperforming some of the current SOTA methods in some of the environments tested. The method is also theoretically grounded.\n\nHowever, there are some less clear points that I list below; specifically, I am also a bit confused regarding the novelty of this work when compared to the previous NA2Q [1] that is mentioned in this paper.\n\n1. There are several losses represented in figure 2 but it is not clear how some of these losses are combined and their contribution/exact meaning (CE loss for example)\n2. The proofs in the appendix are not presented in a very intuitive manner; for example, after reading section 3 and definition 1, the authors refer to a proof but when we check the appendix this proof is difficult to quickly find; I suggest the authors could clarify where in the appendix the proofs are in general\n3. in the end of section 3, two variations of the proposed method are mentioned: CFN-D and CFN-C; however, in the main paper it is not clear how these are different from normal CFN\n4. the architecture of the proposed method seems close to the NA2Q's architecture and it is not very clear how both methods clearly differentiate from each other\n\n### Minor:\n* in figure 9, it would be good to have the titles of the subfigures indicating to which baseline the matrices correspond to\n\n[1] https://arxiv.org/abs/2304.13383", "was_revised": true, "final_justification": "Most of unclarities were addressed by the authors, hence I have reflected that in my score.", "rebuttal": "We sincerely thank you for your time in providing detailed feedback on the paper to better highlight the key contributions and novelties of the paper. We detail our responses to the questions below.\n\n1. **Relation to NA$^2$Q and novel contributions.**\n\n   **Response:** Thank you for raising this important point. While our method shares the high-level goal of modeling interactions among agents with NA$^2$Q, the architectural design and core contributions differ significantly:\n\n   NA$^2$Q uses the generalized additive models (GAM) to enumerate all possible agent coalition terms up to the full interaction order, incurring exponential complexity $\\sum_{i=1}^n C_n^i = 2^n - 1$. This becomes infeasible in large-scale or high-order coordination settings. In contrast, QCoFr introduces a CFN whose depth $d$ corresponds directly to the interaction order, enabling explicit higher-order interaction modeling with linear computational complexity $O(n)$. To our knowledge, QCoFr is able to provide high-order cooperation patterns and tractable learning of arbitrary-order interactions in linear time. Regarding the advantage of QCoFr, please see the details of the response to **Reviewer PxnY**.\n\n   QCoFr adopts a VIB to encode assistive information $\\boldsymbol{m} \\sim q(\\boldsymbol{m}|\\boldsymbol{h})$ predicting $\\boldsymbol{u}^*$, whereas NA$^2$Q relies on a VAE-based module for capturing latent semantics. This improves credit estimation by selectively filtering out confounding factors in partial observability, aligning the encoding with optimal decision-making. We further evaluated NA$^2$Q, with its VAE module replaced by our VIB design and observed consistent performance gains, reinforcing the benefit of our approach.\n\n2. **Intuition behind predicting the joint optimal action.**  \n\n   **Response:** The goal is to address the credit assignment problem in cooperative MARL: how to infer the contribution of each agent or coalition to the team under partial observability. When agents condition their utilities on the global state $\\boldsymbol{s}$, there is a backdoor path  $\\boldsymbol{s} \\leftarrow E \\rightarrow \\tau \\rightarrow Q_{tot}$, where $E$ denotes unobserved environment dynamics and $\\tau$ an agent’s history. This path can induce spurious correlations, causing the mixing network to factorize credit based on confounders rather than genuine causal effects. To break this path, we follow the principle used in other works, such as PAC [R1] and IMAC [R2], generate assistive information by predicting the joint optimal action $\\boldsymbol{u^*}$ from each agent's history and then encode it via VIB.\n\n   In detail, the assistive encoder takes the local history $h_i$ and outputs a latent variable $m_i$. We train this encoder with a cross‑entropy loss to predict the joint optimal action $\\boldsymbol{u^\\*}$; at the same time, a KL‑divergence term constrains the mutual information $I\\left(\\boldsymbol{m}, \\boldsymbol{u}^\\* ; \\boldsymbol{\\phi}\\right)$.  This forms a Markov chain $\\boldsymbol{h}-\\boldsymbol{m}-\\boldsymbol{u^\\*}$.  The intuition is that if $\\boldsymbol{m}$ can predict $\\boldsymbol{u^\\*}$ well, it must encode precisely those features of the agent's history that are relevant for selecting the optimal joint action, while the information bottleneck discourages encoding irrelevant details. This helps the mixing network facilitate more accurate credit assignment conditioning on $\\boldsymbol{m}$ along with the global state $\\boldsymbol{s}$ whose local histories are genuinely informative for the joint decision, and those whose histories are noisy or misleading. We will highlight this causal perspective and the connection to the PAC framework in the revised manuscript.\n\n   [R1] Hanhan Zhou, et. al., PAC: Assisted value factorisation with counterfactual predictions in multi-agent reinforcement learning, NeurIPS, 2022.\n\n   [R2] Rundong Wang, et. al., Learning efficient multi-agent communication: an information bottleneck approach, ICML, 2020.\n\n3. **Credit Assignment.**\n\n   **Response:** In our work, the notion of *contribution* refers to the product of each agent’s individual Q-value (or multi-agent interaction term) and the credit assigned to it. For example, a third-order interaction among agents $i, j, k$ is represented as $Q_i Q_j Q_k$, and its contribution is defined as $\\beta_{ijk} Q_i Q_j Q_k$, where $\\beta_{ijk}$ denotes the assigned credit. Thus, there is a direct and interpretable correlation between the contribution level and the credit allocation for each agent and their interactions. Our method explicitly models $Q_{tot}$ as a sum of contributions from individual agents and their interaction terms through a continued fraction structure, which—together with the global state $\\boldsymbol{s}$ and assistive information—enables the learning of fine-grained credit assignment. As shown in Figure 8, increasing the interaction order in the model leads to better representation of cooperative behavior among agents, and we observe that higher-order interaction terms exhibit significantly increased contribution levels. This behavior aligns with the agents' observed coordination, demonstrating that our method provides a more accurate and interpretable credit assignment than those considering only low-order interactions.\n\n\n4. **Choosing the depth (interaction order).**\n\n   **Response:** Thank you for raising this insightful point. Our ablation in Section 5.2 shows that the performance degrades as the interaction order increase beyond a certain depth. One key observation is that when the order of interactions exceeds the number of agents, e.g., considering interactions beyond the $n$-th order for $n$ agents, it leads to impractical terms ($n+1$-order or higher-order interactions) that cannot be realized by the agents. These unnecessary higher-order terms may still be assigned weights during training, but they are meaningless and may overfit or propagate noise, potentially affecting the convergence speed. Generally, each additional fraction layer captures one higher order of agent interaction. The optimal depth should depend on the intrinsic cooperation complexity of the environment (e.g., pairwise coordination vs. five agent synchronization).  We have not yet derived a theoretical bound on the optimal depth, but we are investigating adaptive techniques to learn the appropriate depth automatically, akin to halting criteria in adaptive computation. We will discuss these ideas in the revised conclusion.\n\n5. **Explanation of Figure 9.**\n\n   **Response:** We appreciate your careful observation. Although the Q-values similarities between QMIX and QCoFr appear close in Figure 9, QCoFr actually produces more diverse behaviors compared to QMIX. From it, we can observe that in QMIX, the cosine similarity between the Q-values of other agents and the current agent’s Q-values ranges from 0.8 to 1, while in our method (QCoFr), the range is from 0.61 to 1. This indicates a significantly larger variance in the Q-values between agents in QCoFr, leading to more diverse behaviors. Furthermore, as shown in Figure 8, QCoFr exhibits more complex collaborative behaviors, which also reflect the increased diversity in its actions. At a given moment, agents may collaborate and take similar actions, which could not always be the case if the Q-values of all agents differ significantly. However, overall, QCoFr demonstrates more diverse behaviors compared to QMIX. We will clarify this interpretation in the revised manuscript.\n\n6. **Loss terms in Figure 2.**\n\n   **Response:** Thank you for your careful comment, and we apologize for the oversight in not providing a clear explanation of the losses. The CE Loss and KL Loss shown in Figure 2 are both part of the VIB Loss (Equation 11). Specifically, the Cross-Entropy (CE) Loss ($\\mathcal{L}\\_{CE}$) is defined as $\\mathcal{L}\\_{CE} = \\frac{1}{N} \\sum_{i=1}^N \\mathbb{E}\\_{\\epsilon \\sim p(\\epsilon)} \\left[-\\log q(u_i^* \\mid f(h_i, \\epsilon)) \\right]$, and the Kullback Leibler (KL) Loss ($\\mathcal{L}\\_{KL}$) is defined as $\\mathcal{L}\\_{KL} = \\mathrm{KL}\\left[p(\\boldsymbol{m} \\mid h_i), \\tilde{q}(\\boldsymbol{m})\\right]$. Thus, the overall VIB Loss can be expressed as $\\mathcal{L}\\_{VIB} = \\mathcal{L}\\_{CE} + \\beta \\cdot \\mathcal{L}\\_{KL}$. In the context of the VIB framework, the two losses serve different purposes:\n\n   A. The CE Loss encourages the model to make accurate predictions by minimizing the difference between the predicted and target distributions, ensuring the learned representations are relevant for task performance.\n\n   B. The KL Loss serves as a regularizer, which aims to encourage the model to make the posterior distribution of the latent variable $\\boldsymbol{m}$ close to the prior distribution $\\tilde{q}(\\boldsymbol{m})$. This controls the flow of information, preventing the network from retaining redundant or irrelevant information, thereby enforcing the information bottleneck.\n   We will add explicit equations and describe the combination strategy for clarity.\n   \n7. **Proofs in the appendix are not presented in a very intuitive manner.**\n\n   **Response:** We apologise for the difficulty in locating proofs.  We will revise the paper to insert cross references after each definition and theorem so that readers can quickly find the corresponding proof.\n\n8. **Minor Points.**\n\n   **Response:** \n\n   1) We agree that adding subfigure titles to Figure 9 will improve clarity and will revise accordingly.\n\n   2) We will refine the manuscript around CFN-D and CFN-C to make their differences explicit."}, {"review_id": "TOpdFznZHN", "reviewer": "Reviewer_fozS", "summary": "This paper presents a computational approach to modeling higher-order interactions in cooperative multi-agent settings. The paper formulates the fully cooperative multi-agent task as a decentralized partially observable Markov decision process. The goal is to learn an optimal joint policy that maximizes the joint value function. The authors employ value decomposition, which breaks down the global objective into local value functions while adhering to the individual-global-max (IGM) principle. These local Q-functions are combined to approximate the overall joint action-value function. The main contribution of the paper is the utilization of Continued Fraction Networks (CFNs) to capture higher-order interactions among the agents' Q-functions. The proposed model constructs the total Q-function by passing the individual Q-functions into a CFN to model higher-order interaction and then aggregating the outputs of the CFN using trained credit assignment functions. The paper's other contribution is the introduction of a Variational Information Bottleneck (VIB) instead of a variational autoencoder, as in previous literature. The VIB captures the most relevant information from history rather than using all of it to create the assistive information.", "questions": "A- According to our understanding, CFNs can suffer from instability with larger depths, this aligns with scalability issues of previous contributions. Can the authors expand on this point?\n\nB- Interpretability is understandable given the power series connection. In terms of assigning credits, is there another interpretable way instead of using ReLUs? \n\nC- What are possible considerations to move to fully decentralized approaches?", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths: \n\n1- The paper is well-written, but it requires some proofreading. The authors spent time comparing literature, providing figures, and writing detailed appendices. \n\n2- The proposed method exhibits reasonable performance across various benchmark tasks.\n\n3- The ablation studies are comprehensive, examining the effects of various models with different depths and structures, as well as studying the impact of the VIB module. \n\n4-  Earlier studies on modeling higher-order interactions have relied on modeling only linear interactions, employing black-box implementations, or considering intractable models. In this case, the output of a fractional neural network can be represented as a power series, reflecting the interpretability of such models in characterizing interactions. \n\nWeakness: \n\n1- Authors refer to variables that are not clearly defined, such as $R_t$ and $Q_{tot}$. Although the intent is clear, such quantities should be defined clearly in the problem statement. If the issue is space, the authors can perhaps remove information such as Definition 1 from the main body (which was already defined in the appendix). \n\n\n\n2- The structure of the paper follows a very similar structure to reference [21], which introduces NA$^2$Q. The intent in replacing the Neural Attention Additive model with CFN is clear in terms of tractability. Furthermore, this paper differs from [21] in the introduction of the VIB module, which extracts essential information from history. A few comments in this regard:\n\n\n\n\n\n\n\nA- The performance of the presented approach is close to the performance of  [21],  except on tasks 3s_vs_5z, 3s5z_vs_3s6z, and 6h_vs_8z. However, as seen in the ablation test, VIB plays a significant role in shaping the performance of the current model in these tasks. For tasks 3s_vs_5z and 6h_vs_8z, the performance of the presented approach without VIB is comparable to that of [21]. To fully demonstrate the effect of modeling higher-order interactions, it is recommended to observe how [21] would perform with the addition of VIB to truly reflect the impact of modeling higher-order interactions using CFNs.\n\n\n\n\n\n\n\nB- For the sake of completeness of comparison with [21], consider showing the performance on the SMAC benchmark for \"Corridor\" and \"8m_vs_9m\". \n\n\n3- Local fixes are needed, some examples below:\n\n- Line 45: \"but also the interpretability\" the sentence needs a verb such as provides\n\n\n\n- Line 73: \"For each coalition, it is formed. The usage of for is weird. Consider just saying, \",where each coalition is formed ...\"\n\n\n- Line 187, the sentence is weird.\n\n\n- Line 30: consider removing the word \"interactions.\"", "was_revised": true, "final_justification": "All my questions were answered as expected. I will keep my score as is.", "rebuttal": "Thank you very much for your positive review! Below, we respond to your critical points and questions.\n\n1. **Differences from NA$^2$Q.**\n\n   **Response:** Due to the limited space, please refer to the response to **Reviewer CAhf**.\n\n2. **The role of CFN.**\n\n   **Response:** To fully demonstrate the effect of modeling higher-order interactions, we conducted experiments with NA$^2$Q combined with the VIB module on three super-hard maps, with the results summarized in Table 1. It is clear that the performance of NA$^2$Q with the VIB module is improved compared to the original NA$^2$Q, but there remains a noticeable gap between it and our method. This indicates that, in more complex tasks, modeling higher-order interactions allows agents to learn refined cooperation strategies, thereby demonstrating the effectiveness of the CFN module.\n    \n    ### Table 1. Performance comparison of NA$^2$Q with the VIB module and our method.\n    | Method        |     corridor    |     3s5z_vs_3s6z  |      6h_vs_8z     |     \n    | :------------:| :---------------: | :---------------: | :---------------: | \n    | VDN           |   0.46$\\pm$0.36   |   0.64$\\pm$0.33   |   0.00$\\pm$0.00   |   \n    | QMIX          |   0.01$\\pm$0.01   |   0.61$\\pm$0.19   |   0.11$\\pm$0.19   |  \n    | NA$^2$Q       |   0.63$\\pm$0.20   |   0.73$\\pm$0.14   |   0.30$\\pm$0.24   | \n    |  NA$^2$Q_VIB  |   0.77$\\pm$0.06   |   0.73$\\pm$0.36   |   0.39$\\pm$0.34   |\n    | QCoFr (Ours)  |   **0.79$\\pm$0.03**   |   **0.87$\\pm$0.07**   |   **0.58$\\pm$0.12**   |  \n    \n\n\n4. **Performance on the SMAC benchmark for \"Corridor\" and \"8m_vs_9m\" compared with NA$^2$Q.**\n\n   **Response:** The performance comparison between our method and NA$^2$Q on the Corridor and 8m\\_vs\\_9m maps is shown in Table 2. In both maps, our method outperforms NA$^2$Q in terms of both convergence speed and win rate, highlighting the advantage of explicitly modeling higher-order interactions among agents.\n\n    ### Table 2. Performance comparison with NA$^2$Q on corridor and 8m_vs_9m.\n\n    | Method        |     8m_vs_9m    |     corridor  |       \n    | :------------:| :---------------: | :---------------: | \n    | VDN           |   0.80$\\pm$0.11   |   0.46$\\pm$0.36   |    \n    | QMIX          |   0.75$\\pm$0.11   |   0.01$\\pm$0.01   |   \n    | NA$^2$Q       |   0.83$\\pm$0.07   |   0.63$\\pm$0.20   |     \n    | QCoFr (Ours)  |   **0.93$\\pm$0.02**   |   **0.79$\\pm$0.03**   | \n\n\n5. **Stability with depths.**\n\n   **Response:** Please refer to the response to **Reviewer CAhf**.\n\n6. **ReLU in the credit assignment mechanism.**\n\n    **Response:** We appreciate your insightful question regarding interpretability in the context of credit assignment and activation choices. Credit assignment mechanism relies on monotonic linear layers with non-negative weights, which aggregate outputs from different levels of interaction modeled by the continued fraction network. Each layer in CFN captures a distinct interaction order—1-order (individual), 2-order (pairwise), and so on. Because CFN approximates a power series, the credit assignment weights can be interpreted analogously to Taylor coefficients, which indicate how much each coalition contributes to the team. Besides, the use of ReLU is not central to interpretability per se but serves as a computationally efficient monotonic nonlinearity. The work [R1] shows that replacing softmax with normalized ReLU in attention mechanisms reduces computation and improves stability. Furthermore, we acknowledge that other monotonic activation functions (e.g., Softplus, Sigmoid) could also be used in the credit assignment module without compromising interpretability. Softplus, in particular, offers a smooth approximation of ReLU and may facilitate optimization in some settings. We chose ReLU for its simplicity and empirical performance, but the framework is compatible with these alternatives.\n\n   [R1] Mitchell Wortsman, et al., Replacing softmax with relu in vision transformers. arXiv preprint arXiv:2309.08586, 2023.\n\n7. **Move toward fully decentralized approaches.**\n\n   **Response:** Thank you for the insightful comments. QCoFr follows the CTDE paradigm that agents have access to global information during training but rely on local policies during execution. It may be a challenge to simply move to fully decentralized learning, where each agent learns without a central critic. Without the centralized training, agents cannot directly optimize joint objectives. One possible solution is to employ decentralized policy gradient methods with local critics that exchange messages subject to a bandwidth constraint. The VIB could be extended to learn compact messages. Another possibility is to use implicit coordination via mutual information regularization, encouraging agents' policies to align with latent team variables.\n\n8. **Variable definitions and typos.**\n\n   **Response:** We appreciate the feedback regarding undefined variables.  In the next revision, we will: 1) Clearly define all symbols when first used. 2) Move Definition 1 and related mathematical preliminaries to the appendix to free space in the main text for clearer definitions of the main quantities. 3) We will also correct the grammatical issues identified around lines 30, 45, 73, and 187."}, {"review_id": "5ci5qJBk2I", "reviewer": "Reviewer_WW9h", "summary": "Th paper introduce a novel multi-agent Q learning algorithm, named QCoFr. QCoFr models arbitrary-order interactions among agents using a Continued Fraction Network (CFN), which scales linearly with the number of agents and enables interpretability via rational approximation theory. The model incorporates a Variational Information Bottleneck (VIB) to distill environment-relevant latent information to aid credit assignment. Empirical results on LBF, SMAC, and SMACv2 benchmarks demonstrate competitive performance and improved interpretability over strong baselines.", "questions": "1. Why the VIB inputs the hidden state instead of the global sate?\n2. Adaptive order could be explored.\n3. Experiment on other multi-agent benchmark with more agent numbers could be added to explore the relation with interaction order and agent number.", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strenghts: \n\n1. The algorithm's performance is excellent on SMAC baselines.\n2. The paper provides rigorous proofs around approximation guarantees, universal approximation, and information bottleneck bounds.\n3. The use of continued fractions to represent high-order agent interactions is original\n\nWeakness:\n1. The VIB is introduced to address the causal confusion from global state. However, the VIB inputs the hidden state instead of the global sate. \n2.  The CFN depth is fixed for per task. Adaptive mechanisms based on interaction complexity could be explored.\n3. The figure could be make clear to illustrate the computation process for Q.", "was_revised": true, "final_justification": "All my concern is addressed . So I tend to accept.", "rebuttal": "Thank you for reviewing our paper and your constructive feedback. We appreciate the time and effort you spent on our paper and the helpful comments you provided. Please find our itemized responses to your questions below.\n\n1. **The VIB take the hidden state as input.**\n\n   **Response:** Regarding the value decomposition in cooperative MARL, credit assignment serves an important role to guide the learning of decentralized policies, which aims to deduce the contributions of individual agents from the overall success. Since each agent observes the global state partially, there exists a potential backdoor path between the global state $\\boldsymbol{s}$ and $Q_{tot}$, specifically $\\boldsymbol{s} \\leftarrow E \\rightarrow \\tau \\rightarrow Q_{tot}$. This backdoor path can lead to a spurious correlation that causes confounding bias for estimating credit assignment. Inspired by the previous works [R1, R2], an effective solution is to impose an intervention function on $\\boldsymbol{s}$ identifying the local history $\\tau_i$ in an unobservable environment $E$. To realize it, we consider the information bottleneck method to produce assistive information $m$ aiding value factorization together global state $\\boldsymbol{s}$, which forms the Markov chain $\\boldsymbol{h}-\\boldsymbol{m}-\\boldsymbol{u^\\*}$ during encoding. To learn an encoding that is maximally informative about the optimal action selection $\\boldsymbol{u^\\*}$, we regard the internal representation of the intermediate layer as stochastic encoding $\\boldsymbol{m}$ of history $\\boldsymbol{h}$. Thus, we write the objective as $J_{I B}(\\boldsymbol{\\phi})=I\\left(\\boldsymbol{m}, \\boldsymbol{u}^\\* ; \\boldsymbol{\\phi}\\right)-\\beta I(\\boldsymbol{m}, \\boldsymbol{h} ; \\boldsymbol{\\phi})$.\n\n\n    [R1] Jiahui Li, et.al., Deconfounded value decomposition for multi-agent reinforcement learning, ICML, 2022.\n\n    [R2] Zichuan Liu, et.al. NA$^2$Q : Neural attention additive model for interpretable multi-agent Q-learning. ICML, 2023.\n\n\n2. **Adaptive order interactions.**\n   \n    **Response:** We appreciate your insightful suggestion regarding adaptive adjustment of CFN depth. As noted in our limitations section, this is indeed a promising avenue for future work. Theoretically, increasing CFN depth allows the model to capture higher-order interactions among agents, improving expressiveness in tasks requiring coordination skills. However, unnecessarily modeling high-order interactions in simpler environments or with fewer agents may lead to overfitting or optimization instability.  To balance expressiveness with efficiency, one potential solution may be valuable for future exploration. Considering using adaptive computation, the CFN can be initialized with a base depth $d_i$ and iteratively explore neighboring depths within a radius $\\delta_{d}$, halting exploration once marginal gains fall below a threshold. Another potential method is to integrate skip connections with learnable gating mechanisms, allowing the network to modulate depth in a differentiable and task-adaptive manner. While our current fixed-depth configuration already delivers strong empirical results, we agree that incorporating such adaptive depth control based on coordination complexity is a valuable future direction, and we will add this discussion to the revised manuscript.\n   \n3. **The computation process for $Q_{tot}$.**\n\n     **Response:** We thank the reviewer for pointing out the lack of clarity in the current figure illustrating the computation of the $Q_{tot}$. While the accompanying text describes the pipeline in detail, we acknowledge that the diagram could better reflect key components such as the recursive structure of the continued fraction network and the role of the auxiliary variable from the VIB module. In the new revision, we will update the figure to more explicitly highlight the sequential fraction aggregation process, clarify the data flow from the individual $Q_i$ values through CFN layers, and provide a clear explanation of the losses.  We appreciate your feedback again and are committed to improving the clarity of our presentation.\n\n4. **Experiment on other multi-agent benchmark.**\n   \n    **Response:**  Thank you for the valuable suggestion. To examine the relationship between interaction order and agent number, we conducted experiments on the **Multi-Agent COordination Challenge (MACO)** benchmark [R3], which comprises diverse multi-agent tasks with varying coordination demands and larger agent populations. We evaluated our method on three representative scenarios: **Pursuit** (12 agents), **Aloha** (15 agents), and **Gather** (10 agents). The results are summarized in Table 1, where $\\pm$ denotes the standard deviation of the average performance over 5 trials, and the best performance in each scenario is highlighted in **bold**.\n\n   As summarized in **Table 1**, our method (QCoFr) significantly outperforms baselines such as VDN, QMIX, and NA$^2$Q with 2-order across all tasks, which should benefit from explicit modeling of higher-order interactions. As the orders increase, QCoFr (d=5) achieves better performance, which confirms that higher-order interactions are critical for coalitions with more agents to learn high-level cooperation skills. These results reinforce the theoretical motivation of our work: that increasing CFN depth improves performance in proportion to the coordination complexity and number of agents. We will include this discussion and full results in the revised manuscript.\n\n    ### Table 1. Performance comparison with baselines on the MACO benchmark.\n   | Method        |     pursuit       |       gather      |      aloha        |     \n   | :------------:| :---------------: | :---------------: | :---------------: | \n   |               |  Test Prey Caught |   Test Win Rate   |  Test Transmitted |\n   | VDN           |   0.00$\\pm$0.00   |   0.22$\\pm$0.01   |   0.00$\\pm$0.00   |   \n   | QMIX          |   0.00$\\pm$0.00   |   0.78$\\pm$0.02   |   0.75$\\pm$0.50   |  \n   | NA$^2$Q       |   3.00$\\pm$0.77   |   0.58$\\pm$0.26   |   21.08$\\pm$12.00  | \n   | QCoFr (d=3)  |     3.23$\\pm$0.26  |    0.79$\\pm$0.05  |   24.37$\\pm$5.34   |\n   | QCoFr (d=5)  |   **3.75$\\pm$0.15**   |   **0.90$\\pm$0.01**   |   **28.50$\\pm$7.13**   |  \n\n     \n    [R3] Tonghan Wang, et al., Context-Aware Sparse Deep Coordination Graphs. ICLR,2022."}]

Author rebuttal:

Meta-review: This paper proposes a new value decomposition architecture for MARL capturing higher order interaction among agents based on continued fractional neural networks. The new method achieves noticeable performance gain in difficult SMAC tasks such as 3s5z_vs_3s6z, showing such hard tasks indeed require higher order interaction modeling. In the research line of value decomposition under CTDE for multiagent reinforcement learning, it seems that the proposed method makes a relevant position. 

This work is basically based on the previous work of Puri et al.'s CoFrNets.  Clearly, cite this in Section 3.  Section 3 should be rewritten properly, especially, lines 93 - 112. Some setences are directly from Puri et al's paper without citation. It seems that replacing $a_k = w_k^T x$ in CF and builing a neural net architecture based on CF is due to Puri et al. This paper does not mention this at all. Line 95 citation [23] for CFN seems not correct. It should be [18].  [23] is a graduate math textbook. It is highly unlikely that it contains neural networks.

Considering above, I recommends conditional acceptance on the condition that the background Section 3 is clearly rewritten. Otherwise, acceptance can be revoked.
Decision: Accept (poster)