Adaptive Context Length Optimization with Low-Frequency Truncation for Multi-Agent Reinforcement Learning
Authors: Wenchang Duan
Yaoliang Yu
Jiwan He
Yi Shi
Venue: NeurIPS 2025 poster ()
Keywords: multi-agent reinforcement learning
adaptive context length
low-frequency truncation
long-term dependencies
Primary area: reinforcement_learning
TL;DR: Large fixed context length limits exploration and introduces redundancy in MARL. We propose an adaptive context length optimization method with Fourier-based low-frequency truncation to improve long-term decision-making.

Abstract: Recently, deep multi-agent reinforcement learning (MARL) has demonstrated promising performance for solving challenging tasks, such as long-term dependencies and non-Markovian environments. Its success is partly attributed to conditioning policies on large fixed context length. However, such large fixed context lengths may lead to limited exploration efficiency and redundant information. In this paper, we propose a novel MARL framework to obtain adaptive and effective contextual information. Specifically, we design a central agent that dynamically optimizes context length via temporal gradient analysis, enhancing exploration to facilitate convergence to global optima in MARL. Furthermore, to enhance the adaptive optimization capability of the context length, we present an efficient input representation for the central agent, which effectively filters redundant information. By leveraging a Fourier-based low-frequency truncation method, we extract global temporal trends across decentralized agents, providing an effective and efficient representation of the MARL environment. Extensive experiments demonstrate that the proposed method achieves state-of-the-art (SOTA) performance on long-term dependency tasks, including PettingZoo, MiniGrid, Google Research Football (GRF), and  StarCraft Multi-Agent Challenge v2 (SMACv2).

Adaptive Context Length Optimization with Low-Frequency Truncation for Multi-Agent Reinforcement Learning
Wenchang Duan', Yaoliang He', Yi Shi
1 Shanghai Jiao Tong University ²University Waterloo (duanwenchang, kih020429, yishi/aajuu edu. yaoliang. youaungguuuuuattroo.ooo
Abstract
Recently, deep multi-agent reinforcement learning (MARL) has demonstrated promising performance for solving challenging tasks, such as long-term dependen- cies and non-Markovian environments. Its success is partly attributed condi- tioning policies on large fixed context length. However, such large fixed context lengths may lead to limited exploration efficiency and redundant information. In this paper, we propose novel MARL framework to obtain adaptive and effective contextual information. Specifically, we design a central agent that dynamically optimizes context length via temporal gradient analysis, enhancing exploration facilitate convergence to global optima in MARL. Furthermore, to enhance the adaptive optimization capability of the context length, we present an efficient input representation for the central agent, which effectively filters redundant information. By leveraging Fourier-based low-frequency truncation method, extract global temporal trends across decentralized agents, providing an effective and efficient representation of the MARL environment. Extensive experiments demonstrate that the proposed method achieves state-of of-theer-t (SOTA) performance on long-term dependency tasks, including PettingZoo, MiniGrid, Google Research Football (GRF), and StarCraft Multi-Agent Challenge v2 (SMACv2).
1 Introduction
Multi-agent reinforcement learning (MARL) has drawn increasing interest in recent years, which provides promise for facing many complex real-world challenging problems such transportation management [ robot control [2], and finance [3). However, due to the long-term dependencies and non-rigorous Markovianity of complex tasks, contextual information is introduced to assist policy making [ [5] 6]. Accordingly, this places substantial demand on how to leverage contextual information and to what extent α
Existing methods are mostly applied to single-agent reinforcement learning (RL), where contextual information performs reasonably well in simple tasks (
10 П.. In comparison, multi-agent reinforcement learning (MARL) involves significantly more complex tasks [12 [3]. where relying solely on short context lengths individual observations often results in suboptimal performance. To address this, one natural approach is to extend the context length. However, the expansion of the context length leads two significant challenges: the first is an increase in necessary computation. and the second is the difficulty of high dimensionality of the input representation and generalization
"Corresponding author: Yi Shi (yishiesjtu..eee edu. cn). Project code available at: https: http//hhhhuocococuuueeeeeeaaaa/a
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

To address the challenge of increasing computation, references 9 [5] optimized the needed context length and the utilization efficiency; references [16 I7 adopted parallel computation; and reference [18] enhanced the performance of modern hardware. However, the above methods involve a long-time pre-training process, and eventually only obtain static context length. These static context lengths are difficult to adapt to changing environments, which potentially leads to suboptimal solutions or inefficient use of computational resources.
The challenge of input representation and generalization remains unresolved in the field of multi-agent reinforcement learning (MARL) [14. While recent improvements in processing long sequences that came with attention models significantly alleviate requirements for generalization, an effective representation is still crucial for obtaining optimal contextual information [19] 20 [2]] Regarding the fields where the contextual information is also significant, natural language processing (NLP) typically involves leveraging large language models (L.LMs) to autonomously learn and generate Therefore, a tailored input
prompts, which does not align well with the principles of MARL 22 23 representation for MARL is required.
According to the aforementioned analysis, we propose an adaptive context length optimization with low-frequency truncation (ACL-LFT) for MARL. Specifically, senior central agent introduced to adaptively optimize context length, and tailored attention-based reward is designed to align with the central agent. Via real-time interacting with environment, the central agent determines the optimal context length to address the challenge of increasing computation. Besides, we apply the Fourier transform map the data from the time domain to the frequency domain, facilitating more effective redundancy filtering compared to direct processing in the time domain. Via truncating the oow-freuuency band, we obtain an effective input representation for the central agent, which captures the global temporal trends from the decentralized agents. With the above designs, our method effectively solves the dual challenges of increasing context length, achieving efficient leverage the contextual information.
We benchmark the proposed method across various environments including Sample Spread in PettingZoo [4]; MiniGrid Soccer Game in OpenAl Gym [25;; Academy 3 vs with Keeper, and Academy Counterattack-Hatt in Google Research Football (GRF) [26]: 3s52 vs 3s6z, 5m vs 6m, and corridor in StarCraft Multi-Agent Challenge v2 (SMACv2) environments (27]. Combined with several types of experiments, including state-of-the-ar (SOTA) sequence processing algorithms and different fixed-length methods, we show that the proposed method significantly enhances the performance of the baseline algorithm in changing environments. The main contributions this paper are summarized as follows:
To the best of our knowledge, ACL-LFT is the first framework systematically address the dual challenges of increasing context length in MARL. Equipped with the central agent, our framework achieves adaptive and efficient leverage of contextual information enhance the decision- decision-making of decentralized agents. Additionally, we present a theorem to theoretically demonstrate the superior performance of adaptive context length over static ones dynamic environments.
We propose a novel Fourier-basee low-frequency truncation to obtain the global tempo- ral trends from context, effectively addressing the challenge of representing the MARL environment and providing an efficient input for the central agent.
We empirically demonstrate that the proposed method outperforms SOTA sequence pro- cessing algorithms across various long-term dependency environments. We also provide experimental results to demonstrate the superior performance of the proposed method over different fixed lengths in dynamic environments.
2 Preliminaries
2.1 Decentralized Partially Observable Markov Decision Process with Historical Information
The Decentralized Partially Observable Markov Decision Process with historical information is defined as a tuple M = (N, A, R, 7), where N the set agents, S denotes the global state space, and A represents the joint action space. At time I, the environment evolves according to the transition function P(*{ (8e (z), which specifies the probability of reaching the next state s given the
). s S and the joint action a, {at af.................. ... a The current global state St {s} s6, ... "","),

---

environment then produces a global reward r. = R(s;,0)) In this decentralized setting, each agent follows local policy that seeks to maximize the expected cumulative discounted reward. given by
((x) = Σ== Y R(s,, ae) [ where y [0, 1) is discount factor that balances the importance of immediate versus future rewards. The classical Markov property assumes that state transitions depend only on the current state and action.
However, in decentralized partially observable environments, agents cannot directly access the full global state s;; instead, they rely local observations that are incomplete and noisy. In many scenarios, the assumption that Docision-makinng can be based solely on the current observation is insufficient. To address this, the framework incorporates **historical information** to approximate the underlying dynamics and capture long-term dependencies. Formally, the extended model can be written as M = (N, S, A, R, 7), where S includes not only the current state S but also contextual information space S representing observation histories, and A A. At time t, the transition expressed as
P ū.) P( = U 13, = =8 U.U. , =at),
and the reward given by
R R(
at).
Unlike the standard Markov Decision Process, this extended Decentralized Partially Observable frame- work enables modeling of complex dynamic environments with long-term temporal dependencies, where leveraging historical information is essential for effective coordination among agents.
2.2 The Fourier Transform and Litllewood-Paley Theory
The Fourier transform provides fundamental tool for analyzing functions in the frequency domain by decomposing signals into their constituent frequency components. Formally, for function L' (I²), the Fourier transform is defined as:
= f(E) = e e-i(dl f(x) dz,
ggd
where (x:(E denotes the inner product in Rd. As a continuous linear map from ('R into L ~ (Rd)), it satisfies If(E) < |lll, ensuring boundedness in the transformed domain. Besides, for any function yp E L and an automorphism L on Rdd the transformation obeys:
1
Fle L) det L
o L
(2)
By mapping state representations from the time domain to the frequency domain, the Fourier transform captures underlying structural patterns, where wwwfrrquency components effectively encode global
trends while filtering high-freag frequencunn noise (8 29
Littlewood-Pad theory provides a decomposition that functions distributions are easier to deal with split into countable sums of smooth functions whose Fourier transforms are compactly supported in ball or annulus [30]. In the complex non-Markovianity environments, such decomposition renders a localization procedure in frequency space, which the derivatives act almost as homotheties on distributions. This property establishes fundamental bounds on the behavior of derivatives in different L" spaces, leading to the following Bernstein incqualities. Let be an annulus and B ball. There exists constant C' such that for any nonnegative integer any pair (p, 9) E [1, ] with q 2 2 1, and any function u E IP, the following holds:
Supp AC
|| u
der
110 S O+++1 b+d( WullIIz lal=k
sup
(3)
Supp c xC C- < [D S &CK+1 C
(4)
The above inequalities highlight key property: if function's Fourier spectrum is restricted within frequency 5, its a-th order derivative amplifies high-frequency components by factor of glal, This property enables an efficient truncation now-frquuncy information, which serves as an effective representation of contextual information while preserving stability in the decision-maken process.
3

---

3 Methodology
In this section, we propose novel MARL framework for obtaining adaptive and effective contextual information, which systematically tackles the dual challenges of increasing context length in MARL. The overall framework shown in Fig. | which is comprised of three main components: (1) the Fourier-based low-freque owwfrrquencyy truncation module, (2) a central agent of adaptive select contextual information, and (3) the structure learning with spatio-temporal decoupling.
Low-Frequency T Truncation
E Environment
Discrete Foutee Transform
Discrete Dyadi: Partitio of Unity
St
",
obs,
(sf ke.)
Central A geent
Mulli-Head A ttetttin aq.aef...
opc
A daptive Context L Length Optimization
Figure I: Schematics of our ACL-LFT. At each time i, the historical state 8, is first processed via the Fourier-based now-frequency truncation module. The central agent leverages the truncated information of as input and then adaptively optimizes the context length. Subsequently, the decentralized agents then integrate the optimized contextual information 8, opt with the current state to achieve decision- making.
3.1 Fourier-based Low-Frequency Truncation
To achieve an efficient representation of the MARL environment and enhance its effectivenness as input for the central agent, we introduce a low-frequency truncation method. By filtering out high- frequency fluctuations while preserving low-frequency parts, this method captures global temporal trends across decentralized agents and provides a more stable basis for downstream Decision-making Specifically, given the discrete nature of historical state data 8; { we first leverage the Discrete Fourier Transform (DFT) to convert time domain data frequency domain data:
S[A]] Σ
k= 0,
1,
(5)
a=0
where S[k] represents the rreueencydomaad coefficient corresponding to frequency index k, while each coefficient encodes particular oscillatory component of the historical sequence. For real-valued signals, the DFT exhibits conjugate symmetry: S[k] = Sit k|", reflecting periodicity in the frequency domain. This transformation effectively disentangles different frequency components of the input sequence, allowing for more interpretable and structured representation of historical states.
Building upon the LitllewoddPPale theory, we then introduce the Dyadie Partition of Unity method
and extend it to the discrete space to truncate the low-frequency information 30) The Dyadic
Partition of Unity method for measurable functions is provided in Appendix [..] We extend this
method to adapt discrete frequency domain historical states. Specifically, we aim for the sum of the window functions approximate unity across the entire frequency domain:
J-1-m m
X
Σ
1
Vk =k=00 = 0,
N 1 1,
(6)
where [k] a low-pass window function that retains only the low-frequency components. 4 represents band-pan window functions that separate different frequency bands in dyadic manner. the maximum decomposition level, and for simplicity, we assume that = 2 m a tunable parameter that determines the truncation frequency, ensuring that 2 1/2.
Within this method, the low-frequency region defined for S 2m k 2 2mh where we set X [k] = 1 and ensure that & [k] 0 for all j, thereby preserving the low- www-frequency information se-

---

In the band-pass pass regions, corresponding to frequency indices satisfying 2 I k < 2/+m+1 (or their symmetric counterparts). single window function $, [k] activated with a value of 1, while all other Φ3 [k] remain zero for jr & ensuring a well-defined partitioning of frequency bands. At transition points, such as k = 2 minor gaps may arise due to non-overlapping support. But as the signal length increases, the gaps become negligible with an error proportionally decreasing as O(1/t), thereby maintaining a stable approximation of equation 6
The details of Dyadie Partition of Unity in Discrete Form and its rigorous proof are provided in Appendix A.2 By leveraging low-frequency truncation, the above method effectively captures the global temporal trends across decentralized agents, reducing the redundancy of contextual information and serving as an efficient input representation for the subsequent central agent.
3.2 The Central Agent of Adaptive Contextual Information Selection
The central agent in our framework serves global information processor, adaptively determining the optimal contextual information length for decentralized agents. is designed to process and analyze only historical information, without directly handling the current state. Specifically, its decision-makin process is structured around three key components: state representation, action space, and reward formulation.
Firstly, the state of the central agent is derived through the Nourier-based low-frequency truncation module, which is elaborated in section 3.1 For the discrete historical states s { time t, this module extracts the truncated representation sf, which effectively represents the global temporal trends across decentralized agents.
Then, the action space the central agent A, is defined as the selection of different low- frequeeyy truncation levels, given by:
of E A, = (m1,,2,-- m2,..,,,.... m
where M represents the dimension of A.s and each action my corresponds to a different range of preserved low-frequency bands, with each band representing temporal trends with varying degrees of long-term dependency. Given the selected truncation level mi the corresponding optimal contextual information s, sp obtained by truncation domain.
Finally, to guide its adaptation process, the reward of the central agent is tailored via the multi-head attention mechanism, which weights the influence of decentralized agents. Specifically, the value function estimates and the policy distributions of decentralized agents serve as keys, while the value function estimate and the policy distributions of the central agent serve as the query. We denote the concatenated representation of the value estimate and the policy distribution of agent as F,, and that of the central agent as F.. For each attention head g. Fi and Fe are projected into the query and key spaces via transformation matrices W and W
e W8
(8)
The attention weight assigned to each decentralized agent is then computed as:
(K-()?))
exp
Vdy
w/
(9)
-2-(K?77
Σ, exp
Vdu
where de represents the dimensionality of the key matrix K., ensuring numerical stability. The final attention weight for each agent at time t is obtained by averaging across all heads wf = head Eg=1 wf. At time t, for the weights {wi) the reward for the central agent is then derived as a weighted aggregation of the rewards of decentralized agents ri:
(10)
where E w 1.
Combined with these three components, the parameters of the central agent are updated using gradient- based optimization with advantage estimation. The value function V V(s{) is trained to approximate
5

---

the expected return through the temporal difference error:
8f - + > -(sf),
(11)
where 1 denotes the next state. The policy parameters 0 are then adjusted by maximizing the advantage- objective:
0 0 (Ve =
(12)
while simultaneously minimizing the value function error || through gradient descent.
Building upon the above design, the central agent achieves adaptive optimization of the context length, ensuring that decentralized agents receive the optimal contextual information 8, s-
Furthermore, to theoretically establish long-term advantage lower bound of the proposed method fixed-length methods, present Theorem
Theorem (Long-Terma Advantage Lower Bound of Adaptive Length) At time let Lddup be the adaptive context length, Las be a fixed context length, and the mutual information loss of I, be denoted as L.(..) Under mild ussumptions the expected cumulative reward difference between adaptive and fixed context length satisfies the following regret bound:
Σ (Ce(Lpp)) Lo(Ladd))) 2 Q(T) - -(TT)
(13)
= =Q(T) (when T is sufficiently large)
where O S < 1, with a being a non-deterministic parameter whose formal definition is provided Appendix
parumett
This theorem demonstrates the long-term advantage of adaptive length policies with the increasingly unstable environment. The result suggests that adaptively adjusting the context length enables more effective information retention time, leading significantly lower regret accumulation. The details and proof are provided in Appendix
3.3 Structure of Learning with Spatio-Tempon Decoupling
In this section, we discuss how to leverage the spatio- temporl decoupling to train the proposed learning framework. Specifically, the training process is divided into two components: the central agent, which is responsible for selecting the optimal contextual information s, -p and the decen- tralized agents, which leverage this information and their current state & to optimize their policies. In this framework, the central agent is trained independently to optimize the temporal information component, while the decentralized agents undergo joint training refine their policies with filtered temporal information and their spatial information. The policy-making and training process of the ACL-LFT algorithm illustrated
pseudocode provided Appendix C.3
The global optimization objective of the framework is given by the expected sum of discounted rewards over time for decentralized agents:
J((T)= E
Σ 7 R. (5t, a.)
(14)
The central objective is:
Je(
E(R mm]
(15)
0=0
By structuring training in this manner, our framework mitigates the challenge of the excessively large parameter search space that typically arises from the joint optimization of both contextual and current information, a factor known to hinder convergence. As result, our framework not only accelerates the learning process but also ensures that agents can efficiently leverage temporal trends, thereby improving decision decision-making in complex multi-agent environments.

---

Experiments and Analysis
This section evaluates the proposed ACL-LFT framework across various MARL environments. Section 4.1 outlines the experimental setup and baselines. Section 4.2 and Section 4.3 compare
ACL-LFT with sequence processing and fixed-length methods. Section 4.4 and Section 4.5 present ablation and case analyses. Finally, Section examines the decentralized setting without cross-agent information sharing.
4.1 Experiment Setup
Environments We consider various tasks, including Sample Spread in PettingZoo (24). MiniGrid Soccer Game in OpenAl Gym [25]. Academy vs with Keeper, and Academy Counterattack-
in Google Research Football (GRF) [26]. The overview of environments is shown in Fig. 2 while the details of environments and their reward design are provided in Appendix C. All experiments implemented based the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm [I] Furthermore, to verify the effectiveness of our proposed method under both complex and large-scale conduct additional experiments based on MAPPO, QMIX [32), and QPLEX (33 in the StarCraft Multi-Agent Challenge v2 (SMACv2) environments [27]. including 3s5z_ 3352_vy_33000 vy_3x67, 5m_vs mm__s_6me 6m, and corridor. The detailed
scenarios, as well to analyze the impact of removing the MAPPO backbone,
experimental results are presented in Appendix C.4
(a) Sample Spread
(b) Minigrid Saccer Game (c) 3 vs 1 with Keeper (d) Counterattack-Hin -Had
Figure 2: Sample Spread (a) a search game where agents learn to cover all the landmarks while avoiding collisions. Minigrid Soccer Game (b) is 15x15 environment where agents (triangles) rewards by kicking the ball (circle) into same-colored goalmouths (squares). Academy 3 vs I with Keeper (c) is scenario where three offensive agents attempt score against defender and goalkeeper. Academy Counterattack-Hane (d) is scenario where four agents must execute a rapid counterattack while avoiding defenders.
Given the varying maximum episode lengths across different environments, adapt the context length accordingly for each case. Specifically, the maximum episode steps for Sample Spread, MiniGrid Soccer Game, Academy vs 1 with Keeper, and Academy Counterattack- Hard are 25, 512, 400, and 400, respectively. Therefore, the corresponding context lengths are set to 4, 64, 64, and 64 steps. These values define the maximum selectable context lengths for the proposed method, consistent with the central agent's input dimension. Further details are provided in Appendix
Baselines We first benchmark the sequence processing algorithms, including Transformer (34), Token Statistics Transformer (ToST) [35), and AMAGO [36). The introduction of these methods is provided in Appendix C Then, we benchmark the proposed method against different fixed context lengths.
4.2 Performance Comparison with Sequence Processing Methods
In this section, benchmark the proposed method against Transformer, TOST, and AMAGO. As shown in Fig. 3 the performances are depicted via data from every 100 episodes and averaged over seeds. seen that the proposed method outperforms all scenarios. Specifically, Transformer and ToST need more time to explore sophisticated policies and demonstrate large oscillations in exploration process. AMAGO demonstrates strong performance during the policy exploration phase, owing to its effective handling of long sequences in parallel. However, due to the fact that its contextual information fixed length, which tends to have lot of noise, performs poorly compared to the proposed method after convergence.
7

---

Tati Ecloate Nhwcoet 16
500
500
AMAGO
TooT
Total Training Epicodes (x10)
3ota Traning Eiplcodes (x10))
Tosa Training Eploodes (x100
Total Training Eploodes (x100)
(a) Sample Spread
(b) Minigrid Soccer Game
(c) I with Keeper
(d) Counterattack-Hon
Figure 3: Performance Comparison with Sequence Processing Methods in Four Environments
In the Sample Spread, Academy 3 vs 1 with Keeper and Academy Counterattack Hard, the proposed method demonstrates the fastest exploration efficiency and consistently achieves the highest post- convergence performance. Notably, as the complexity of the scenarios increases from Sample Spread Academy 3 vs with Keeper and Academy Counterattack Hard the performance gap between the proposed method and the baseline methods becomes more evident. In the Minigrid Soccer Game, the proposed method and AMAGO exhibit comparable exploration efficiency; however, AMAGO converges prematurely and fails to achieve strong final results. In contrast, the proposed method utilizes oow-frequency truncation of historical information, significantly mitigating the impact of redundant data. By adaptively selecting the optimal context length, the proposed method achieves superior performance across all environments.
4.3 Performance Comparison with Fixed-Length
In section 3.2 we presented the Theorem D which demon-
100 908
strates the long-term advantage of adaptive length policies over fixed-length. In this section, we benchmark the pro- posed method against different fixed context lengths (8, 16, 32, and 64 steps) in Academy 3 vs with Keeper and Academy Counterattac--Hard.
80
6:21
860
250
840
30
We computed the average performance of the proposed method and four fixed-length methods during two rela- tively stable periods after convergence. Specifically, the results were averaged from the 0.9 millionth to the mil- lionth episode the Academy 3 vs with Keeper, and from the 1.4 millionth to the 1.5 millionth episode in the Academy Counterattack -ard.. As shown Fig. the proposed method significantly outperforms all fixed-length methods, further demonstrating the effectiveness and ef- ficiency of adaptive context length optimization. Notably, among the fixed-length methods, the 16-step and 8-step show the best performance in Academy vs with Keeper and Academy Counterattack- Hard, respectively. These findings suggest that longer context lengths do neces- sarily lead to better performance, as excessive historical information may introduce significant noise. This observation further underscores the critical role of our oww-frequnncy truncation in enhancing overall performance.
10
1 with Keeper
100
80
70
60.11
660
51.00
40
10
(b) Countcrattack
Figure 4: Performance Comparison with
Different Fixed Lengths on GRF
4.4 Ablation Experiments
To understand the contribution each component in the proposed ACL-LFT framework, we carry out ablation studies test the contribution of adaptive context length (ACL) and low-frequency truncation (LFT). Specifically, we utilize the best-performing fixed-length configurations in the Academy with Keeper (32-step) and Academy Counterattack-HH Hard (16-step) environments. These configurations are applied to evaluate ACL-LFT-NO-ACI and ACL-LFT-Raw, which test the performance the methods without ACL and without both ACL and LFT, respectively. Furthermore, the ACL-LFT-NO-LFFT is input with 32 and 32 steps, which align with the maximum step that can be selected by the ACL-LFT.

---

As shown in Fig. 5 the most impact performance is the ablation of ACI, which further demonstrates the sig-
90
nificant effect of adaptive context length optimizing. Ad- ditionally, the results reveal that ACI-L-FT-OO-CL. out-
performs ACL-LFT-Ram indicating that low-frequency truncation (LFT) contributes notably to the overall per- formance of the proposed framework. In conclusion, the results highlight the critical importance of both ACL and LFT in enhancing the efficiency and effectiveness of the overall framework. Moreover, they demonstrate these two components complement each other, contributing synergistically to performance improvement, especially in environments with varying complexities and temporal dynamics.
60
40
COLLLF-OOLFT
10
0.0
0.2
04
0.8
0.8
1.0
Total Training Episodes (x10")
80
gt Win
4.5
Case Study
50
To intuitively demonstrate the performance benefits brought by the adaptive context length mechanism, conduct a case study on the MiniGrid Soccer Game. This environment features a tailored reward function. which is highly sensitive to the agent's ability to leverage historical information for effective path planning and co- operative strategies. Specifically, we set the maximum step of environment 64. Accordingly, all baseline methods use fixed context length of 16, which aligns proposed
AGU-LFT
20
AOL-LFTNN (0L
1.2
1.5
Total Training Episodes (x10)
Figure 5: Comparison of ablation studies:
(a) vs with Keeper; (b) Counterattack- Hard.
with the maximum selectable length method.
As shown Table we record the reward value every steps and highlight the time step at which the first goal is achieved bold. The numbers in paren- theses indicate the context length adaptively selected by ACL-LFT at each time step. The results show that ACL-LFT quickly justs its context length after ob- taining positive rewards (c.g., step 15, 20), enabling timely re-planning to avoid inefficient exploration, while other meth-
Table 1: ktep-Reward Comparison on MiniGrid Soccer Game
Step
ACL-LFT
Transformer
ToST
AMAGO
0.00 (0)
0.00
0.00
0.00
5
2.71 (8)
2.96
0.00
0,00
10
1.52 (16)
1.73
2.17
2.63
15
1.77 (8)
1.94
1.59
2.08
20
2.41 (4)
1.65
1.26
1.54
25
3.19 (4)
2.36
1.85
1.93
30
3,85 (2)
2.07
2.26
2.45
35
3.30 (8)
2.85
2.51
2.16
40
4.06 (2)
3.57
2.90
2.48
41
14.31 (1)
3,45
2.97
2.65
ods remain the aimless ex- ploration phase. Notably, ACL-
45
4.12
3.36
3,03
47
13.98
3.52
3,41
LFT dynamically selects shorter yet effective context lengths (e.g..
50
3.79
3.66
55
4.09
3.85
length 2 at step 40), significantly improving path efficiency and ul- timately achieving goal by step 41. In contrast, methods with fixed-length contexts adapt more slowly, require longer to identify viable paths. This indicates that ACL-LFT enhances exploration efficiency and mitigates the im- pact of redundant information via adaptive context length optimization, thereby achieving superior performance.
56
/
14.25
4.02
59
13.62
4.6 Ablation on the Absence Cross-Agent Historical Information
To further examine the influence of centralized sequence processing and verify that our method does not rely on cross-agent information sharing, we conduct an additional ablation study which each agent only access its own local historical observations and actions, without any global or inter-

---

agent historical information. In this variant, the central agent is disabled from aggregating histories across agents; instead, it independently processes the local sequences for each agent, ensuring that no centralized communication channel exists during decision-making.
Table 2: Performance comparison without cross-agent historical information.
Task
AMAGO
Mamba
ACL.-LFT
3s5z vs 3s6z
76.1 + 2.9
72.6 + 3.2
78.9 + 2.8
5m vs_6m
48.1 + 4.0
46.2 + 4.5
52.7 + 4.2
corridor
74.3 + 4.8
69.0 5.9
77.9 + 5.3
As shown in Table 2 ACL-LFT consistently outperforms existing temporal modeling methods such as AMAGO and Mamba even when cross-agent historical information is available. This confirms that the performance improvement of ACL-LFT originates from proposed low-frequency temporal representation and adaptive contextual- optimization rather than from any implicit inter-agent information sharing. Moreover, this ablation validates that ACL--LFT preserves the partially observ- able nature of the SMACv2 environment and remains effective under purely decentralized historical settings, demonstrating the soundness and generality of our framework.
5
Conclusion
To systematically address the dual challenges of increasing context length in MARL, we propose an adaptive context length optimization with low-frequency truncation (ACL-LFT) for MARL. The proposed method adaptively optimizes context length via a central agent. Equipped with a Fourier- based wwwfrreueecyy truncation, we address the challenge of representing the MARL environment and provide an efficient input for the central agent. The experimental results demonstrate the proposed method significantly enhances the performance of the baseline algorithm in changing environments. In addition, we demonstrate both theoretically and experimentally the long-term advantage of adaptive context length over fixed-length.
References
[1] Yu Han, Meng Wang, and Ludovic Leclercq. Leveraging reinforcement learning for dynamic traffic control: A survey and challenges for field implementation. Communications in Trans- portation Research, 3: 1:0004 2023.
[2] Chen Tang, Ben Abbatematteo, Jiaheng Hu, Rohan Chandra, Roberto Martin-Martín, and Peter Stone. Deep reinforcement learning for robotics: A survey of real-world successes. Annual Review of Control, Roboties, and Autonomous Systems, 2025.
[3] Noella Nazareth and Yeruva Venkata Ramana Reddy. Financial applications of machine learning: A literature review. Expert Systems with Applications, 219:1 19640, 2023.
[4] Vinzenz Thoma, Barna Pásztor, Andreas Krause, Giorgia Ramponi, and Yifan Hu. Contextual bilevel reinforcement learning for incentive alignment. In Advances in Neural Information Processing Systems, 2024.
[5] Guy Tennenholtz, Nadav Merlis, Lior Shani, Martin Mladenov, and Craig Boutilier. Rein- forcement learning with history dependent dynamic contexts. In International Conference on Machine Learning, pages 34011-34053 PMLR, 2023.
[6] Andrew Wang, Andrew C Li, Toryn Q Klassen, Rodrigo Toro Icarte, and Sheila A Mcllraith. Learning belief representations for partially observable deep rl. In International Conference on Machine Learning, pages 35970-35998 PMLR, 2023.
[7] Annie Wong, Thomas Bäck, Anna Kononova, and Aske Plaat. Deep multiagent reinforcement learning: Challenges and directions. Artificial Intelligence Review, 5666):5023-55566 2023.
10

---

[8] Ashish Kumar Shakya, Gopinatha Pillai, and Sohom Chakrabarty. Reinforcement learning algorithms: A brief survey. Expert Systems with Applications, 231:120495, 2023.
[9] Botao Hao, Tor Lattimore, and Chao Qin. Contextual information-dires directed sampling. In International Conference on Machine Learning, pages 1146-8464. PMLR, 2022.
[10] Chris Lu, Yannick Schroecker, Albert Gu, Emilio Parisotto, Jakob Foerster, Satinder Singh, and Feryal Behbahani. Structured state space models for in-context reinforcement learning. Advances in Neural Information Processing Systems, 37:47016-47031, 2023.
[11] Xudong Gong, Feng Dawei, Kele Xu, Bo Ding, and Huaimnn Wang. Goal-conditioned on-polic reinforcement learning. Advances Neural Information Processing Systems, 38:45975 2024.
[12] Xinran Li and Jun Zhang, Context-aware communication for multi-agent reinforcement learning. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, pages 1156-115111 1164, 2024.
[13] Muning Wen, Jakub Kuba, Runji Lin, Weinan Zhang, Ying Wen, Jun Wang, and Yaodong Yang. Multi-agent reinforcement learning is a sequence modeling problem. Advances in Neural Information Processing Systems, 36: 165091 2022.
[14] Matthew Riemer, Khimya Khetarpal, Janarthanan Rajendran, and Sarath Chandar. Balancing context length and mixing times for reinforcement learning at scale. In Advances in Neural Information Processing Systems, 2024.
[15] Wenhan Xiong, Jingyu Liu, Igor Molybog, Hejia Zhang, Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Rungta, Karthik Abinav Sankararaman, Barlas Oguz, et al. Effective long-context scaling of foundation models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies (Volume 1: Long Papers), pages 4643 4663, 2024.
[16] ZiRui Wang. Yue Deng, Junfeng Long, and Yin Zhang. Parallelizing model-based reinforcement learning over the sequence length. In Advances in Neural Information Processing Systems, 2024.
[17] Hao Liu and Pieter Abbeel. Blockwise parallel transformers for large context models. Advances in Neural Information Processing Systems, 37, 2023.
[18] Norman P Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, et al. In-datacenter performance analysis of a tensor processing unit. In Proceedings of the 44th annual international symposium on computer architecture. pages 1-12, 2017.
[19] Jonathan Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, and Emma Brunskill. Supervised pretraining can learn in-context reinforcement learning. Advances in Neural Information Processing Systems, 37, 2023.
[20] Taku Yamagata, Ahmed Khalil, and Raul Santos-Rodriguss Q-learning decision transformer: Leveraging dynamic programming for conditional sequence modelling in offline rl. In Interna- tional Conference on Machine Learning, pages 09989-39007, PMLR, 2023.
[21] Shengchao Hu, Li Shen, Ya Zhang, Yixin Chen, and Dacheng Tao. On transforming reinforce- ment learning with transformers: The development trajectory. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12):8580-855 -5599, 2024.
[22] Eghbal Hosseini and Evelina Fedorenko. Large language models implicitly learn straighten neural sentence trajectories to construct predictive representation of natural language. Ad- vances in Neural Information Processing Systems, 37, 2023.
[23] Kenton Lee, Mandar Joshi, Tulia Raluca Ture, Hexiang Hu, Fangyu Liu, Julian Martin Eisensch- los, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. Pix2struct: Screenshot parsing as pretraining for visual language understanding. In International Confer- ence on Machine Learning, pages 18893-18913 PMLR, 2023.

---

[24] Jordan Terry, Benjamin Black, Nathaniel Grammel, Mario Jayakumar, Ananth Hari, Ryan Sullivan, Luis S Santos, Clemens Dieffendahl, Caroline Horsch, Rodrigo Perez- Vicente, et al. Pettingzoo: Gym for multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 55320032- 2021.
[25]
Arnaud Fickinger. -ultt- gridworld environment for openai gym. https://githe com/ Somm/AnauuFFikkkingean httpr//awwa.iiknngggggg/gmmmmmaa
2020.
[26] Karol Kurach, Anton Raichuk, Piotr Stanczyk, and Michat Zajac. Google research football: A novel reinforcement learning environment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 4501-4510, 2020.
[27] Benjamin Ellis, Jonathan Cook, Skander Moalla, Mikayel Samvelyan, Mingfei Sun, Anuj Mahajan, Jakob Foerster, and Shimon Whiteson. Smacv2: An improved benchmark for cooperative multi-agent reinforcement learning. Advances Neural Information Processing Systems, 37:37567-37593, 2023.
[28] Jibyeong Jeon, Jiwon Park, Chanhee Park, and U Kang. Frequant: A reinforeementtleemming based adaptive portfolio optimization with multi-frequency decomposition. In Proceedings of the 30th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pages 1211-1221, 2024.
[29] Nate Gillman, Daksh Aggarwal, Michael Freeman, and Chen Sun. Fourier head: Helping large language models learn complex probability distributions. In International Conference on Learning Representations, 2025.
[30] Hajer Bahouri, Jean- Chemin, and Raphaël Danchin. Fourier analysis and nonlinear partial differential equations. Springer, 2011.
[31] Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in neural information processing systems, 36:24611-24624, 2022.
[32] Tabish Rashid, Mikayel Samvelyan, Christian Schroeder De Witt, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research, 21 (1 2020.
[33] Jianhao Wang, Zhizhou Ren, Terry Liu, Yang Yu, and Chongjie Zhang, Qplex: Duplex dueling multi-agent g-learning International Conference on Learning Representations, 2021.
[34] Vaswani, N Shazeer, N Parmar, Uszkoreit, L Jones, A Gomez, Kaiser, and Polosukhin. Attention is all you need. In Advances in neural information processing systems, 2017.
[35] Ziyang Wu, Tianjiao Ding, Yifu Lu, Druv Pai, Jingyuan Zhang, Weida Wang, Yaodong Yu, Yi Ma, and Benjamin D Haeffele. Token statistics transformer: Linear-time attention via variational rate reduction. International Conference Learning Representations, 2025.
[36] Jake Grigsby, Linxi Fan, and Yuke Zhu. Amago: Scalable in-context reinforcement learning for adaptive agents. In International Conference on Learning Representations, 2023.
[37] Simo Särkkä and Lennart Svensson. Bayesian filtering and smoothing, volume 17. Cambridge university press, 2023.
[38] Nicolaas G Van Kampen. Stochastic differential equations. Physics reports, 24(3):171-228, 1976.
[39] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.063441 2017.
[40] Dmitriy Drusvyatskiy and Adrian S Lewis. Error bounds, quadratic growth, and linear conver- gence of proximal methods. Mathematics of operations research, 43(3):919 9488 2018.
[41] Naftali Tishby, Fernando C Pereira, and William Bialek, The information bottleneck method. arXiv preprint physics/0004057, 2000.
12

---

[42] Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathematical statistics, pages 400-407, 1951.
[43] John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In International conference machine learning, pages 1889-1897. PMLR, 2015.
[44] Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael J Cree. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning, 110(2)393344) 416, 2021.
[45] Thomas M Cover. Elements of information theory. John Wiley Sons, 1999.
[46]
Richard Sutton and Andrew G Barto, Reinforcement learning: An introduction, volume MIT press Cambridge, 1998.
13

---

Appendix Overview: In this Appendix we provide important details that could not be included in the main text due to space constraints. First Appendix [ we provide detailed proofs for the low-frequency truncation in discrete space. Next in Appendix B we provide the proof of the approximately long-term advantage lower bound the contextual information over static counterparts in dynamic environments. Finally, in Appendix C we provide additional details about our experiments discussed in the main text.
A Proof of Low-Frequency Truncation in Discrete Space
A.1 Dyadic Partition of Unity
The following construction of a dyadic partition of unity is standard in harmonic analysis; see, for example, [30]. We include it here for completeness.
Let C be the annulus defined as C' { Rd | 1 s K<l = )) There exist measurable functions X
and ye, taking values in the interval [0, 1], belonging respectively to ((, {)) and D(C), such that:
Vg TRdd x(5) + Σ (2 =
(16)
120
and for all:
AA E Rd (0), Σ ((2 1.
(17)
jEZ
These functions satisfy the disjoint support conditions:
b jز 2 - Supp (( ((2 0,
(18)
j 2 1 - Supp X ( Supp
0.
(19)
Defining the translated annulus C as Cdef = B(0, 3 + C, we note that C' remains an annulus and satisfies:
|j - jز| 2 5 = 2d C n 2CC = 0. Furthermore, the functions X and o satisfy the bounds:
(20)
Vg E Rdd
1
xE) + Σ (2 1
(21)
2
E YEERR (0),
Σ (2 (2) $1. JEZ
(22)
The aforementioned method establishes a smooth dyadic decomposition of frequency space using radial functions x and y ensuring partition of unity while maintaining disjoint support conditions. The construction guarantees that the frequency space is effectively covered while avoiding excessive overlap, making it well-suited for applications in harmonic analysis and function space theory. The inequalities further confirm that the decomposition remains stable, with bounded sums ensuring proper reconstruction properties.
A.2 Proof of Dyadic Partition of Unity in Discrete Form
To extend this formulation to the discrete setting, we consider similar approach where the continuous frequency domain is replaced by discrete grid. In this section, we first supplement the details of some important definitions. Then supply the essential proofs of our method.
To facilitate a dyadic-like decomposition in the discrete frequency domain, we introduce set of window functions (the low-pass window function and the band-pas, window functions) that partition the frequency spectrum into complementary regions. These functions ensure that different frequency components of a signal are captured separately, allowing for a structured analysis of its spectral content. Specifically, the low-pass window function X [k] is defined as:
1,
k S 2d or k 2 t - 2
X(k]
(23)
0,
otherwise,
14

---

where m is an integer satisfying 0 < m < J which determines the cutoff for low- low-frequency retention. This function effectively captures the low-frequency trends of signal while discarding high-frequency components.
In contrast, the band-pass window functions [y(k] isolate specific frequency bands and are defined as:
1, + I < 2 or t - 2/+m+1 < k S - -
(24)
0, otherwise,
j J m, each function spans high-frequency dyadic band. These band-pass functions systematically cover the frequency spectrum, ensuring that different frequency components are separately analyzed while maintaining structured partitioning. Using these window functions, the signal s(u] can decomposed into distinct components. The low-frequency component, which encapsulates the long-term trend of the signal is obtained as discrete inverse Fourier transform. And the band-pass components, which capture fluctuations at specific dyadic frequency scales are given by:
Ays|u] = IDF (1)))
s( j 0....... J 1
(25)
By summing these components, the original signal can be approximately reconstructed as:
s|u] 1ssu +
Σ Ays(u].
(26)
This decomposition demonstrates that the essential features of the signal are effectively captured across multiple frequency scales, allowing for a detailed analysis of its spectral characteristics.
key property of our method is that the window functions form an approximate partition of unity in the frequency domain. This ensures the decomposition provides stable and comprehensive representation of the signal. Specifically, the functions satisfy the relation:
J-1-m Σ
X
1
(27)
for most frequency indices k. This property can be verified by examining different frequency regions.
1. Low-Fre Fre-Feeqqunyy Regions:
When the frequency index falls within the low- oow-freuuency range, the low-pass function fully retains the frequency components, while all band-pass functions are inactive:
X 1,
1 0,
k Uk 2 2 Vj.
(28)
Consequently, the summation property holds:
J-1-m
X
+
= 10= 0=
(29)
This ensures that the low-frequency components are preserved without interference from frequenyyyccy bands.
2. Band-Pass Regions:
In the band-pass regions, the low-pass function does not contribute, while exactly one band-pac pass function is active:
X 0, J! s.t. 1a[k] 1, 25 < k < 2
(30)
This guarantees that the sum remains unity:
J- Σ 0
X +
(31)
Thus, each frequency component assigned uniquely to one of the band-pass filters, ensuring overlap or redundancy.
15

---

3. Boundary Points:
At the boundary points (k = 22-m. 22+++++++11 , 2 t - -1+m+1
where transitions occur between different frequency bands, both the low-pass and band-pase pasd-pass window functions may be inactive, leading to:
X [k]
0,
Φ [k] 0, Vj.
(32)
Consequently, at these discrete boundary points, the summation deviates from unity:
J-1-m
[
Σ
(33)
To further analyze the convergence properties of this decomposition, we define the error function:
J-1-m
E[k] = 1
[
+
Σ
[k]
(34)
5=
The of indices where E[K] is given by:
& [k B 03.
(35)
Since E[k) is nonzero only at a finite number of boundary points, the measure of its support satisfies:
E < t.
(36)
Thus, as é + the fraction of affected indices vanishes, leading to an almost everywhere convergence:
3
O.
(37)
The above proofs ensure that the partitioning scheme provides a stable and asymptotically exact decomposition in the frequency domain.
Another fundamental aspect of our method is the disjointness and independence of the window functions, which guarantees that the extracted components remain distinct and do not interfere with each other. This separation is maintained through the following two properties:
1. Between different band-pass windows: For any j + jr, the support intervals of the band-pass window functions are disjoint:
[2/+m 2 n [2/++m 2/+m+1)= = 0.
(38)
As a result, the corresponding window functions satisfy:
+ [k] [k = 0, Vk.
(39)
2. Between the low-pass and band-pass windows: The low-pass function X[k] is supported
in the low-frequency regions & < 2 or h: > ( 2 On the other hand, each band-pass
function [ supported in the range 2 < k < 2 Since 2 >
follows that the support of X[k] and by [k] mutually exclusive, ensuring:
[[k] O [ O, Vj
Based on the above method, the annulus C' is substituted with a corresponding set in the discrete Fourier domain, and the dyadic scaling operations are adapted to respect the discrete nature of the transform. The goal remains to construct a stable decomposition that preserves the essential properties of the continuous case while accommodating the constraints imposed by discrete sampling.
B Proof of Long-Term Advantage Lower Bound of Adaptive Length
Theorem (Advantage Lower Bound of Adaptive Length): time I, let Tadao be the adaptive context length, Lix be fixed context length, and the mutual information loss of L be denoted as
16

---

(L).. Under mild assumptions. the expected cumulative reward difference between adaptive and fixed context length satisfies the following regret bound:
Σ -C(
Lr(Ladap)) 2 ((7) -
(40)
=Q(T) (when T is sufficiently large)
where 0 S a < 1, with a being a non-deterministic parameter that varies with the environment.
Proof: To prove this theorem, we first model the non-stationary environment, including the ob- servation and its latent variable. Specifically, at each time i. the agent receives noisy observations Or = g(E)) + ez, where (6:)=1 represents the latent variable, and the error term 6, NN o [77. The latent variable sequence (Er)==1 governs environmental dynamics and follows diffusion process (38
de: = rوt dt + n(&:)dW,.
(41)
where W. represents standard Brownian motion, pa() and 17(-) denote the drift and diffusion coefficients, respectively. This diffusion model should be viewed as a modeling approximation of the discrete-time dynamics (e.g., an Maruyama discretization with unit time step), and is imposed to capture gradual stochastic drift in the latent state.
For any window length I., the mutual information [aa; L.) quantifies the information content of the action ar about the latent variable ε
I(a E H()) (Edaa, L),
(42)
where H(G)) represents the entropy of & (prior entropy), and (EE L) represents the conditional entropy of & given a and window length L.. With the theoretical optimal window length [,, the mutual information loss of I. can be obtained:
LL( I(ari())))) (ar; L)
(43)
Since the action a, is generated from the observation sequence 0t-1.t = for I...,...., 00} via the stochastic policy (a, ot)La), the data processing inequality implies:
laa () S Ex
(44)
With the equation 42
we obtain:
I(o L) = HE.) H (EelorLLa).
(45)
Thus, the upper bound on the mutual information loss is given by:
(L( < I(o:-La; & - [(o ((oL-L:t:: EL = H (Eelo:LLa)) - (EE)oo--274)
(46)
Based on the -ourie truncation, we assume the policies approximately fully utilize the observed information; thereby, we obtain:
(T) (( (E 14:))-
(47)
Next, considering the general situation that the latent variable &, can be regarded as a Gaussian form for the posterior distribution p(E, (ot- (37] This setting is reasonable under several mild and widely satisfied conditions: (i) the dynamics and observation models can locally approximated as
linear or weakly nonlinear in the neighborhood of the true latent state L (37 38]
; (ii) the observation
noise Gaussian, consistent with the structure of the Kalman filter [37;; and (iii) the functions u(Ex) and r7(E), often parameterized by neural networks, are smooth and differentiable, allowing for local Taylor expansions or moment-baat based approximations such as sigma-poini propagation [37 38 Therefore, the posterior distribution can be reasonably approximated as:
p(E: (()))))).))) r.te) -V(,,,
(48)
where & denotes the posterior mean and of the posterior variance. Consequently, the conditional entropy can be computed as:
1
H HErroo, n.)
2 1og(2Teo7)))
(49)
17

---

Notably, this Gaussian approximation is particularly justified in policy optimization algorithms such as PPO (3) where updates are constrained to remain close to the current policy, effectively preserving local linearity in the latent-to-ohservation mapping.
Due to the fact that I is the optimal window length at time step i, we assume that the posterior vari- ance of behaves in a manner where it decreases as the window length approaches the optimal value L; [4]]. This behavior is common in many estimation problems, where the system's 's performance improves as it gets closer to the optimal configuration. In particular, reference [4 suggests that the most useful information for decision-making is concentrated around certain "bottleneck" points, and small deviations from the optimal choice result progressively diminishing returns in terms of information processing. Given this intuition, we assume that the posterior variance dgjr can be regarded as behaving convexly in the neighborhood of L; because this ensures that small changes around the optimal window length lead to an increase in variance, which represents deterioration in the quality of the estimation. Specifically, there exists a positive constant k >> 0 such that:
o2 2 min + + -
(50)
where min o1,1; denotes the posterior variance achieved at the optimal window length.
Combining equations 47 and 49 the mutual information loss can be lower bounded in terms of the posterior variance ratio:
(L) ~
2 log
L
(51)
q min
Substituting the lower bound in equation
50 into equation 51 we obtain:
1 log 1 k(L ((L-1;) L2)
(.)
(52)
2
20 min
Applying the inequality log(1 + x) 2
for x > 1, we further derive:
I+x
K(L 1
2n
k(L L;)
(L( 2 1
(53)
R(L-L,)
4 min
2k 1;)²
20
For a fixed window length Lax, the total information loss over T time steps can then be bounded from below:
C.(LL))
>
Σ
k
Li)²
1
(54)
4o min
(Trxx 12
t=1
20 min
When Lix Li exceeds a threshold Ç. the denominator in equation 53 is bounded, and the loss is On the other hand, when LLix Li1 < the
lower bounded by positive constant Io +2RCZ quadratic term dominates and the loss scales (T(xs)
kça
2 4riin (Lix 22)2 Therefore, for every we have the pointwise lower bound (T(ss) 2 minf c, 2oinm -Ton I,)²]. Then, due Lix does not coincide with the time-varying optimum for all but a vanishing fraction of rounds. Therefore, we assume there exist constants p (0, 1] and Ç > 0 such that [ft (1..... T} |Lxx L; >
cH 2 PT, or, more generally, that ΣT (Lax L2)² = 2(T). Under either condition, the above per-step lower bounds imply
Σ Ce(Lx)) Q(T),
(55)
demonstrating that any fixed window length incurs linear cumulative loss over time unless it tracks the optimal I.; (in the sense of maintaining non-vanishing tracking error frequency linear-order cumulative squared tracking error).
Considering the adaptive strategy for selecting the context length Ladap at each time t. We adopt the most universal polynomial rate to describe the adaptive policy converging to the optimal window length 42 :
Ladap,t Lil S E Oct 8), 3 0.
(56)
18

---

which are commonly used in general convex problems and in stochastic algorithms. In contrast, exponential convergence, etc., often requires extremely strong assumptions such as strong convexity and global smoothness.
Next, based on the equation 53, for small deviations IT. T.El < S. applying the first-order approxi- mation log(1 + z) Rs I, we obtain:
k (I. - 12)
(L)
(57)
46 min
This shows that in a local region -L- Lil < Ç, the mutual information loss behaves quadratically with respect to (L- - 1 This local quadratic behavior is compatible with the global Lipschitz condition via the boundedness of gradients and compactness of the parameter space.
Moreover, in reinforcement learning algorithms such as TRPO 43 or PPO [39], policy updates are performed under a trust-region style constraint that limits distributional shift, typically expressed via a KL-divergence threshold (or a clipped surrogate that approximates such constraint). As canonical form, TRPO enforces an average KL constraint between successive policies, i.e.,
ES DKt To. (-1 o) Te (- (1o))] ŌKL.
(58)
which motivates the assumption that the learned policy is locally stable.
To relate the policy distributions induced by different context lengths, adopt fixed-dimensional representation (0r,(0r r.x) fed into the policy network, and make the following standard smoothness assumptions: (i) the representation varies Lipschitz-continuusk) with respect in neighborhood
of L", P (ot-L:) - Φ =+>== < Cip L L'I; (ii) the policy network is Lipschitz with
respect its input representation in TV distance, || (p) T.- (') I Cnet lle r'1l 44
Combining (i)- yields that there exists a constant C₂ = Cnet Cip > 0 such that, for sufficiently close to L",
|| ot-La) - *(. (0---)))l S C2 - L*,,
(59)
and in particular for the adaptive choice Lradap-
|| ot Ltdsp:)) (-- : I Cx| L'),
(60)
where C₂ depends on the representation and network structure, and Ladap denotes the adaptive context length.
Next, considering that mutual information larii L) is continuous with respect to the under- lying policy distribution (under standard regularity conditions, e.g., finite action space and local boundedness) [45], we assume there exists a constant C₁ > 0 such that, for L sufficiently close to L*,
[(acc & L) Iaac& L')l S Cr || L) x L')[lv.
(61)
Finally, combining (60) and 61 we obtain
[[(ae:E: Ladap)
[a L*) L L- = K Ladep L*
(62)
For any L, L", we consider two cases: (i) Local region (L L'| < Q): In this case, we directly
apply the previously established equation 62 (ii) Global region (L L*I 2 (): Since the mutual information is upper bounded by the entropy H HG)), we have:
[Taa & [.) I(a;; & [+)] S H())) S H(E)) [L I.-
(63)
To obtain a time-uniform Lipschitz constant, we assume that & takes values in a finite alphabet (a standard finite-state setting in reinforcement learning (46]). Then the entropy is uniformly bounded as
(&)) < log E =: Hmax < xc,
Vt,
(64)
where the bound follows from standard ioormattiontheoretii inequalities 45 cases, define:
Combining both
:= max CiC., H(E,)
(65)
C
19

---

then for any L, , the mutual information satisfies a global Lipschitz condition:
(a4; La - (ay Ex - KLL -*
(66)
Further, we can obtain the following bound of adaptive context length on single-step mutual informa- tion loss:
(Lddap) = [[(a; & L) - 1(ari 1*) S K Ladap - - -
(67)
This establishes the global Lipschitz continuity of mutual information with respect to the context length L.
Summing over t to T, get:
Σ Cu S K 8
(68)
Now we apply standard results from numerical analysis of p-series:
If 3 > 1, then the series Σ 1-8 converges. Hence, the cumulative information loss is
bounded: Σ Le(Ladap) = (1)).
If 1, then the series becomes harmonic and grows logarithmically:
Ei=r 1-1
Ollog T). Thus, Σ (Ladap)) = Olog T).
0 < 3 < 1, then the series grows polynomially: E t- = Consequently, ΣT= Le Laddo)))) = O(T2-8).
In all cases, the cumulative information loss is sublinear in T, i.e. there exists a < such that:
T
-
(69)
Combine with the equation 55 and the equation 69 we obtain:
Σ (Lo(Lax) Le (o(Lada)) (2()) O
(when T' is sufficiently large)
(70)
C Additional Details for Experiments
C.1 Environments
Sample Spread The Sample Spread environment is cooperative multi-agent task where agents must coordinate their movements to cover multiple static landmarks, aiming to minimize the overall distance between agents and landmarks while avoiding inter-agent collisions. In the original setting, the number of agents and landmarks is equal (3 each), which may lead to agents remaining stationary. To encourage exploration and promote more dynamic coordination behavior, modify the setting to include 4 agents and 3 landmarks. The action shape is 5. The reward design is as follows:
Each agent receives a local penalty of 11.0 for every collision with other agents, encouraging collision avoidance.
The global reward is defined as = Σ1 -- miny |lp, - 4l]. where py and I denote the positions of agent j and landmark i, respectively, encouraging agents minimize the overall distance to landmarks.
The observation shape is 24:
Minigrid Soccer Game The MiniGrid Soccer Game is multi-agent, competitive and cooperative environment in which agents must coordinate to score goals using shared balls. Agents pass
20

---

Table The Observation Features of Sample Spread
Feature
Dim
Description
Self Velocity
2
Agent's own velocity vector
Self Position
2
Agent's own position in world coordinates
Landmark Relative Positions
8
Relative positions of 4 landmarks to the agent
Other Agents' Relative Positions
6
Relative positions of the other agents the agent
Communication Vectors
6
Communication features from the other 3 agents
Total
24
Final observation dimension per agent
the balls, intercept opponents, and strategically position themselves to influence the game outcome. In our specific configuration, we use balls (red), 3 teams (blue, green, and yellow), with agents per team, and each team is assigned goal of corresponding color. The ciiion-bbervattinn space partially observable. The rewards are global-shared, which all agents receive the same reward whenever any agent scores or concedes goal:
Each agent receives a shared reward of +1 upon successfully picking up the ball from the ground. Each ball only provides this reward once.
A shared reward +10 is given when any agent scores a goal into its own team's designated goal area.
A shared penalty of is applied if the ball is accidentally scored into an opponent's goal.
When holding the ball, agents receive a step-wise penalty of -0.02 x dist based on the distance between the ball and the agent's own goal.
When holding the ball, agents receive a dense reward of +0.2 x progress based on the positive progress made toward their own goal.
Each agent observes a 3x3 local grid, and the shape of each cell is 6:
Table 4: The Observation Features of Minigrid Soccer Game
Feature
Dim
Description
Object Type
Type index (wall/door//geet/keyl..
Color
Color index (green/blue//..)
State
Object state (0-2 for door open/closed/loceed)
Carried Type
Type of carried object (0 if none)
Carried Color
Color of carried object (0 if none)
Direction/ Direction/Marker
Agent direction (0-3) or current agent flag (0/1)
Total
Academy 3 vs with Keeper Academy vs with Keeper environment a multi-agent scenario where 3 offensive agents cooperate to score against goalkeeper and a defender. The action space is discrete with 19 actions, covering basic football behaviors like passing, shooting, and movement. The reward structure is sparse: +100 for scoring goal, -1 if the episode ends without scoring. Besides, we evaluate the model every 100 training episodes, each time over 30 test episodes to compute the average win rate. The observation shape is 26:
Academy Counterattack- Hard environment a multi-agent scenario where 4 offensive agents cooperate to score against goalkeeper and a defender. The action space is discrete with 19 actions, covering basic football behaviors like passing, shooting, and movement. The reward structure is sparse: +100 for scoring goal, -1 if the episode ends without scoring. Besides, we evaluate the model every 100 training episodes, each time over 30 test episodes to compute the average win rate. The observation shape is 34:
Academy Counterattack-HHe
21

---

Table The Observation Features of Academy 3 vs 1 with Keeper
Feature
Dim
Description
Ego Player Position
2
(xt, y) position of the observing agent
Teammates Relative Positions
4
Relative positions of 2 teammates ego
Ego Player Direction
2
Velocity vector (direction) of the ego agent
Teammates Directions
Directions of teammates
Opponents Relative Positions
6
Relative positions of 3 opponents w.r.t. ego Directions of opponents
Opponents Directions
6
Ball Relative Position
2
Ball position relative to ego
Ball Height
1
Ball coordinate (height)
Ball Direction
3
Ball velocity in (z, 2)
Total
26
Table 6: The Observation Features of Academy Counterattack-Haun
Feature
Dim
Description
Ego Agent Position
2
Agent's own (æ. y) coordinates
Teammates Relative Positions
6
Relative (dx, dy) of 3 teammates
Ego Agent Direction
2
Movement vector (Vz, By)
Teammates Directions
6
Movement vectors of 3 teammates (Uz; Uy) x 3
Opponents Relative Positions
6
Relative (dz, dy) of 3 opponents
Opponents Directions
6
Movement vectors of 3 opponents (Uz, Uy) x 3
Ball Relative Position
2
Ball (I,, y) relative ego agent
Ball Height
1
Ball z-coordinate (altitude)
Ball Direction
3
Ball velocity (Ves By Ug)
Total
34
C.2 Experiments with Sequence Processing Methods
All experiments were conducted using an NVIDIA A100 GPU, with the longest single training run taking approximately one month.
Baseline Methods
Transformer 34 A deep learning architecture that utilizes self-attention and positional encoding to model complex dependencies across sequences.
Token Statistics Transformer (ToST) 35
A recent Transformer variant, using data- dependent low-rank projection based on the second moment statistics of input token features, and achieving linear computational complexity.
AMAGO An in-context reinforcement learning algorithm that enables long-equennee Transformers to process entire trajectories parallel, overcoming the memory capacity and long-term planning bottlenecks of traditional recurrent networks.
Hyperparameter We provide the hyperparameters used in each environments as follows:
Central Agent with Low-Frequency Truncation Based on the Dyadic Partition of Unity in Discrete Form, we assign distinct www-frquency truncation lengths each environment. Specifically, after applying the Discrete Fourier Transform (DFT), we retain the first 4, 128, 64, and 64 frequency components for the Sample Spread, MiniGrid Soccer Game, Academy 3 vs with Keeper, and Academy Dounterattack-Hare environments, respectively. The above frequency components are then inputted to the central agent, for which the input dimension defined as kg- The action space for the central agent in each environment is defined follows:
22

---

Table 7: Hyperparameter Configuration
Parameter
Value
Learning Rate
0.001
Discount Factor (7)
0.98
GAE Coefficient (A)
0,95
PPO Clip (e)
0.2
Training Epochs
10
Batch Size (Sample Spread)
25
Batch Size (Minigrid Soccer Game)
128
Batch Size (Academy
with Keeper)
50
Batch Size (Academy Counterattack-Hary
50
Entropy Coefficient (19)
0.01
MLP Hidden Layers (Sample Spread)
[256, 64, 16]
CNN Hidden Layers (Minigrid Soccer Game)
[16, 32, 64]
MLP Hidden Layers (Minigrid Soccer Game)
[256, 128, 64]
MLP Hidden Layers (Academy 3 vs I with Keeper)
[1024, 256, 64]
MLP Hidden Layers (Academy 3 vs 1 with Keeper)
[1024, 256, 64]
Activation Function
ReLU
Optimizer Type
Adam
Table 8: Central Agent Action Space
Environment (Step> Threshold)
Action Space
Sample Spread
0, 1, 2,
MiniGrid Soccer Game
0, 1, 2, 4, 8, 16, 32, 64
Academy vs with Keeper
0, 0, 0, 0, O,
1, 16, 32, 64
Academy Counterattack-Hart
0, 0, O.
8, 16, 32, 64
The action space of the central agent in each environment is adaptively constructed based on the current time step using dyadic partitioning rule. Specifically, for each environment, define threshold step value: Sample Spread (t > 7), MiniGrid Soccer Game (t > 255), Academy vs with Keeper and Academy Counterattack- Hard
127). When the step t exceeds the corresponding threshold, the action space composed of a set of dyadic components {2), 2 ,,..,. ,,.).,,)), where k min (logg (ko), [log2 !] 1-11 1), and the vector left-padded with zeros match the fixed dimension. The resulting component sets for each environment under this truncation rule are summarized in Table
This work develops an adaptive context length optimization method with Fourier-based low-frequency truncation for multi-agent reinforcement learning (MARL). The proposed approach significantly improves the efficiency and effectiveness of MARL systems, enabling better decision- maaiing in complex, dynamic environments.
The positive societal impacts of this research include advancing intelligent multi aagtt- systems in diverse domains such as transportation management, robotics, and resource allocation. These improvements can contribute to enhanced safety, reduced energy consumption, optimized traffic flows, and overall better management of complex systems benefiting society.
By promoting more efficient learning and adaptation in multi-agent environments, this work helps pave the way for scalable and practical AI applications that address real-world challenges with improved reliability and performance.
C.3 ACL-LFT Algorithm
All methods in our experiments (including ours and the baselines) follow the same structural setting: a centralized module processes only the historical information and transmits distributed agents for decision-making All methods share the same network architecture, ensuring fair comparison across all baselines. The pseudocode of the ACL-LFFT training process under this unified framework provided in Algorithm [
23

---

Algorithm 1: ACL-LFT Policy-Making and Training Algorithm
Input:
Distributed agents' policies [0:] shared value function øi central agent's policy tc. value function Φet horizon T; epochs Ke Ka-
Output: Updated policies {0:)i=1 0g and value functions é, Oc-
Initialize per-agent episode buffers {B; } reset environment;
for episode 1 to max mai do do for i to N do
for 1 = 0 10
Extract historical state S,' and perform Fourier Transform;
Obtain low-frequency section sf:
Obtain optimal contextual information 8, ",";;
end for 1 N do
Obtain of by st opt and s/: perform of:
Obtain s141 and r/:
Store transition T/. 8/41 in B;;
end
Obtain rf; store center transition T end
Compute advantages Af using GAE with Pe; for epoch = 1 to K₂ do
Update policy Oc. value function dci end
Construct centralized critic input st global
Compute advantages A using GAE with of (solobal)
Collect 7f into shared buffer B; for epoch 1 to Ka do Update shared policy O,, value function Φ
end end
C.4
Additional Experiments
Table Performance on SMACv2 with different MARL backbones and coordination mechanisms.
SMACv2
RL-Method Transformer
ToST
AMAGO
Mamba
ACL-LFT
Task
MAPPO
71.5 + 3.9
72.2 3.4
76.1 2.9
72.6 + 3.2
78.9 + 2.8
3s5z 3s6z
QMIX
72.8 = 3.7
73.7 3.6
76.3 3.1
75.1 + 3.7
79.4 + 2.9
QPLEX
73.6 + 3.3
75.1 +
77.5 2.8
75.0 + 3.3
80.1 + 3.0
MAPPO
44.5 + 4.9
46.3 + 4.4
48.1 4.0
46.2 - 4.5
52.7 + 4.2
5m vs 6m
QMIX
44.3 + 5.5
46.9 + 4.8
48.3 + 4.3
46.1 + 5.1
52.4 + 4.6
QPLEX
45.6 = 5.7
47.3 + 4.6
49.5 + 4.8
47.8 + 4.9
53.9 + 4.3
MAPPO
65.6 + 5.7
68.1 + 6.6
74.3 + 4.8
69.0 5.9
77.9 + 5.3
corridor
QMIX
68.5 - 5.9
70.3 + 6.8
75.0 5.3
71.2 + 6.2
78.6 + 5.4
QPLEX
70.6 = 4.7
72.9 4.6
76.3
5.8
73.5 5.5
79.2 + 4.9
As shown in Table 9 ACL-LFT consistently outperforms all evaluated duselines-incldee MAPPO, QMIX, and QPLEX -across all SMACv2 scenarios, with average improvements +2.6% to +4.6% over the best perforrnrnn baseline (typically AMAGO).

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The abstract and introduction clearly summarize the paper's main contributions and accurately reflect its scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
25

---

Justification: he limitations of the work are briefly discussed in the appendix, including the assumptions made and directions for future improvement.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: The paper includes all necessary assumptions and complete proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides sufficient methodological details and experimental settings reproduce the main results.
Guidelines:
26

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide access to the core codebase and key experimental results in the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips.ccppulic//guue http://icc/giiie////eeeuubiiiiii
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
27

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
The paper provides comprehensive details on training and testing settings, including data splits, hyperparameter choices, optimizer types, and selection criteria. Addi- tional specifics are included in the appendix ensure full reproducibility.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
The paper reports error bars and/or confidence intervals for key experimental results. The sources of variability (e.g., random initialization, data splits) are clearly stated, and the methods for calculating error bars are described. Assumptions related the statistical analysis are also discussed.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2- 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
28

---

Justification:
The paper specifies the hardware used and total computational resources
required.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httppp///eerrrpsscc///bbll//ieie
Answer: [Yes]
Justification: The research presented in this paper fully adheres to the NeurlPS Code of Ethics. All experiments and data collection procedures comply with ethical standards, including respect for privacy, avoidance of harm, and fairness. No conflicts with applicable laws regulations are present, and no special ethical concerns arise from the methods or applications discussed.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See Appendix for a discussion of potential broader societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
29

---

11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper does not involve the release of high-risk models or datasets. It proposes novel framework for multi-agent reinforcement learning that operates on simulated environments (c.g., PettingZoo, MiniGrid, GRF), which do not pose significant risks of misuse require special safeguards.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: This paper existing environments. All assets properly cited in the paper with references to their original sources, and their licenses and terms of use have been fully respected.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pwpprssittcooeee thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release any new datasets, models, or other assets.
Guidelines:
The answer NA means that the paper does not release new assets.
30

---

Researchers should communicate the details of the dataset/codeemmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve any crowdsourcing or human subject research. All experiments are conducted simulation environments without human interaction.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: This work does not involve any human subjects or crowdsoureing experiments. All experiments are conducted in simulation environments without human interaction.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
The research does not use large language models (LLMs) as important, original, or non-standard part of its methodology. Any I.IM involvement, any, was limited to general writing editing assistance and does not impact the core scientific contributions.
Justification:
31

---

Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy http:///aaarrpsscc/ooooooerrrr/e for what should or should not be described.
32


Reviews:
[{"review_id": "I7ZSm4Kv7z", "reviewer": "Reviewer_jva4", "summary": "- The paper develops an adaptive context length optimization method with low-frequency truncation for MARL. This is well motivated by the fact that decisions can be non-Markovian — that is, actions may depend on historical states and actions — but extending the MDP with long context lengths can be computationally expensive.\n- The authors propose using a central agent to optimize the context length, along with a Fourier-based low-frequency truncation approach to capture global temporal trends from the context.\n- Experiments on several multi-agent tasks (Sample Spread in MiniGrid Soccer Game and Google Research Football (GRF)) show that the proposed method appears to outperform other baselines that focus on sequence processing, such as Transformer-based methods and AMAGO.", "questions": "- Is the use of the Fourier Transform for context length optimization novel in reinforcement learning? This is not clearly explained in the current version.\n- In Theorem~1, what is $\\Omega$, and why does $\\Omega(T) - O(T^{\\alpha}) = \\Omega(T)$ when $T$ is large?\n- Can the proposed method work in an offline learning setting?\n\n\nPlease also address the questions and comments raised in the \\textbf{Weaknesses} section above.", "limitations": "- There is no discussion of the limitations of the current work.\n- I do not see any major potential negative societal impact of this paper.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n- The problem is well-motivated, and the paper appears to be the first in MARL to develop adaptive context length optimization for multi-agent learning.\n- The idea of using a Fourier-based low-frequency truncation approach to capture global temporal trends is interesting and sound.\n- The paper is well written and clear.\n- The experiments are quite convincing, except that the tasks used seem rather simple (as discussed below).\n\n**Weaknesses**\nWhile I think the methodological part of this paper is quite strong, the experiments are weak:\n\n- The chosen benchmarking tasks seem to be quite simple to illustrate the efficiency of the proposed method for genuinely non-Markovian tasks. Specifically, tasks such as Sample Spread and MiniGrid Soccer Game appear to be mostly Markovian (actions depend primarily on the current state, not long historical states or actions). I believe it would be valuable to include more complex tasks, such as SMACv2 [1], to better demonstrate the method’s efficiency in truly non-Markovian environments.\n- The sequence processing methods are combined with PPO — but PPO is inherently a single-agent algorithm. It is not clear how PPO is applied in the multi-agent tasks. If the authors run PPO independently for each individual agent, this is not an appropriate approach. The authors should consider using MAPPO [2]  (a multi-agent version of PPO) to make the experiments more convincing.\n- Beyond PPO, it would be useful to know how the sequence processing methods perform when combined with different MARL algorithms (e.g., MAPPO, QMIX [3], or IMAX-PPO [4] ). The paper lacks such ablation studies.\n\n[1] llis, B., Cook, J., Moalla, S., Samvelyan, M., Sun, M., Mahajan, A., Foerster, J. N., & Whiteson, S. (2023). SMACv2: An improved benchmark for cooperative multi-agent reinforcement learning. NeurIPs 2023\n\n[2] Yu, C., Velu, A., Vinitsky, E., Gao, J., Wang, Y., Bayen, A., & Wu, Y. (2022). The surprising effectiveness of PPO in cooperative multi-agent games. NeurIPS 2022\n\n[3] Rashid, T., Samvelyan, M., Schroeder de Witt, C., Farquhar, G., Foerster, J., & Whiteson, S. (2018). QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning, ICML 2018\n\n[4] Bui, T. V., Mai, T., & Nguyen, T. H. (2024). Mimicking To Dominate: Imitation Learning Strategies for Success in Multi-Agent Games. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024).", "was_revised": true, "final_justification": "The authors have addressed most of my concerns. I believe the paper makes a meaningful contribution, and I maintain my evaluation as a borderline acceptance.", "rebuttal": "We sincerely thank the reviewer for the insightful comments that were instrumental in strengthening our paper. Below we address each point, with major new experimental results on SMACv2 central to our responses.\n\n**Weaknesses1 & Weaknesses3 & Question3：**  \n1.The chosen benchmarking tasks seem to be quite simple to illustrate the efficiency of the proposed method for genuinely non-Markovian tasks. Specifically, tasks such as Sample Spread and MiniGrid Soccer Game appear to be mostly Markovian (actions depend primarily on the current state, not long historical states or actions). I believe it would be valuable to include more complex tasks, such as SMACv2 [1], to better demonstrate the method’s efficiency in truly non-Markovian environments.  \n2.Beyond PPO, it would be useful to know how the sequence processing methods perform when combined with different MARL algorithms (e.g., MAPPO, QMIX [3], or IMAX-PPO [4] ). The paper lacks such ablation studies.\n\n**Answer：**  \nWe would like to clarify that the chosen tasks may already involve non-Markovian characteristics, as they require long-term path planning and delayed coordination—especially in the GRF. To further validate our method in more complex and representative non-Markovian settings, we have additionally conducted extensive experiments on the challenging SMACv2 benchmark [1]. To simultaneously demonstrate the generality of ACL-LFT, we performed comprehensive ablation studies by integrating it with three distinct MARL algorithms: MAPPO [2], QMIX [3], and QPLEX [4]. The results are presented in the table below.  \n| SMACv2 (win rate)| RL-Method | Transformer | ToST| AMAGO| ACL-LFT|\n|:------:|:-----------:|:----------:|:---------:|:---------:|:---------:|\n| 3s5z vs 3s6z  | MAPPO  | 71.5 ± 3.9  | 72.2 ± 3.4 | 76.1 ± 2.9 |78.9 ± 2.8 |\n|               | QMIX   | 72.8 ± 3.7  | 73.7 ± 3.6 | 76.3 ± 3.1 |79.4 ± 2.9 |\n|               | QPLEX   | 73.6 ± 3.3  | 75.1 ± 3.6 | 77.5 ± 2.8|80.1 ± 3.0 |\n| 5m_vs_6m | MAPPO  | 44.5 ± 4.9  | 46.3 ± 4.4 | 48.1 ± 4.0 |52.7 ± 4.2 |\n|               | QMIX | 44.3 ± 5.5  | 46.9 ± 4.8 | 48.3 ± 4.3| 52.4 ± 4.6 |   \n|               | QPLEX   | 45.6 ± 5.7  | 47.3 ± 4.6 |49.5 ± 4.8 |53.9 ± 4.3 |\n| corridor | MAPPO  | 65.6 ± 5.7  | 68.1 ± 6.6 | 74.3 ± 4.8 |77.9 ± 5.3 |\n|               | QMIX   | 68.5 ± 5.9  | 70.3 ± 6.8 | 75.0 ± 5.3 |78.6 ± 5.4 |\n|               | QPLEX  | 70.6 ± 4.7  | 72.9 ± 4.6 | 76.3 ± 5.8 |79.2 ± 4.9 |\n\nAs shown in the table above, ACL-LFT consistently outperforms all evaluated baselines—including MAPPO, QMIX, and QPLEX—across all SMACv2 scenarios, with average improvements of +2.6% to +4.6% over the best-performing baseline (typically AMAGO). These consistent gains across diverse algorithms and environments strongly support ACL-LFT’s effectiveness in addressing complex non-Markovian dependencies.\n\n**Weaknesses2：**  \nThe sequence processing methods are combined with PPO — but PPO is inherently a single-agent algorithm. It is not clear how PPO is applied in the multi-agent tasks. If the authors run PPO independently for each individual agent, this is not an appropriate approach. The authors should consider using MAPPO [2] (a multi-agent version of PPO) to make the experiments more convincing.\n\n**Answer：**  \nWe would like to clarify that the PPO was referenced because it is used for training the central agent, whereas the distributed agents—in both our method and all baselines—are trained using the MAPPO framework. Algorithm 1 below illustrates the training procedure of the central agent and distributed agents, where the distributed agents follow the MAPPO framework. We will make the distinction clear and include the proper citation to MAPPO [2] in the final version.\n\n---  \n\n**Algorithm 1**    \nInitialize: each distributed agent's policy \\\\( \\\\theta_i\\\\), shared value function \\\\( \\\\phi \\\\) ; center agent's policy \\\\( \\\\theta_c \\\\), value function \\\\( \\\\phi_c\\\\) .  \n**for** episode = 1, 2, \\\\( \\\\ldots\\\\) **do**  \n  Initialize episode buffer \\\\( B_i\\\\);   \n  Reset environment;  \n  **for**  \\\\( t = 0 \\\\) to \\\\( T-1 \\\\) **do**   \n    **for** agent \\\\( i = 1 \\\\) to \\\\( n \\\\) **do**  \n      Extract historical state \\\\(s_t^{-1}\\\\);  \n      Discrete Fourier Transform \\\\( \\\\text{FFT} ( s_t^{-1} ) \\\\);  \n    Obtain low-frequency section \\\\(s_t^c\\\\);   \n    Obtain optimal contextual information \\\\(s_t^{-opt}\\\\);    \n    **end**  \n    **for** agent \\\\( i = 1 \\\\) to \\\\( N \\\\) **do**  \n      Obtain \\\\(a^i_t\\\\) by \\\\(s_t^{-opt}\\\\) and \\\\(s_t^{i}\\\\); \n                  Perform the \\\\(a^i_t\\\\);  \n      Obtain  \\\\(s^i_{t+1}\\\\) and \\\\(r^i_t\\\\), store \\\\(s^i_{t+1}\\\\) in \\\\( B_i\\\\);  \n      Store \\\\(\\\\tau_t^i\\\\);  \n    **end**  \n    Compute attention weights;     \n    Obtain \\\\( r^c_t = \\\\sum_{i=1}^n \\\\omega^i_t \\\\cdot r^i_t \\\\);  \n    Store \\\\(\\\\tau_t^c\\\\);     \n    **end**  \n    Compute returns \\\\( R_t^c\\\\); Compute advantages \\\\(A_t^c\\\\) using GAE;    \n    **for** epoch = 1 to \\\\(K_c\\\\) **do**  \n        Update policy \\\\(\\\\theta_c\\\\), value function\\\\(\\\\phi_c\\\\) ;   \n    **end**  \n    Construct centralized critic input \\\\( s_t^{global}\\\\);   \n    Compute returns \\\\( R_t^i \\\\); Compute advantages \\\\( A_t^i \\\\) using GAE;  \n    Aggregate all \\\\( \\tau_t^i \\\\) in shared buffer \\\\( B \\\\);;  \n    **for** epoch = 1 to \\\\( K_d \\\\) **do**  \n        Update policy \\\\( \\\\{ \\\\theta_i \\\\}_{i=1}^n \\\\); Update value function \\\\( \\\\phi \\\\);  \n    **end**  \n**end**\n\n---\n\n**Question1：**   \nIs the use of the Fourier Transform for context length optimization novel in reinforcement learning? This is not clearly explained in the current version.\n\n**Answer：**  \nIn MARL, the input dimensionality grows exponentially with the number of agents, introducing substantial redundancy and noise into the feature space. Existing methods leveraging Fourier Transform (FT) in RL primarily focus on enhancing feature representation—for example, in state sequence modeling or market dynamics separation [5][6]—by preserving the full frequency spectrum. While this improves expressivity, it fails to suppress irrelevant or redundant components.  \nIn contrast, our work introduces a novel paradigm: strategic spectral truncation for context compression in discrete MARL. Rather than treating FT as a pure representational tool, we employ it as an information filtering mechanism, selectively retaining low-frequency components to capture long-term dependencies while discarding high-frequency noise. \n\n**Question2：**   \nIn Theorem~1, what is \\\\(\\Omega\\\\), and why does \\\\( \\\\Omega(T) - O(T^{\\\\alpha}) = \\\\Omega(T) \\\\) when T is large?\n\n**Answer：** \nIn the given expression, \\\\( \\\\Omega(T) \\\\) denotes an asymptotic lower bound, meaning there exist constants \\\\( c_1 > 0 \\\\) and \\\\( T_1 > 0 \\\\) such that \\\\( \\\\Omega(T) \\\\geq c_1 T \\\\) for all \\\\( T > T_1 \\\\); while \\\\( O(T^{\\\\alpha}) \\\\) represents an asymptotic upper bound, implying there exist constants \\\\( c_2 > 0 \\\\) and \\\\( T_2 > 0 \\\\) such that \\\\( O(T^{\\\\alpha}) \\\\leq c_2 T^{\\\\alpha} \\\\) for all \\\\( T > T_2 \\\\). When \\\\( T \\\\) is sufficiently large and \\\\( 0 \\\\leq \\\\alpha < 1 \\\\), \\\\( O(T^{\\\\alpha}) = o(T) \\\\) is a higher-order infinitesimal relative to \\\\( T \\\\), meaning \\\\( T^{\\\\alpha} \\\\) grows slower than \\\\( T \\\\) and \\\\( O(T^{\\\\alpha}) \\\\) becomes negligible compared to \\\\( \\\\Omega(T) \\\\) in the limit. Consequently, \\\\( \\\\Omega(T) - O(T^{\\\\alpha}) \\\\geq c_1 T - c_2 T^{\\\\alpha} \\\\), and since \\\\( c_2 T^{\\\\alpha} \\\\) is \\\\( o(T) \\\\), it follows that \\\\( c_1 T - c_2 T^{\\\\alpha} = \\\\Omega(T) \\\\) as the subtraction term does not alter the linear growth rate, leading to  \\\\( \\\\Omega(T) - O(T^{\\\\alpha}) = \\\\Omega(T) \\\\) for sufficiently large \\\\( T \\\\).\n\n**Question3：**  \nCan the proposed method work in an offline learning setting?\n\n**Answer：** \nThe proposed method, which includes Fourier low-frequency truncation and adaptive context optimization is inherently compatible with offline reinforcement learning. Firstly, low-frequency feature extraction operates deterministically on historical trajectories, requiring no interaction with the environment. Secondly, the context-length optimization module leverages temporal gradients from stored transitions, making it fully trainable in an offline setting. Moreover, our method is plug-and-play, independent of the specific RL backbone or whether learning is conducted in an online or offline regime.\n\n**Limitations Supplement**: As the number of agents increases, the central agent be subject to increased computational demands, which could impact overall processing efficiency. In future work, this may be addressed by adopting more advanced or specialized network architectures to improve scalability and parallelization. This discussion will be included in the appendix of the revised version.\n\n[1] llis, B., Cook, J., Moalla, S., Samvelyan, M., Sun, M., Mahajan, A., Foerster, J. N., & Whiteson, S. (2023). SMACv2: An improved benchmark for cooperative multi-agent reinforcement learning. NeurIPs 2023\n\n[2] Yu, C., Velu, A., Vinitsky, E., Gao, J., Wang, Y., Bayen, A., & Wu, Y. (2022). The surprising effectiveness of PPO in cooperative multi-agent games. NeurIPS 2022\n\n[3] Rashid, T., Samvelyan, M., Schroeder de Witt, C., Farquhar, G., Foerster, J., & Whiteson, S. (2018). QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning, ICML 2018\n\n[4] Wang, J., Ren, Z., Liu, T., Yu, Y., & Zhang, C. (2020). Qplex: Duplex dueling multi-agent q-learning. arXiv preprint arXiv:2008.01062.\n\n[5] Ye, M., Kuang, Y., Wang, J., Rui, Y., Zhou, W., Li, H., & Wu, F. (2023). State sequences prediction via fourier transform for representation learning. Advances in Neural Information Processing Systems, 36, 67565-67588.\n\n[6] Jeon, J., Park, J., Park, C., & Kang, U. (2024, August). Frequant: A reinforcement-learning based adaptive portfolio optimization with multi-frequency decomposition. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 1211-1221)."}, {"review_id": "oiFao1Gfp6", "reviewer": "Reviewer_mfjH", "summary": "The work addresses the challenge of long context lengths in MARL with a new technique named ACL-LFT. The two novel components are (1) a method for adaptively optimizing context length using a central agent and (2) using Fourier transforms for low-frequency truncation. Their experiments demonstrate that adaptive context lengths outperform fixed context lengths and their work outperforms existing sequence processing methods.", "questions": "- Why is a dynamic context length with transformer-based models better than using a modern RNN (like RWKV/Mamba/TTT) with truncated BPTT?\n- How large are the transformers used (layers, hidden dimension)? Are both the actor and critic transformers for the agents?\n- Theorem 1 seems to make the implicit assumption that the theoretic optimal window length is finite, meaning that longer lengths would actually result in worse mutual information. Is there any justification for this idea?\n- Given that a central agent is choosing the context length, is decentralized execution possible? Additionally, the incorporation of the historical (global) state data into decision-making (as seen in figure 1) also seems to violate decentralized execution.\n- What loss function is used to optimize $r_t^c$? It seems like this reward is used for training the value function, but it is unclear how the transformation matrices prior to the attention weight are optimized ($W^g_Q$ and $W^g_K$).", "limitations": "Unclear; although the checklist says it is in the appendix, I cannot find it.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The evaluation settings include challenging MARL problems instead of only toy settings, demonstrating the effectivness of the technique in real-world scenarios.\n- The proposed method is novel\n\nWeaknesses:\n- The method is generally unclear, especially in describing the central agent (see questions below). Specifically, it does not seem like decentralized execution is possible, which would give this method an unfair advantage over true CTDE methods. I would strongly advise providing pseudocode (in the appendix) to clarify how all parts interact.\n- The benefits of the method seem very marginal relative to AMAGO despite being significantly more complex. \n- I do not see a hyperparameter sweep (especially over the learning rate), which may be giving a disadvantage to some baseline methods.\n\nOther Notes:\n- The statement \"a tailored input representation for MARL is required\" (lines 45-46) does not follow from the earlier statements in the paragraph. It is unclear why a learned input representation from standard MARL is insufficient.\n- The \"extended MDP\" is a non-standard formulation for MARL; a more appropriate framework would be the Dec-POMDP.\n- The simplifications in Equation 16 do not seem justified. In particular, $w_t^i$ still depends on $t$ despite being outside of the sum, and it cannot simply be dropped out since the sum of weights over agents is 1.", "was_revised": true, "final_justification": "In the rebuttal, all my concerns with the paper were addressed, including promises to revise the text to follow more standard MARL formulations (Dec-POMDP) and clarify assumptions about the setting (communication of historical information). The authors also provided additional results to demonstrate that their work is helpful in the standard CTDE domain (no communication), which demonstrates to me that their method is general.\n\nI still feel that the method is more complex than prior baselines while only providing marginal benefit, but I think this work should be accepted as it pushes the frontier of long-context MARL methods.", "rebuttal": "We sincerely thank the reviewer for the insightful comments that were instrumental in strengthening our paper. \n\n**Weakness1 & Other Notes2 & Question4 & Question5:**   \n1.The method is generally unclear, especially in describing the central agent (see questions below). Specifically, it does not seem like decentralized execution is possible, which would give this method an unfair advantage over true CTDE methods. I would strongly advise providing pseudocode (in the appendix) to clarify how all parts interact.  \n2.The \"extended MDP\" is a non-standard formulation for MARL; a more appropriate framework would be the Dec-POMDP.  \n3. Given that a central agent is choosing the context length, is decentralized execution possible? Additionally, the incorporation of the historical (global) state data into decision-making (as seen in figure 1) also seems to violate decentralized execution.  \n4.What loss function is used to optimize \\\\(r_t^c\\\\)? It seems like this reward is used for training the value function, but it is unclear how the transformation matrices prior to the attention weight are optimized \\\\(W^g_Q\\\\) and \\\\(W^g_K\\\\).  \n\n**Answer:**  \nWe sincerely appreciate the reviewers' constructive feedback, we will supplement the appendix with the pseudocode. And to clarify concerns, pseudocode is provided below.\n\n---  \n\n**Algorithm 1**    \nInitialize: each distributed agents' policies \\\\( \\\\theta_i\\\\), shared value function \\\\( \\\\phi \\\\) ; center agent's policy \\\\( \\\\theta_c \\\\), value function \\\\( \\\\phi_c\\\\) .  \n**for** episode = 1, 2, ... **do**  \n  Initialize episode buffer \\\\( B_i\\\\);   \n  Reset environment;  \n  **for**  \\\\( t = 0 \\\\) to \\\\( T-1 \\\\) **do**   \n    **for** agent \\\\( i = 1 \\\\) to \\\\( N \\\\) **do**  \n      Extract historical state \\\\(s_t^{-1}\\\\) and Fourier Transform;  \n    Obtain \\\\(s_t^c\\\\);   \n    Obtain \\\\(s_t^{-opt}\\\\);    \n    **end**  \n    **for** agent \\\\( i = 1 \\\\) to \\\\( N \\\\) **do**  \n      Obtain \\\\(a^i_t\\\\) by \\\\(s_t^{-opt}\\\\) and \\\\(s_t^{i}\\\\); \n                  Perform \\\\(a^i_t\\\\);  \n      Obtain  \\\\(s^i_{t+1}\\\\) and \\\\(r^i_t\\\\);  \n      Store transition \\\\(\\\\tau_t^i\\\\), \\\\(s^i_{t+1}\\\\) in \\\\( B_i\\\\);  \n    **end**      \n    Obtain \\\\( r^c_t\\\\);  \n    Store center transition \\\\(\\\\tau_t^c\\\\);     \n    **end**  \n    Compute advantages \\\\(A_t^c\\\\);    \n    **for** epoch = 1 to \\\\(K_c\\\\) **do**  \n        Update policy \\\\(\\\\theta_c\\\\), value function\\\\(\\\\phi_c\\\\) ;   \n    **end**  \n    Construct centralized critic input \\\\( s_t^{global}\\\\);   \n    Compute advantages \\\\( A_t^i \\\\);  \n    Collect \\\\( \\tau_t^i \\\\) into shared buffer \\\\( B \\\\);  \n    **for** epoch = 1 to \\\\( K_d \\\\) **do**  \n        Update policy \\\\( \\\\{\\\\theta_i\\\\}_{i=1}^n \\\\), value function \\\\( \\\\phi \\\\);  \n    **end**  \n**end**\n\n---\nAll methods in our experiments (including ours and baselines) adopt the same setting: a centralized module only processes historical information and then sends it to distributed agents (all methods share the same network architecture) for decision-making. Thereby this design ensures a fair comparison across all baselines. The motivation for our centralized processing is to reduce redundant computation: rather than letting each agent independently encode its history, a single processing step (either jointly extracting global patterns or separately embedding each agent’s trajectory in batch) can substantially reduce the computational cost.\nFor our framework, we firstly indeed consider the standard Dec-POMDP setting. However, the reliance on historical information introduces non-Markovian dependencies, rather than merely violating partial observability assumptions. To address this, we follow the extended MDP formalism introduced in [1], which is used to characterize non-Markovian dependencies.  \nThe loss function that used optimize \\\\(r_t^c\\\\) as below:\n\\\\[L_t(\\\\theta_c,\\\\phi_c) = \\\\min \\\\left( \\\\frac{\\\\pi_{\\\\theta_c}(a_t^c|s_t^c)}{\\\\pi_{\\\\theta_c^{\\\\text{old}}}(a_t^c|s_t^c)} \n\\\\left( \\\\sum_{k=t}^{T} \\\\gamma^{k-t} r_k^c-V_{\\\\phi_c^{\\\\text{old}}}(s_t^c) \\\\right),\\\\text{clip} \\\\left( \\\\frac{\\\\pi_{\\\\theta_c}(a_t^c|s_t^c)}{\\\\pi_{\\\\theta_c^{\\\\text{old}}}(a_t^c|s_t^c)},1-\\\\epsilon,1 + \\\\epsilon\\\\right)\\\\left(\\\\sum_{k=t}^{T} \\\\gamma^{k-t} r_k^c - V_{\\\\phi_{\\\\text{old}}}(s_t^c) \\\\right)\\\\right)+ \\\\lambda_1\\\\left( V_{\\\\phi_c}(s_t^c)-\\\\sum_{k=t}^{T} \\\\gamma^{k-t} r_k^c \\\\right)^2+ \\\\lambda_2 \\\\sum_{a} \\\\pi_{\\\\theta_c}(a|s_t^c) \\\\log \\\\pi_{\\\\theta_c}(a|s_t^c)\\\\]\nRegarding the \\\\(W^g_Q\\\\) and \\\\(W^g_K\\\\), these are used only during data collection to compute \\(r_t^c\\\\) via an attention mechanism over distributed agents. During PPO training, \\(r_t^c\\\\) is treated as fixed,  \\\\(W^g_Q\\\\) and \\\\(W^g_K\\\\) are excluded from the trainable computation graph. They do not receive gradients and have no role in the optimization process.\n\n**Weakness2 & Weakness3 & Question1 & Question2:**  \n1.The benefits of the method seem very marginal relative to AMAGO despite being significantly more complex.  \n2.I do not see a hyperparameter sweep (especially over the learning rate), which may be giving a disadvantage to some baseline methods.  \n3.Why is a dynamic context length with transformer-based models better than using a modern RNN (like RWKV/Mamba/TTT) with truncated BPTT?  \n4.How large are the transformers used (layers, hidden dimension)? Are both the actor and critic transformers for the agents?  \n\n**Answer:**  \nAs for the rationale behind our method, unlike transformer models which apply attention to all time steps without a clear boundary on what constitutes “useful” history, our method explicitly selects a minimal but sufficient segment of historical information, effectively eliminating redundant context and reducing computation. To further validate the effectiveness of our method, we conduct additional experiments on the more challenging SMACv2 benchmark. Due to space constraints, we report representative results using MAPPO as the backbone, and include ablations across different RL algorithms (QMIX, QPLEX) as well as comparisons against a state-of-the-art RNN variant (Mamba). \n\n|SMACv2 (win rate)|RL-Method|Transformer|ToST|AMAGO|Mamba|ACL-LFT|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|3s5z vs 3s6z|MAPPO|71.5±3.9|72.2±3.4|76.1±2.9|72.6±3.2|78.9 ± 2.8|\n|5m_vs_6m|MAPPO|44.5±4.9|46.3±4.4|48.1±4.0|46.2±4.5|52.7±4.2|\n|corridor|MAPPO|65.6±5.7|68.1±6.6|74.3±4.8|69.0±5.9|77.9±5.3|\n\nWe clarify that the transformer model is used only as baselines for historical information processing to compare with central agent. In the central agent, neither the actor nor the critic is transformer-based, but employs architectures including MLP or CNN, as detailed in Appendix C.2. For the transformer-based baselines, we adopt commonly used configurations(2 layers, 128 hidden dimensions, 4 heads), and all methods are tuned with the same learning rate (0.001), detailed hyperparameters will be provided in the appendix due to space constraints.\nBesides, we initially select transformer-based baselines as they offer dynamic attention to relevant past information, which better captures historical dependencies than RNNs with truncated BPTT.  \n\n**Other Notes1:**  \nThe statement \"a tailored input representation for MARL is required\" (lines 45-46) does not follow from the earlier statements in the paragraph. It is unclear why a learned input representation from standard MARL is insufficient.  \n\n**Answer:**  \nMARL faces challenges unlike NLP because it involves asynchronous, multi-source, and unstructured data from multiple agents, with sparse rewards and diverse learning objectives, making unified, end-to-end input modeling ineffective and hindering generalization. And this issue was identified but unresolved in [2].\n\n**Other Notes3:**  \nThe simplifications in Equation 16 do not seem justified. In particular, \\\\(\\\\omega_t^i\\\\) still depends on \\\\(t\\\\) despite being outside of the sum, and it cannot simply be dropped out since the sum of weights over agents is 1.  \n\n**Answer:**  \nWe apologize for the oversight in handling \\\\(\\\\omega_t^i\\\\). Due to space constraints, we provide a minimal proof; details will be in the final version.  Goal alignment holds via gradient correlation:\n\\\\[\\\\nabla_{\\\\pi} J_c(\\\\pi) = \\\\sum_{t=0}^{\\\\infty} \\\\gamma^{t} \\\\sum_{i=1}^{n} \\\\omega_{t}^{i} \\\\nabla_{\\\\pi} \\\\mathbb{E}[R_{i} \\\\mid \\\\pi]\\\\]\n\\\\[\\\\nabla_{\\\\pi} \\\\left( \\\\sum_{j=1}^{n} J_j(\\\\pi) \\\\right) = \\\\sum_{t=0}^{\\\\infty} \\\\gamma^{t} \\\\sum_{j=1}^{n} \\\\nabla_{\\\\pi} \\\\mathbb{E}[R_j \\\\mid \\\\pi]\\\\]\n\\\\[\\\\left\\\\langle \\\\nabla_{\\\\pi} J_c(\\\\pi), \\\\nabla_{\\\\pi} \\\\sum_{j=1}^{n} J_j(\\\\pi) \\\\right\\\\rangle > 0 \\\\quad \\\\text{when } \\\\omega_{t}^{i} > 0\\\\]\n\n**Question3:**  \nTheorem 1 seems to make the implicit assumption that the theoretic optimal window length is finite, meaning that longer lengths would actually result in worse mutual information. Is there any justification for this idea?  \n\n**Answer:**  \nThe theoretic optimum window length is finite, aligning with mutual information (MI)'s definition \\\\(I(X;Y) = H(X) + H(Y) - H(X,Y)\\\\) . Longer windows degrade MI due to: noise contamination in distant data, non-stationarity making old data irrelevant, and crucially, biased empirical MI estimation in high dimensions under finite samples. \n\n**Limitations Supplement**: As the number of agents increases, the central agent be subject to increased computational burden. In future work, this may be addressed by adopting more advanced or specialized network architectures to improve scalability and parallelization. This discussion will be included in the appendix of the revised version.\n\n[1] Mutti, M., De Santi, R., & Restelli, M. (2022, June). The importance of non-markovianity in maximum state entropy exploration. In International Conference on Machine Learning (pp. 16223-16239). PMLR.\n[2] Riemer, M., Khetarpal, K., Rajendran, J., & Chandar, S. (2024). Balancing context length and mixing times for reinforcement learning at scale. Advances in Neural Information Processing Systems, 37, 80268-80302."}, {"review_id": "YL1qHSFAOV", "reviewer": "Reviewer_FF8i", "summary": "This paper introduces ACL-LFT, a framework designed to address the limitations of fixed context lengths in multi-agent reinforcement learning (MARL). The approach leverages a central agent that adaptively optimizes context length during training by performing temporal gradient analysis and employing a Fourier-based low-frequency truncation for input representation. This truncation allows for global temporal trends to be captured among agents while filtering redundancy, and the central agent's decisions are further refined using attention mechanisms over decentralized agent policies and value estimates. Theoretical results provide regret bounds for adaptive versus static context lengths, and the empirical evaluation demonstrates strong performance on multiple challenging long-term dependency environments (PettingZoo, MiniGrid, Google Research Football).", "questions": "1. Did the authors consider or test other dimensionality reduction or context selection techniques, such as learnable attention over history, PCA, or other compression methods? What guided the choice of Fourier/Littlewood-Paley methods, and are there observed advantages?\n\n2. How is the maximum/context length range decided for each environment/task? Could an automated or meta-learned approach further improve adaptability?\n\n3. Did the authors consider or test other dimensionality reduction or context selection techniques, such as learnable attention over history, PCA, or other compression methods? What guided the choice of Fourier/Littlewood-Paley methods, and are there observed advantages?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths: \n\n1. The proposed combination of a central agent with adaptive context length selection and a Fourier-based redundancy reduction module is well motivated by the challenges faced in MARL. Theoretical support is provided for the advantage of adaptive context length.\n\n2. The method is evaluated on a diverse set of challenging environments, specifically those known for long-term dependency and partial observability issues; these include PettingZoo, MiniGrid Soccer Game, and GRF Academy scenarios. \n\n3. The methodology is grounded with interpretable theoretical backing (Section 2.2, Littlewood-Paley theory) and ensures the frequency-domain truncations are mathematically justified.\n\nWeaknesses: \n\n1. While the methodology proposes to reduce redundancy and computation, there is little concrete analysis (quantitative or qualitative) of the computational overhead introduced by the central agent or the multi-head attention mechanism, especially in large-scale settings. The claims of efficiency could thus be overstated.\n\n2. The environments used are well-chosen for long-term dependencies, but results could be strengthened by reporting on additional MARL domains with severe non-stationary dynamics, environments with different reward delays, or settings with very large agent populations to more robustly demonstrate generalization.", "was_revised": true, "final_justification": "I have updated my review; if the authors address my concerns, I will raise my score.", "rebuttal": "We believe this review may have been intended for a different submission, as the content discussed does not relate to our work.\n\nOur paper does **not** involve in-context learning (ICL), prompt construction, or large language models (LLMs). Instead, we study a **multi-agent reinforcement learning (MARL)** setting and introduce a central agent that dynamically optimizes the context length for each agent via temporal gradient analysis, enabling improved exploration and faster convergence to globally optimal policies.\n\nIn our paper, “adaptive context length” refers to the dynamically obtain optimal contextual information for distributed agent uses. This is fundamentally different from selecting the number of examples in a prompt for ICL.\n\nGiven the clear mismatch in topic, methodology, and terminology, we respectfully ask the reviewer to verify whether this review was submitted in error."}, {"review_id": "MiMflVXYIh", "reviewer": "Reviewer_N3eq", "summary": "This paper proposes a multi-agent reinforcement learning (MARL) framework, ACL-LFT, that adaptively adjusts the context length for decision-making via a central agent. The central agent leverages a Fourier-based low-frequency truncation module to process historical state sequences, with the goal of capturing global temporal trends and reducing redundant information. The authors claim their method improves efficiency in handling long-term dependencies and demonstrate experimental improvements over existing baselines on environments such as PettingZoo, MiniGrid, and Google Research Football.", "questions": "1. Why is the low-frequency truncation method removing *low* frequencies instead of *high* frequencies, if the goal is to retain global temporal information?\n2. Can the authors provide any empirical analysis (e.g., frequency spectrum plots) that demonstrates the benefit of the truncation in capturing relevant dynamics?\n3. How does the proposed method generalize to MARL environments with partial observability or non-stationarity, where context length adaptation may be more crucial?\n4. Why are standard MARL baselines (e.g., QMIX, MAPPO) omitted from experimental comparisons?\n5. Could the authors clarify how the central agent is trained online without inducing instability in the decentralized policies?", "limitations": "* The paper’s main methodological contribution—Fourier-based low-frequency truncation—is not clearly grounded in MARL-specific insights, and its theoretical support is weak.\n* The proposed approach assumes that removing low-frequency components enhances representation, which contradicts the usual interpretation in signal processing and undermines the core justification.\n* The experimental setup lacks tasks that truly stress long-range temporal dependencies, weakening the claimed benefits of adaptive context length.\n* Important baselines from the MARL literature are omitted, casting doubt on the significance of the reported performance gains.\n\nThe paper introduces some interesting architectural ideas, but its core justification is flawed and the empirical evaluation does not convincingly demonstrate the claimed benefits. More theoretical and experimental rigor is needed before this work would be suitable.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "*Strengths*:\n\n* The paper addresses a relevant challenge in MARL—context length selection—which can impact both computational efficiency and performance.\n* The method introduces a novel combination of low-frequency truncation via Fourier Transform and adaptive policy mechanisms.\n* The paper is well-organized and includes ablation studies and comparisons to fixed-length baselines.\n\n*Weaknesses*:\n\n* The motivation for using low-frequency truncation to extract long-range temporal dependencies is conceptually unclear and methodologically unjustified. In fact, low-frequency truncation typically removes low-frequency components, which encode global (long-term) trends, which appears counter-intuitive if the goal is to *preserve* long-term dependencies.\n* The theoretical section lacks meaningful insight and fails to provide a compelling justification for why this method would outperform fixed context models in environments with long-term dependencies.\n* The environments used in experiments (e.g., PettingZoo Sample Spread, MiniGrid Soccer, GRF scenarios) are not particularly representative of settings with significant long-range temporal dependencies. Therefore, experimental validation of the method’s core claim remains unconvincing.\n* The method is not compared against strong and relevant MARL baselines (e.g., QMIX, QPLEX, MAPPO), making it difficult to assess its true performance advantage.\n* Despite the technical depth in the mathematical formalism, the link between the theoretical analysis and the practical performance of the algorithm remains weak and mostly qualitative.", "was_revised": true, "final_justification": "The authors’ rebuttal has satisfactorily resolved my earlier concerns; I will therefore raise my score.", "rebuttal": "We thank the reviewer for the thoughtful comments.\n\n**Weakness1 &  Question1 & Question2 & Limitations 1 & Limitations 2:**  \n1.The motivation for using low-frequency truncation to extract long-range temporal dependencies is conceptually unclear and methodologically unjustified. In fact, low-frequency truncation typically removes low-frequency components, which encode global (long-term) trends, which appears counter-intuitive if the goal is to preserve long-term dependencies.  \n2.Why is the low-frequency truncation method removing low frequencies instead of high frequencies, if the goal is to retain global temporal information?  \n3.Can the authors provide any empirical analysis (e.g., frequency spectrum plots) that demonstrates the benefit of the truncation in capturing relevant dynamics?  \n4.The paper’s main methodological contribution—Fourier-based low-frequency truncation—is not clearly grounded in MARL-specific insights, and its theoretical support is weak.  \n5.The proposed approach assumes that removing low-frequency components enhances representation, which contradicts the usual interpretation in signal processing and undermines the core justification.\n\n**Answer:**  \nWe think you may have misunderstood the core idea of low-frequency truncation as used in our work. The term refers to retaining low-frequency components while removing high-frequency ones—not the reverse. This terminology is consistent with [1] page 393, where low-frequency truncation is introduced in the context of dyadic partition of unity to preserve large-scale structures.  Furthermore, while [1] applies this concept to continuous function spaces, we extend it to discrete temporal sequences and demonstrate its theoretical soundness in our setting, as detailed in Appendix A. The misunderstanding may due to an unclear explanation in the main text. We will include an explicit definition of low-frequency truncation in the revision to avoid further confusion.\nIn MARL, the challenges of input representation and increasing computation remain fundamental issues, as discussed in prior work [2]. To address this, we explore the use of Fourier-based low-frequency truncation and design a central agent, which offers two key advantages: (1) low-frequency components in the spectral domain tend to capture long-term patterns while discarding short-term noise, thus mitigating the complexity introduced by high-dimensional historical observations; (2) the degree of truncation provides a natural mechanism to control the granularity of long-term trends, which aligns well with the discrete action space of rl agents. Building on these, we introduce a central agent that adaptively selects the optimal level of low-frequency information, ensuring both efficiency and sufficiency in capturing global temporal dependencies.\n\n**Weakness2 & Weakness5:**  \n1.The theoretical section lacks meaningful insight and fails to provide a compelling justification for why this method would outperform fixed context models in environments with long-term dependencies.  \n2.Despite the technical depth in the mathematical formalism, the link between the theoretical analysis and the practical performance of the algorithm remains weak and mostly qualitative.  \n\n**Answer:**  \nIn this paper, Theorem 1 formally characterizes how adaptive context length reduces the loss of relevant historical information—measured via mutual information—compared to any fixed-length method. This guarantees that over time, the adaptive method preserves more useful history, leading to better long-term performance in environments with long-term dependencies. Though this advantage is hard to measure directly, section 4.3 renders the performance comparison between the overall method and fixed-length to quantify this advantage. To more clearly compare the performance difference between fixed and adaptive context lengths while excluding the effect of Fourier low-frequency truncation, we re-extracted and reorganized the data into the table below:  \n|GRF (win rate)|4 step|8 step|16 step|32 step|Adaptive step|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|3 vs 1 with Keeper|59.4±4.5|65.4±3.8|71.9±4.3|63.5±3.5|84.2±2.6|\n|Counterattack-Hard|60.3±5.1|71.5±4.4|62.1±3.9|53.3±4.7|82.7±2.9|\n\n**Weakness3 & Weakness4 & Question 4 & Limitations3 & Limitations4:**  \n1.The environments used in experiments (e.g., PettingZoo Sample Spread, MiniGrid Soccer, GRF scenarios) are not particularly representative of settings with significant long-range temporal dependencies. Therefore, experimental validation of the method’s core claim remains unconvincing.  \n2.The method is not compared against strong and relevant MARL baselines (e.g., QMIX, QPLEX, MAPPO), making it difficult to assess its true performance advantage.  \n3.Why are standard MARL baselines (e.g., QMIX, MAPPO) omitted from experimental comparisons?\n4.The experimental setup lacks tasks that truly stress long-range temporal dependencies, weakening the claimed benefits of adaptive context length.\n5.Important baselines from the MARL literature are omitted, casting doubt on the significance of the reported performance gains.\n\n**Answer:**  \nWe would like to clarify that the chosen tasks may already involve non-Markovian characteristics, as they require long-term path planning and delayed coordination—especially in the GRF. To further validate our method in more complex and representative non-Markovian settings, we have additionally conducted extensive experiments on the challenging SMACv2 benchmark [3], as suggested. Moreover, there may be a misunderstanding: the central agent and Transformer modules only process historical information and are not involved in decision-making. The output is passed as an extra feature to distributed agents (which are identical across our method and all baselines), which make decisions based on their current state and this historical representation. In our original implementation, distributed agents used MAPPO, while the central agent uses PPO as a single-agent module. To rule out MAPPO-specific gains, we also conducted experiments with QMIX and QPLEX as backbones, while the central agent uses DQN.\n|SMACv2 (win rate)|RL-Method|Transformer|ToST|AMAGO|ACL-LFT|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|3s5z vs 3s6z|MAPPO|71.5±3.9|72.2±3.4|76.1±2.9|78.9±2.8|\n| |QMIX|72.8± 3.7|73.7±3.6|76.3±3.1|79.4±2.9|\n| |QPLEX|73.6±3.3|75.1±3.6|77.5±2.8|80.1±3.0|\n|5m_vs_6m|MAPPO|44.5±4.9|46.3±4.4|48.1±4.0|52.7±4.2|\n| |QMIX|44.3±5.5|46.9±4.8|48.3±4.3|52.4±4.6|   \n| |QPLEX|45.6 ± 5.7|47.3±4.6|49.5±4.8|53.9±4.3|\n|corridor|MAPPO|65.6± 5.7|68.1±6.6|74.3±4.8|77.9±5.3|\n| |QMIX| 68.5±5.9|70.3±6.8|75.0±5.3|78.6±5.4|\n| |QPLEX|70.6±4.7|72.9±4.6|76.3±5.8|79.2±4.9|\n\n**Question 3:**  \nHow does the proposed method generalize to MARL environments with partial observability or non-stationarity, where context length adaptation may be more crucial?\n\n**Answer:**  \nAs you rightly pointed out, in complex non-Markovian MARL environments with long-term dependencies, non-stationary behavior patterns can severely hinder decision-making. Our method is specifically motivated by these challenges. Existing methods rely on fixed-length historical inputs, which may introduce substantial noise and increasing computational overhead. To address this, we introduce a central agent that adaptively extracts minimal but sufficient global contextual information, enabling agents to effectively make robust decisions under non-stationary. Moreover, although most real-world and benchmark environments are partially observable, global historical states are often accessible after the fact. This allows to reconstruct a fuller picture of the environment’s temporal evolution in next time step.  \nIn fully decentralized settings where global history is unavailable, our framework can be extended by letting the central agent process each agent’s local historical information independently to generate personalized context summaries. This variant preserves the core idea of the proposed method, while remaining fully compatible with partial observability constraints.\n\n**Question 5:**  \nCould the authors clarify how the central agent is trained online without inducing instability in the decentralized policies?\n\n**Answer:**  \nFirst, the optimization directions of the central and decentralized agents are largely aligned. Formally, the gradient of the central agent’s objective correlates positively with the combined gradient of all decentralized agents’ objectives, i.e.,\n\\\\[\\\\nabla_{\\\\pi} J_c(\\\\pi) = \\\\sum_{t=0}^{\\\\infty} \\\\gamma^{t} \\\\sum_{i=1}^{n} \\\\omega_{t}^{i} \\\\nabla_{\\\\pi} \\\\mathbb{E}[R_{i} \\\\mid \\\\pi]\\\\]\n\\\\[\\\\nabla_{\\\\pi} \\\\left( \\\\sum_{j=1}^{n} J_j(\\\\pi) \\\\right) = \\\\sum_{t=0}^{\\\\infty} \\\\gamma^{t} \\\\sum_{j=1}^{n} \\\\nabla_{\\\\pi} \\\\mathbb{E}[R_j \\\\mid \\\\pi]\\\\]\n\\\\[\\\\left\\\\langle \\\\nabla_{\\\\pi} J_c(\\\\pi), \\\\nabla_{\\\\pi} \\\\sum_{j=1}^{n} J_j(\\\\pi) \\\\right\\\\rangle > 0 \\\\quad \\\\text{when} \\\\omega_{t}^{i} > 0\\\\]\n Second, the training of the central agent and decentralized agents proceeds independently and sequentially within each iteration: the central agent updates first to provide a stable context representation, followed by decentralized agents updating their policies using the updated context. This staged training further mitigates instability by decoupling gradient updates and allowing progressive convergence.\n\n[1] Bahouri, H. (2011). Fourier analysis and nonlinear partial differential equations. Springer.\n[2] Riemer, M., Khetarpal, K., Rajendran, J., & Chandar, S. (2024). Balancing context length and mixing times for reinforcement learning at scale. Advances in Neural Information Processing Systems, 37, 80268-80302.\n[3] Ellis, B., Cook, J., Moalla, S., Samvelyan, M., Sun, M., Mahajan, A., ... & Whiteson, S. (2023). Smacv2: An improved benchmark for cooperative multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 36, 37567-37593."}]

Author rebuttal:

Meta-review: This paper proposes adaptive context length optimisation with low-frequency truncation for MARL to handle long-range non-Markovian problems.

All the reviewers are positive about this paper. The only reviewer with a negative score also increased their score during the rebuttal. There were concerns about the scale of the environments, but the authors provided results on complex SMACv2 during the rebuttal. The authors properly addressed all the concerns, which resulted in an increase in scores by all reviewers.

As one of the reviewers mentioned, it is worth publishing this work because it pushes the state-of-the-art in long context MARL.
Decision: Accept (poster)