Kati8/ICLR_2023_2025 huggingface dataset:
authors: Chengqi Zheng
authors: Jianda Chen
authors: Wen zheng terence Ng
authors: Ivor Tsang
authors: Haiyan Yin
reviews: 
Comments: This work introduces SSDE, a novel structure-based continual RL method. SSDE formulates an efficient co-allocation algorithm that enhances sub-network allocation by increasing the capacity for trainable parameters while leveraging frozen parameters for effective forward transfer from previous policies. SSDE introduces a trade-off parameter to balance these two groups of parameters in its fine-grained inference process.
Strengths: (1)SSDE not only achieves SOTA stability standards but also achieves competitive plasticity even when compared to strong behavior cloning baselines that benefit from data replay.
(2)Experimental results demonstrate that SSDE outperforms current state-ofthe-art methods and achieves a superior success rate of 95% on the CW10-v1.
Weaknesses: As shown in Table 3, SSDE takes no obvious advantages in F and FT metrics. These two metrics usually represent backward and forward transfer. So why Average Performance (P) is so good while F and FT not? I am a little confused. Also, ClonEx-SAC seems to perform comparably with SSDE on CW 20, although it replays data.
Recommendation: 5
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: This paper proposes a structure-based method, SSDE, to enhance the plasticity-stability trade-off in continual reinforcement learning. It introduces a fine-grained allocation strategy that decomposes network parameters into fixed forward-transfer and task-specific trainable components. Furthermore, to improve the model’s exploration capability, this paper presents an exploration technique based on sensitivity-guided dormant neurons. Experiments conducted on the Continual World benchmark demonstrate that the proposed method achieves a superior success rate and outperforms current state-of-the-art methods in the CW10 task.
Strengths: This paper addresses a critical issue in continual reinforcement learning: balancing plasticity and stability to mitigate catastrophic forgetting. The proposed method offers greater computational efficiency than existing approaches. Experimental results are promising, demonstrating that the proposed method achieves a higher success rate and outperforms other baseline methods on the CW10-v1 Continual World benchmark.
Weaknesses: 1. The paper can be improved in terms of writing/presentation.
2. From Table 3, it is evident that the author’s proposed method generally performs weaker than the ClonEx-SAC method. Therefore, will the ClonEx-SAC method continue to outperform the author’s method as the number of sequential tasks increases? Additionally, why doesn’t the forward transfer ability, or generalization ability, of the author’s method improve as the number of tasks increases?
3. The proposed sensitivity-guided dormant neurons offer limited novelty.
4. The author does not include comparisons with the latest regularization-based methods in continuous reinforcement learning, and the multi-task methods referenced are also outdated.
Recommendation: 5
Reviewer confidence: 4
Soundness/correctness: 2
Clarity: 2