Kati8/ICLR_2023_2025 huggingface dataset:
authors: Sicong Liu
reviews: 
Comments: This paper proposes an algorithm for multi-task multi-agent skill discovery from offline data, which is a quite meaningful and challenging research direction. The novelty (compared with ODIS) mainly lies in jointly learning common and task-specific skills.
Strengths: (a) This paper targets at a challenging problem setup, i.e., recovering generalizable multi-agent skills from multi-task offline data.

(b) The evaluation results are sufficient and show consistent improvements over the SOTA algorithm -- ODIS.
Weaknesses: (a) The algorithm framework, including both the multiple components to learn (Fig. 1) and the objective design (Eqs. 4, 7, 11), is quite convoluted compared to ODIS, which could potentially bring training difficulty.

(b) The writing needs to be significantly improved, especially for methodology part in Section 3. 

(c) It's not clear why the common skills can integrate cooperative temporal knowledge and enable an offline exploration.

(d) Since all tasks share the same space for the local skills z^{1:K}_t, why z can embed task-specific knowledge?

(e) Multi-agent skills should provide abstractions at both temporal-level and subgroup-level. That is, each multi-agent skill should embed a (short) control sequence of a subgroup of agents. How is this aspect shown in the algorithm design?

(f) Thanks for releasing the code package. But it seems that the authors only made minimal changes to the readme file provided by ODIS. Please provide a detailed instruction (in the readme file) on how to reproduce all the reported results in this paper. Also, the released yaml files do not include the ones for MAMuJoCo.
Recommendation: 6
Reviewer confidence: 3
Soundness/correctness: 2
Clarity: 2
reviews: 
Comments: This paper introduces Hierarchical and Separate Skill Discovering (HiSSD), a method for learning generalizable skills in offline multi-task multi-agent reinforcement learning. HiSSD's hierarchical framework simultaneously learns common skills (cooperative behaviors across tasks) and task-specific skills (unique task characteristics). It uses an objective function combining reward maximization and next-state prediction for common skills, and contrastive learning for task-specific skills. HiSSD trains on offline data and enables decentralized execution with local information during testing.
Strengths: This paper introduces Hierarchical and Separate Skill Discovering (HiSSD), a method for learning generalizable skills in offline multi-task multi-agent reinforcement learning. HiSSD's key strength is its hierarchical framework that simultaneously learns common and task-specific skills, overcoming existing limitations. It enables fine-grained action execution and trains using only offline data. During testing, HiSSD allows decentralized execution using local information and learned skills without centralized control. Experiments on SMAC and MAMuJoCo benchmarks showed HiSSD's performance improvements, especially in unseen tasks.
Weaknesses: While the paper claims to integrate cooperative temporal knowledge into the process of learning common skills, it fails to adequately explain the clear causal relationship or mechanism between these two concepts. There appears to be a logical leap in assuming that simply performing global state prediction and value estimation directly leads to the learning of cooperative knowledge. Furthermore, the paper lacks a clear definition of what exactly "cooperative temporal knowledge" means and how it is measured and evaluated. This ambiguity makes it difficult to verify whether this knowledge has been learned.
Recommendation: 6
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: This paper proposes a hierarchical framework, HiSSD, for jointly learning common and task-specific skills from offline multi-task datasets. The common skills represent the general cooperation patterns, while task-specific skills achieve a task-guided fine-grained action execution. The algorithm is broken up into two phases. In the first phase, the high-level planner discovers common skills. In the second stage, the low-level controller learns task-specific skills.
Strengths: 1. The paper is well written, with thorough experiments, and a variety of tasks across two environments.
2. In order to achieve better performances in cooperative multi-agent environments, the paper brings cooperative temporal knowledge into common skills, which is reasonable.
Weaknesses: 1. According to the paper, HiSSD is superior to previous skill-based methods like ODIS because it brings cooperative temporal knowledge into common skills and discovers task-specific skills that can distinguish each task’s specific knowledge. However, HiSSD fails to outperform ODIS in some tasks.
2. In MAMuJoCo, the paper didn't use ODIS as a baseline, which has a good performance in SMAC.
3. The process of learning skills is similar to imitation learning, so it failed to perform well in unseen tasks, indicating low-level generalization ability. For example, in Table 1, the win rates for 10m12m and 13m15m are lower than 10% with the medium-quality dataset.
4. In the passage, some verbs sound a bit odd. For example, 'contain' in the sentence 'Equipping offline multi-task MARL with skill learning to improve policy transfer still contains issues.'
5. One concern for the paper is to evaluate HiSSD in maps that are more difficult (like the Stalker-Zealot task set in ODIS), to validate its effectiveness.
Recommendation: 6
Reviewer confidence: 3
Soundness/correctness: 4
Clarity: 4