reviews: 
Comments: This paper addresses a fundamental challenge in multi-agent reinforcement learning (RL): the difficulty of achieving robust performance across environments beyond those the agents were specifically trained on. To tackle these issues, the authors propose and evaluate their approach in a well-established environment for multi-agent RL, Hanabi, which is commonly used to test cooperation in complex multi-agent settings. The paper makes two primary contributions. First, it reformulates the task using textual representations, motivated by evidence that language can improve transferability. By representing the Hanabi game through text, the authors aim to create agents that are more adaptable and capable of generalizing beyond their training environment. Second, they introduce a novel neural network architecture that combines a language model with the Deep Recurrent Relevance Q-network (DRRN). The results presented indicate that the proposed method achieves high intra-algorithm cross-play scores when trained in a self-play setting, underscoring its capacity for adaptability and teamwork. Additionally, the method performs exceptionally well in inter-algorithm play, highlighting its capability to cooperate with agents utilizing different algorithms.
Strengths: 1. One of the primary strengths of this work is its success in achieving effective transfer learning in a zero-shot setting. The proposed method shows adaptability across varying scenarios without requiring additional retraining, which addresses a significant limitation in existing multi-agent RL models.

2. Through careful evaluation, the authors demonstrate that their approach outperforms existing methods in inter-algorithm play, achieving high scores in cross-play with agents using different algorithms.

3. By representing the Hanabi game through text, this work leverages advances in natural language processing (NLP) and large language models (LLMs), marking a novel direction in multi-agent RL. Their approach allows the potential for more seamless integration with future advancements in language models.
Weaknesses: 1. The reliance on text-based representation, while novel, may face significant challenges in more complex or non-linguistic domains. For instance, domains where numeric data dominates—such as algorithmic trading, where key data points include stock prices and volumes, or robot-taxi applications, where visual and sensory data are crucial—may not naturally fit into a textual framework. An extension or discussion on the applicability limits of textual representation in various domains would provide valuable context and guide future research on adapting the method to broader applications.

2. The self-play performance is relatively poor. Insights into possible contributing factors, such as model structure or training limitations, would be useful. Additionally, outlining potential directions for improving self-play performance, whether through modifications to the architecture, training process, or environment setup, would provide actionable guidance for future work.

3. There is insufficient discussion on the choice of neural network architecture, particularly regarding the component where the element-wise dot product is applied between the embedded observations and actions. Similar ideas have been explored in the literature, such as in https://proceedings.mlr.press/v202/ma23e/ma23e.pdf. The authors may consider incorporating insights from existing explanations in the literature to better articulate the advantages and underlying mechanisms of this architectural choice.
Recommendation: 6
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 3