Title: Mixing Expert Knowledge: Bring Human Thoughts Back To the Game of Go
Venue: NeurIPS 2025
Accepted: yes
Keywords: LLM reasoning

Reinforcement Learning

Go
Abstract: Large language models (LLMs) have demonstrated exceptional performance in reasoning tasks such as mathematics and coding, matching or surpassing human capabilities. However, these impressive reasoning abilities face significant challenges in specialized domains. Taking Go as an example, although AlphaGo has established the high performance ceiling of AI systems in Go, mainstream LLMs still struggle to reach even beginner-level proficiency, let alone perform natural language reasoning. This performance gap between general-purpose LLMs and domain experts is significantly limiting the application of LLMs on a wider range of domain-specific tasks. In this work, we aim to bridge the divide between LLMs' general reasoning capabilities and expert knowledge in domain-specific tasks. We perform mixed fine-tuning with structured Go expertise and general long Chain-of-Thought (CoT) reasoning data as a cold start, followed by reinforcement learning to integrate expert knowledge in Go with general reasoning capabilities. Through this methodology, we present LoGos, a powerful LLM that not only maintains outstanding general reasoning abilities, but also conducts Go gameplay in natural language, demonstrating effective strategic  reasoning and accurate next-move prediction. LoGos achieves performance comparable to human professional players, substantially surpassing all existing LLMs. Through this work, we aim to contribute insights on applying general LLM reasoning capabilities to specialized domains. We will release the first large-scale Go dataset for LLM training, the first LLM Go evaluation benchmark, and the first general LLM that reaches human expert-level performance in Go.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents LoGos, a large language model trained to achieve professional-level Go performance while maintaining general reasoning capabilities. The authors use a two-stage approach: (1) mixed fine-tuning with structured Go expertise data and general Chain-of-Thought reasoning data, followed by (2) reinforcement learning using GRPO. The model reportedly achieves 88.1% and 88.6%  accuracy on their KataGo-Bench-1K benchmark and wins against various Go AI models.\n\nSTRENGTHS AND WEAKNESSES:\nStrength: \n- Novel and ambitious application: Attempting to bridge domain expertise with general LLM capabilities is an interesting research direction\n- Comprehensive experimental design: The paper includes thorough ablation studies, multiple baselines, and evaluation across both Go-specific and general reasoning tasks\n- Transparent methodology: Detailed description of dataset construction, training procedures, and reward function design\n- Strong empirical claims: If validated, achieving professional-level Go performance while maintaining general capabilities would be significant\n\nLimitation:\n1. Fundamental Architectural Mismatch\nThe core premise is questionable: using sequential text encoding (\"1.X-Q16 2.O-D16...\") for a inherently spatial, visual game like Go. LLMs processing 150+ move sequences cannot realistically maintain spatial board understanding. This encoding forces the model to reconstruct 2D spatial relationships from 1D text, which is fundamentally inefficient.\n\n2. Insufficient Justification for Approach\nThe paper fails to adequately address why this text-based LLM approach is preferable to existing methods. AlphaGo and successors use direct board representations for good reason. The authors should justify why forcing spatial reasoning through text is beneficial.\n\n3. Questionable Performance Claims\n- The dramatic improvement from 60-70% to 88%+ through RL seems implausibly large\n- Win rates against specialized Go AIs need independent verification\n- The KataGo-Bench-1K benchmark is self-constructed and may not reflect true playing strength\n- Human evaluation shows only 55.6% correct explanations, suggesting pattern matching rather than understanding\n\n4. Limited Understanding Analysis\n- The paper lacks analysis of what the model actually \"understands\" about Go. The context curse analysis (Figure 8) shows severe performance degradation as games lengthen, suggesting the model doesn't maintain coherent board state understanding.\n\n5. Evaluation Concerns\n- Evaluation primarily on authors' own benchmark\n- Limited diversity in opponent strength assessment\n- No analysis of actual game quality or strategic understanding\n- Self-play evaluation would strengthen claims\n\nQUESTIONS:\nAs a Go game player, I genuinely like the topic of this paper. \n\nI have one major question: Can you provide game records and independent evaluation against established Go engines at known strength levels? The reason I'm asking is that I tried to quickly reproduce the strategies of using text encoding with commercial LLMs (Claude, OpenAI, etc.), yet I do not really observe consistent responses from LLMs. In particular, when I provide the sequential text encoding steps of Go games to LLMs, I do not see reasonable plays from the responses. The model more likely just hallucinates and generates a random response. I don't know if this is because of the issues of not fine-tuned on Go-related materisl, but I would like to see more evidence that the approaches proposed by the authors work. Therefore, it would be great if the authors could open-source the dataset, or at least provide some samples. \n\n1. Core Justification: Why use text encoding for Go when spatial representations are more natural? What advantages does this approach offer over existing methods?\n2. Understanding vs. Pattern Matching: How do you distinguish between the model learning Go principles versus memorizing patterns from training data?\n3. Conversation Logs: Can you share actual conversation histories showing the model's reasoning process during gameplay?\nSpatial Reasoning: How does the model handle complex spatial relationships (life/death, territorial evaluation) through text sequences?\n4. Generalization: How does performance vary across different board sizes or rule variants not seen in training?\n\nLIMITATIONS:\n- The text encoding approach may fundamentally limit spatial reasoning capabilities\n- Training methodology may overfit to evaluation metrics rather than genuine understanding\n- Limited analysis of failure modes and edge cases\n- Dependence on KataGo annotations may constrain learning to that engine's biases\n- Computational efficiency compared to specialized Go AIs is unclear", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces LoGos, a large language model (LLM) that is specialized in playing the game of Go. The model is trained on a novel dataset that has been constructed based on real, annotated game states and Go commentary. A novel training paradigm, using the aforementioned dataset and chain-of-thought (CoT) reasoning data from a variety of domains for fine-tuning, is applied. Reinforcement learning is applied to improve the model’s next-move prediction performance, using a reward function that encourages the model to output the correct next move, given a game state, alongside the win rate estimation for the predicted move. LoGos outperforms other LLMs and a competitive AI system, KataGo, in accuracy on predicting the next move given a game state, while also maintaining general capabilities on math, reasoning, and coding benchmarks. According to human evaluators, LoGos produces accurate move predictions 96.5% of the time, but the explanations given in natural language are correct only 55.6% of the time.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nIntegrating natural language explanations into Go-playing AI systems could be very a valuable entertainment and educational resource for the Go-playing community. The proposed model achieves admirable performance on a benchmark constructed to measure the accuracy of next-step prediction, given a game state. This paper represents a promising direction in Go AI research. \n\nWeaknesses:\nPerformance concerns: The following is acknowledged in the paper, but remains a considerable concern for the utility of the proposed model. Over the course of a game of Go, the context contained in the LLM input becomes increasingly long, and performance degrades. Additionally, the natural language explanations, which could be a core highlight of the proposed system, have accuracy issues. \n\nExposition: The description of the game of Go in Section 2.1 could be improved. There is not explicit information about the number of players, nor what constitutes a winning state for the game. It may also be helpful to explain basic mechanics of the game, like “capture”. The paper is not very accessible to people who don’t already know how to play Go. \n\nReproducibility concerns: There is insufficient information about the source of the Go commentary data. It would be helpful to have information about the properties of the data, e.g. average length of comments, language(s) included, and whether the commentary is automatically transcribed from video/audio data or inherently text-based. This data is crucial to the approach presented, so it seems important for the audience to be able to assess its provenance and quality. Additionally, there are potential ethical issues: even though the data is ostensibly ‘public,’ do the authors have permission to collect and redistribute it?\n\nQUESTIONS:\n- Do the authors have suggestions for future work that could address the ‘context curse’ issue? \n- The proposed benchmark includes a measure of accuracy for next-move prediction, given game state, but is there a way to assess LoGos’ overall win rate in complete games? \n- Can the authors provide a reference for: \"Go contains more than 600 terms\" (p. 8)?\n\nLIMITATIONS:\nThe aforementioned concerns regarding the data sourcing should be addressed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies how to incorporate domain-specific knowledge in Go into a large language model while preserving general reasoning capabilities. The authors collected a large-scale next-move prediction Go dataset, formatted with a heuristically generated template, and a moderate-scale commentary is also processed to be part of the training data (give commentary with terminologies in natural language). For training general-purpose LLMs to equip Go knowledge, the authors further propose a two-phrase pipeline (also widely seen in recent RL for LLM reasoning papers) including a mixed cold start fine-tuning and reinforcement learning (GRPO) with KataGo annotations as reward signals.\n\nComprehensive experiments were made to evaluate both the Go-specific performance and general reasoning competence of the two-phrase trained models. Trained models (with Qwen as base) can attain professional-level Go capabilities and preserve general mathematical reasoning performance (for base models, it can even boost mathematical reasoning).\n\nSTRENGTHS AND WEAKNESSES:\n#### Strengths\n- The paper is well-written and enjoyable to read. In general, it is interesting to see how general-purpose large language models can be turned into professional-level Go models with some curated data. It's an interesting topic and I think Go itself is a good domain for testing these RL bootstrap reasoning methods.\n\n- The collected next-move prediction + commentary dataset is good, and I acknowledge the authors' effort made on this. It would be a good resource for the community to have easy access to Go-specific knowledge in a natural language format.\n\n- Extensive experiments show the effectiveness of the proposed method. It's also good to see how general reasoning performance is preserved. It is also a bit surprising to see that being trained on Go dataset can lead to improvements on mathematical reasoning tasks (even outperforming classic instruction fine-tuning in some cases). \n\n#### Weaknesses\n- Limited evaluations. For evaluating Go-specific performance, this paper uses the self-proposed KataGo-Bench. Although there is a correlation analysis between Elo rating and KataGo-Bench, I'm still curious about the Elo score of the trained models. In my memory (correct me if I'm wrong), GPT-3.5/4 already has an Elo score of 1700+ (which is about 85 on KataGo-Bench in Figure 9), But why did closed-source models in Table 1 perform so poorly? \n\n- Limited methodological contributions. It is not surprising that cold-start + GRPO can be applied to LLMs to boost domain-specific reasoning capabilities like Go. I see the main contribution of this paper as the KataGo-Bench and the Expert-level go dataset. The two-phase RL-style training is more like a baseline evaluation of the effectiveness of the proposed dataset rather than a major contribution itself.\n\nQUESTIONS:\nWhat are the Elo scores of the models? What is the performance of Mixed Cold Start (without RL)?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method for integrating domain-specific expert knowledge into general-purpose LLMs, using the game of Go as a case study. The authors introduce LoGos, a general LLM that achieves professional-level Go play while maintaining strong performance on general reasoning tasks like math and code. The key idea involves a two-phase process: (1) mixed cold start fine-tuning using a combination of domain-specific (Go) structured data and general CoT data; and (2) reinforcement learning via GRPO using a custom reward function designed around move quality and win rate estimation. The authors also construct a large-scale Go dataset and a new evaluation benchmark (KataGo-Bench-1K), releasing both to the community. Experimental results show that LoGos surpasses all existing general LLMs in Go performance and matches or exceeds strong amateur and professional Go models, while still performing competitively on standard general benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The paper tackles an interesting challenge - how to extend LLMs to perform effectively in domains with limited natural language corpora but rich structured expert data. Go serves as a compelling testbed with clear evaluation metrics.\n\n2. The methodology is sound and rigorously evaluated. Dataset construction and training details are clearly documented.\n\n**Weaknesses:**\n\n1. While the mixed fine-tuning with structured Go expertise and general long Chain-of-Thought (CoT) reasoning data as a cold start, followed by reinforcement learning is well-executed, the overall training strategy closely resembles existing practices in LLMs.\n\n2. The authors claim that the method can generalize to other specialized domains, yet the paper lacks concrete examples or preliminary results beyond the Go domain. While the approach is demonstrated effectively on Go, the feasibility of applying this method to other specialized domains remains speculative. The reliance on structured expert data (e.g., KataGo) and hand-crafted heuristic rules may limit scalability to less formalized domains.\n\nQUESTIONS:\n1. Could the authors clarify what specific aspects of the training pipeline are novel? How does this approach differ from existing fine-tuning, RLHF or domain-alignment methods?\n\n2. The paper posits that the proposed method can be extended to other specialized domains. Could the authors discuss concrete application areas where similar structured expert knowledge and heuristic rules could be constructed at scale? Are there any early results or experiments in these domains? \n\n3. Does the Go domain offer any general insights that can inform applications in other fields?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-08"}