reviews: 
Comments: In this paper, the authors are exploring the use of visual language models (VLMs) to play action-role playing video games (ARPGs), specifically using one AAA title as a case study. They propose:
1) a VLM-based agent framework called VARP (Vision Action Role-Playing) which takes game screenshots as inputs and one of the 13 pre-defined tasks and outputs game actions (defined as combinations of atomic game actions)
2) a human-gameplay dataset with 1000 records, collected in-house with 200 mostly novice human players
3) an evaluation task set with 13 tasks specific to a AAA ARPG game Black Myth: Wukong (BMW), of different levels of difficulty: 9 easy, 1 medium, 1 hard and 3 very hard.
Strengths: It is great to see the complexities of a AAA title being described and addressed, helping the research community make progress beyond simpler 2D environments. The authors do a good job of explaining their data collection process, mentioning the number of participants, as well as the compute resources utilised for the experiments. They also present results on ablating some of the key optimizations they added to the VARP framework to aid action selection. 

The work is original in the sense that it addresses the use of VLMs for ARPGs, providing a framework which is able to update the action library, as opposed to using a static one, as highlighted in the comparison study against the closest baseline, Cradle. The authors are also open to share publicly the code and the datasets to contribute to the community and encourage reproducibility and extension upon their findings.
Weaknesses: In evaluating VARP, authors highlight 2 limitations of using VLMs for action video games, such as their slow reasoning speed for the more difficult tasks, as well as the challenges they face when tasked with long horizon navigation queries. Given the premise in the title, I would have expected a more detailed critical analysis of the strengths and limitations of using VLMs used as agents in ARPGs.

Another opportunity to expand and strengthen the submission would be to present a clearer discussion on the value of the additional modules that distinguish it from the Cradle framework. Sections 4.5 and 4.6 are a great step in this direction, but I feel they could be further strengthened by including the very hard tasks. First by mentioning this in the context of related work, and secondly by considering the very hard tasks in the comparison with Cradle and the ablation study. 

Even though the title does mention a case study on a specific game title, it would add a lot more strength to the submission to include a second game environment. One option would be for the authors to consider rephrasing the title. Having the initial question formulated as "Can VLMs play ARPGs?" and show insights from only one title, on a series of limited tasks, makes it harder to claim that the question is being comprehensively addressed.

It would be good to see a more detailed discussion on how the choice of VLM driving VARP makes a difference on the overall success rates. Along similar lines, in order to best support the community, it would be good to see stronger reasoning supporting the authors’ initial choice of VLM models - why were GPT-4o, Claude and Gemini selected in experiments, and not others? What makes them suitable for this type of tasks?
Recommendation: 5
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 2