Kati8/ICLR_2023_2025 huggingface dataset:
authors: Timo Bertram
authors: Johannes Fürnkranz
authors: Martin Müller
reviews: 
Comments: This paper provides some experimental results on evaluating the information set. The motivation is unclear and I cannot foresee any possible applications. The presentation is poor and I barely understand the point the authors want to argue.
Strengths: None.
Weaknesses: * The unclear motivation: The authors argue that the motivation comes from evaluating the game tree of the imperfect information game. However, for evaluating imperfect information game, one should also take opponents' policy into account and this paper completely ignore this. Hence, I don't think this paper can have any help for imperfect information games. Learning with noisy labels can be an alternative understanding but I don't think the authors show some rigorous consideration on that.
* The poor presentation: I believe statements in Section 2 have no significance -- they are just standard Monte Carlo approximations. Until I go through the experiment I realize the authors may want to argue that how to generate samples that can help the neural network training. I don't think the authors make this clear in previous sections and there should be much more discussion on this before experiments.
* Limited significance: For me I think this is just an experimental report and I cannot get any meaningful conclusion from this paper. The authors make no discussion on the reason to use these methods and what are the potential benefits/drawbacks using these methods in a rigorous manner. I also believe that the proposed problem is a delicate learning theory problem and at least there are lots of work on learning with noisy labels that propose different algorithms with theoretical justifications to achieve strong empirical performances. This paper instead shows no effort on understanding the problem and improving the performance upon existing baselines.
Recommendation: 1: strong reject
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 1 poor
Clarity: 1 poor
reviews: 
Comments: This paper investigates the impact of sampling of an information set on the approximation of the  corresponding information set value in imperfect-information games. Specifically, it studies how many states from a given information set should be sampled, and how many times should a selected state be sampled. Different experiments are performed, and some observations are obtained.
Strengths: + Partial observability is a practical issue in real-world decision making problems, and this paper tackles this issue from a perspective based on information set.
Weaknesses: - This paper is not scientifically sound. It does not provide a concrete mathematical problem formulation, and it does not provide a principled approach to address a well defined problem either. The main contribution of this work is some empirical observations, which does not provide much insight or guideline on how to design an efficient sampling strategy to mitigate the partial observability in decision making problems.
Recommendation: 1: strong reject
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 1 poor
Clarity: 1 poor
reviews: 
Comments: This work presents experimental findings regarding the assessment of information sets, yet the underlying rationale remains vague and practical utility is not evident. The manuscript suffers from inadequate clarity in exposition, making it difficult to discern the central thesis that the authors intend to communicate.
Weaknesses: * Vague theoretical foundation: While the authors reference evaluation of game trees in imperfect information settings as motivation, this framing overlooks the essential role of opponent strategies, which are conspicuously absent from the analysis. This omission undermines claims of relevance to imperfect information games. Though learning from corrupted annotations might offer an alternative perspective, the authors do not demonstrate sufficiently rigorous treatment of this angle.
* Inadequate exposition quality: The content presented in Section 2 consists primarily of conventional Monte Carlo sampling techniques without substantive contribution. Only upon reaching the experimental section does one infer that the authors may be addressing sample generation strategies for neural network optimization. This critical point is insufficiently articulated in the preceding discussion and warrants considerably deeper treatment prior to empirical validation.
* Insufficient depth and contribution: This manuscript reads predominantly as an empirical study lacking coherent takeaways. The authors provide no systematic analysis of methodological choices or rigorous evaluation of advantages and disadvantages across proposed approaches. Given that this constitutes a nuanced statistical learning challenge with extensive prior literature on learning from corrupted data offering theoretically-grounded algorithmic solutions and strong results, this submission demonstrates minimal engagement with existing knowledge or advancement beyond established baselines.
Recommendation: 1: strong reject
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 1 poor
Clarity: 1 poor