KrwEmd: Revising the Imperfect Recall Abstraction from Forgetting Everything

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 17:00:31

Abstract: Recent research has shown that an extreme interpretation of imperfect recall abstraction —- completely forgetting all past information —- has led to excessive abstraction issues. Currently, there are no hand abstraction algorithms that effectively integrate historical information. This paper aims to develop the first such algorithm. Initially, we introduce the KRWI abstraction for Texas Hold'em-style games, which categorizes hands based on K-recall winrate features that incorporate historical information. Statistical results indicate that, in terms of the number of distinct infosets identified, KRWI significantly outperforms POI, an abstraction that identifies the most abstracted infosets that forget all historical information. Following this, we introduce the KrwEmd algorithm, the first hand abstraction algorithm to effectively use historical information by combining K-recall win rate features and earth mover's distance for hand classification. Empirical studies conducted in the Numeral211 Hold'em environment show that under identical abstracted infoset sizes, KrwEmd not only surpasses POI but also outperforms state-of-the-art hand abstraction algorithms such as EHS and PAEMD. These findings suggest that incorporating historical information can significantly enhance the performance of hand abstraction algorithms, positioning KrwEmd as a promising approach for advancing strategic computation in large-scale adversarial games.

Main paper: 
 introduction: 
 Imperfect recall abstraction has proven to be very important for solving large-scale computational games, significantly reducing computational complexity. Recently, AI using imperfect recall abstraction has developed better-than-human strategies for Texas Hold'em testbed-even when using limited computational resources [23, 7, 8] . The task of hand abstraction in Texas Hold'em aims to reduce computational overhead by applying the same strategy to similar hands. In an imperfect recall setting [29, 20] , the hand abstraction in the later phase does not strict depend on the results of the hand abstraction in the earlier phase.However, the term imperfect recall is often interpreted in an extreme manner in practice. Researchers typically understand it as completely forgetting all past information-in other words, considering only future information-and design abstraction algorithms based on this understanding [16, 17, 19, 15, 14] . There are two major factors that mainly affect the results of abstraction for each phase: the number of clustering centers (i.e. centroids), which can be set man-Submitted to 38th Conference on Neural Information Processing Systems (NeurIPS 2024). Do not distribute.ually, and the number of distinct features that are used to categorize hands at each phase. Recent research [12] has found that constructing hand features solely based on future information can lead to excessive abstraction. For example, as shown in the Figure 1 , two hands: A and B constructed with only future information can have the same hand features. As the game progresses, the rate of feature repetition among different hands gradually increases, while the distribution of distinct hand features assumes a spindle-shaped pattern. Additionally, constructing hand features with historical information in addition to the future may differentiate two hands sharing the same future information and hence makes more features available for clustering as well as enhances the performance of hand abstraction.However, there still remain two unsolved issues. First, Fu et al. [12] have introduced a K-recall outcome feature, which incorporates historical information. This feature can only identify if elements are identical or not, but it lacks the capability to discern the extent of differences between features.Therefore, it is difficult to adjust the number of clusters appropriately, which makes it challenging to construct an effective hand abstraction algorithm that integrates historical information. Second, due to the inability to modify the number of clusters, Fu et al. [12] only compared the performance between the maximum clusters cases of integration of historical information (KROI) and no integration at all (POI). In this condition, although KROI significantly outperforms POI, the comparison is inconclusive because KROI recognizes more abstracted infosets than POI. Thus, it does not prove that the performance of abstraction algorithms that integrate historical information is necessarily superior under the condition of having the same number of abstracted infosets.This paper introduces a framework for constructing hand features based on winrates, with the Krecall winrate feature being the most crucial one. Based on this, we developed the K-recall winrate isomorphism (KRWI), an abstraction that integrates historical information. Across the same game phases, KRWI identifies slightly fewer hand features than KROI but significantly more than POI.Importantly, the K-recall winrate feature is capable of discerning the extent of differences between features. Therefore, by combining the earth mover's distance with the K-recall winrate feature, we developed the first hand abstraction algorithm that integrates historical information, named KrwEmd, and designed an efficient computational method. We validated our approach in the Numeral211 game environment, where KrwEmd demonstrated superior performance to POI under the same infosets conditions. Additionally, in clustering settings, KrwEmd also outperformed the Ehs and PaEmd algorithms, with PaEmd being the current state-of-the-art hand abstraction algorithm. 
background and notation: 
 Generally, Texas Hold'em-style poker games are modeled as imperfect information games. However, for the task of hand abstraction, games with ordered signals [18, 12] offer a better theoretical tool.The game with ordered signals is a subclass of imperfect information games in that they further subdivide the nodes (also called histories, states, or trajectories) in imperfect information games into mutually independent signals and public nodes. This allows for each aspect to be studied in isolation.Under this framework, the hand abstraction task in Texas Hold'em-style games is modeled as signal abstraction.In a game with ordered signals Γ = Ñ , H, Z, ρ, Ã, χ, τ , γ, Θ, ς, O, ω, ⪰, ũ , there is a set of players Ñ = N ∪ {c, pub}, which includes not only the main participants N = {1, . . . , N } but also a special nature player c who controls the randomness and an observer player pub who can see everything but doesn't take any actions. The game progresses through a series of public nodes X = H ∪ Z. Some of these public nodes are terminal public nodes Z where the game ends and outcomes are determined, while the others are non-terminal public nodes H. Among the non-terminal public nodes, some are where players make decisions within the action space Ã, and the remaining are chance public nodes where the nature player reveals signals, with the special action Reveal within Ã.At every non-terminal public node, ρ : H → N c (i.e., N ∪ {c}) specifies which player is responsible for making an action, and χ : H → 2 Ã confines the possible actions they can take. When the nature player makes a move, it reveals signals θ ∈ Θ that carry information relevant to the game. These ∈ {z ′ ∈ Z | ω i (z ′ ) = ω j (z ′ ) = true}, if ⪰ (θ, i, j) = true, then ũi (θ, z) ≥ ũj (θ, z).Players make decisions based on their observations of signals and the current non-terminal public node. A player may have the same observation for different signals, forming a signal infoset for signals they cannot distinguish. For a player i ∈ N , the signal infoset for a signal θ is denoted asϑ i (θ) = {θ ′ ∈ Θ | O i (θ) = O i (θ ′ ) ∧ O pub (θ) = O pub (θ ′ )}.Specifically, for the nature player, The signal infoset, combined with public nodes, can be transformed into the infoset of an imperfect information game. Fu et al. [12] detailed this transformation process.ϑ c (θ) = {θ ′ ∈ Θ | O pub (θ ′ ) = O pub (θ)}.The game with ordered signals model allows us to study the issue of signal abstraction independently.For this purpose, we introduce a signal (infoset) abstraction profile, α = (α 1 , ., α N ), where for each player i ∈ N , α i is a partition of Θ called the signal (infoset) abstraction. Any θ ∈ α i then is said to be an abstracted signal infoset for player i, and it can be further divided into several signal infosets within Θ i . These finer signal infosets collectively form a partition of θ. In general, two signal abstractions cannot be directly compared in terms of performance, but in a few specific cases there does exist a special relationship between them, which is called refinement. Consider two abstractions α i and β i . If ∀ θ ∈ β i , there exists one or more abstracted signal infosets in α i such that the union of these forms a partition of θ, then we said that α i refines β i , symbolically α i ⊒ β i . The signal abstracted game Γα was derived by substituting Θ i with α i across all x ∈ X.Perfect/imperfect recall originally describes a property of imperfect information games, indicating that players do not need to remember all the information they have observed throughout the game.Since games with ordered signals are a subset of imperfect information games, we derived the concept of signal perfect/imperfect recall from them. A player i in a game Γ is said to have signal perfectrecall if, for any θ ′ 1 , θ ′ 2 ∈ ϑ ′ , any predecessor θ 1 of θ ′ 1 has a corresponding predecessor θ 2 of θ ′ 2 such that θ 2 ∈ ϑ(θ 1 ). If all players have signal perfect recall, the game Γ is said to have signal perfect recall. For a game Γ with signal perfect recall, if α i is the signal abstraction of player i ∈ N , let (α i , Θ -i ) denote the signal abstraction profile where player i adopts the signal abstraction α i while other players do not do abstraction. If Γ(αi,Θ-i) retains signal perfect recall, then α i is considered a signal abstraction with perfect recall; otherwise, it is an signal abstraction with imperfect recall.In games with ordered signals, the strategy π i for player i maps from a non-terminal public node and a signal infoset to a probability distribution over actions, with the strategy profile denoted as π = (π 1 , . . . , π N ). When all players adopt the strategy profile π, the expected sum of future rewards, also known as expected value, for player i at public node x and signal θ is denoted as v π i (θ, x), and the expected value for the entire game is denoted as v i (π). A Nash equilibrium is a strategy profile where no player can obtain a higher expected value by changing their strategy. Formally, π * is a Nash equilibrium if for every player i, v i (π * ) = max πi v i (π i , π * -i ), where π -i denotes the strategies of all players except i. In two-player zero-sum scenarios, the exploitability of π is denotedas ϵ(π) = max π ′ 1 vi(π ′ 1 ,π2)+max π ′ 2 vi(π1,π ′ 2 ) 2Both players are dealt one private card face down, known as the hole card.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [NA] Justification: This paper introduces a novel hand abstraction algorithm that has been experimentally validated to outperform the previous state-of-the-art (SOTA) algorithm, PaEmd, and no significant flaws have been identified thus far.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.• The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.• The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.• The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.• The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.• If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.• While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
related work: 
 Our research focuses on hand abstraction techniques in AI systems for Texas Hold'em-style games (i.e. the signal abstraction in games with ordered signals), building on the initial works of Shi and Littman [25] and Billings et al. [4] . These seminal works introduced the concept of game abstraction, which aims to simplify games while preserving essential characteristics. The researchers started by manually forming hand buckets as a result of their expertise with game-playing strategy. The first automated hand abstraction was that of Gilpin and Sandholm [16] . Later, a model of games with ordered signals was given for Texas Hold'em by Gilpin and Sandholm [18] ; lossless isomorphism (LI) was developed with signal rotation. Despite the elegance of LI, its low compression rates hinder its application in large-scale games, whereas lossy abstraction shows potential for such application.An expectation-based clustering method was proposed by Gilpin and Sandholm [17] in their work, and a histogram-based clustering method was introduced by Gilpin et al. [19] . The former is known as Ehs, while the latter is referred to as the potential-aware method. Subsequent studies by Gilpin and Sandholm [15] and Johanson et al. [20] compared Ehs and potential-aware methods, concluding that the latter holds an advantage in large-scale games. Johanson et al. [20] also introduced the use of earth mover's distance1 (EMD) in potential-aware methods. Ganzfried and Sandholm [14] introduced a more efficient approximation algorithm for earth mover's distance in potential-aware methods (PaEmd). Brown et al. [9] further applied PaEmd to distributed environments for solving large-scale imperfect-information games. This paradigm has found success in Texas Hold'em AI systems and is considered state-of-the-art in hand abstraction. Very recently, Fu et al. [12] proposed several novel tools, such as abstraction resolution and common refinement. They introduced two signal abstraction: one is the potential outcome isomorphism (POI), which identifies the maximum number of abstracted signal infosets considering future information only; The other is the K-recall outcome isomorphism (KROI), which identifies the maximum number of abstracted signal infosets considering historical information. They emphasized that current imperfect recall signal abstraction algorithms, which consider only future information, are prone to excessive abstraction. However, they did not provide practical signal abstraction algorithms.Other abstraction techniques for decision-making problems include action abstraction [13, 6, 21] and general imperfect recall abstraction [10, 11] in extensive-form games, as well as state abstraction and action abstraction in reinforcement learning [1, 2] .The deck consists of a standard poker deck, excluding the Jokers, Kings, Queens, and Jacks, resulting in a total of 40 cards. There are four suits: spades (♠), hearts (♡), clubs (♣), and diamonds (♢), each containing ten cards numbered 2 through 9, and including the ten (T) and ace (A). : N → N. Potential winrate isomporphism map. rc = (pc (r) , . . . , pc (r-k) ). K-recall winrate feature of the input centroid.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [NA] Justification: This paper introduces a novel algorithm and validates its effectiveness through experiments, without involving theory or proofs.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.• All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.• Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.• Theorems and Lemmas that the proof relies upon should be properly referenced. 
winrate isomorphism: 
 The first contribution of this paper is an isomorphism framework of winrate-based features, including the potential winrate isomorphism (PWI) and the k-recall winrate Isomorphism (KRWI). Compared with outcome-based features, winrate-based features offer a streamlined approach, focusing exclusively on the distribution of loss, draw, and win outcomes of signals emanating from a signal infoset (and its predecessors) as it evolves towards the terminal signals. Winrate-based features are numerical vectors of consistent length. In this section, an identical Winrate-based feature uniquely determines an abstracted signal infoset. It is worth noting that the similarity of Winrate-based features reflects the similarity among signal infosets, allowing for clustering based on these features (see Section 5).Both PWI and KRWI share the similar isomorphism construction process for player i in phase r, as illustrated in algorithm 1. The difference lies only in the construction operator for the winrate-based features, FEATURE, used in lines 5 and 12. The isomorphism construction process starts by iterating through all signal infosets of Θ (r) i and collecting their features. Next, these features are deduplicated and stored in lexicographical order within set C (r) i , which is implemented as a vector data structure.i , the index of a feature serves as an identifier for an abstracted signal infoset. Then, by utilizing a hash table CI (r) i , we can identify an abstracted signal infoset's identifier based on its feature. In the final step, we traverse Θ (r) i again, associating the identifier of a signal infoset with the identifier of its corresponding abstracted signal infoset, and this relationship is recorded inD (r)i , an isomorphism map. The function Index i (r, •) is a domain-specific mapping that assigns a unique identifier to each signal infoset at phase r, within the numeric range of 0 to|Θ (r) i | -1. In Algorithm 1 Isomorphism Constructor Require: r = 1, . . . , R. Phases. Θ (r) i . Signal infoset space for player i. Index i (r, •) : Θ (r) i → N. Signal infoset index function for player i. 1: procedure ISOMORPHISMCONSTRUCTOR(r, Θ (r) i , FEATURE(•)) 2:Initialize C (r) i vector as empty. f eature ← FEATURE(ϑ). for ϑ ∈ Θ (r) i do 12: f eature ← FEATURE(ϑ), idx ← Index i (r, ϑ). 13: Update D (r) i [idx] with CI (r) i [f eature]. 14: end for 15: return (C (r) i , D (r) i ).16: end procedure Texas Hold'em-style games, one optional approach for implementing this function is through lossless isomorphism [18, 27] .Potential winrate isomorphism (PWI) is a signal abstraction that classify signal infosets based on its potential winrate features. These features focus on the distribution of a player's winrate over terminal signals after passing through a given signal infoset, without considering the history of how the player reached the signal infoset. Specifically, for player i in phase r, the potential winrate feature associatedEQUATION• pf (r),0 i (ϑ) denotes the probability that player i ranks lower than least one other player in the terminal signals, after passing through ϑ.• pf (r),l i (ϑ), for l > 0, denotes the probability that player i ranks no lower than any other player and ranks higher than exactly l -1 other players in the terminal signals, after passing through ϑ.In the terminal phase, the winrate feature is calculated by directly statisticing the game outcomes for players in the given signal infoset. Moreover, in the non-terminal phases, we use a recursive approach to simplify the computation of the winrate feature, thereby avoiding the need to enumerate every signal infoset down to the terminal phase. The recursive formula is In non-terminal phases, the potential winrate feature is a simplified version of the potential outcome feature. Unsurprisingly, PWI also results in excessive abstraction similar to POI. As shown in Table 2 , in heads-up limit hold'em (HULHE) and heads-up no-limit hold'em (HUNL), the number of abstracted signal infosets identifiable by lossless isomorphism increases with each phase, indicating that the game becomes increasingly complex. However, the number of abstracted signal infosets identifiable by PWI and POI first increases and then decreases, showing a spindle-shaped pattern.EQUATIONAnd we observed that when only future information is considered, winrate-based features may lead to greater information loss compared to outcome-based features. For instance, in the River phase, the number of abstracted signal infosets identified by PWI is only 79.16% of that identified by POI.As Fu et al. [12] mentioned, supplementing historical information can enhance the ability of signal abstraction to identify abstracted signal infosets. Inspired by KROI's construction approach, we developed the k-recall winrate isomorphism (KRWI). The key difference is that instead of using k-recall outcome features to distinguish between different signal infosets, KRWI utilizes k-recall winrate features.In a game with signal perfect recall, all signals within the signal infoset ϑ have their predecessors at phase r ′ , which belong to the identical signal infoset ϑ ′ . For player i at phase r, the signal infosetϑ ∈ Θ (r) ihas a k-recall winrate feature (k < r) represented as a numerical array with a dimension ofEQUATIONWhen r ′ is less than r, pf(r ′ ) i(ϑ) denotes the potential winrate feature for the predecessor signal infoset ϑ ′ of ϑ at phase r ′ . Since we have stored all the potential winrate features of ϑ ∈ Θ (r)i through PC (r) i , PD (r) iand assigned them unique identifiers in Algorithm A1. To save storage space and facilitate retrieval, what we actually store isEQUATIONPD (r ′ ) i [ϑ]is the identifier for the potential winrate feature of the predecessor ϑ ′ of ϑ in the r ′ phase, r ′ ≤ r. For algorithm details, please refer to Appendix A.2.Just as the potential winrate feature is a simplified version of the potential outcome feature, the k-recall winrate feature is a simplified version of the k-recall outcome feature. Table 1 shows the number of signal infosets that KRWI and KROI can identify and their ratio in HUNL&HULHE. We were pleasantly surprised to find that while the ratio of PWI to POI resolution can drop below 80%, when k is set to its maximum value, i.e. r -1, the ratio of KRWI to KROI resolution can reach nearly 90% at a minimum, with most of the information preserved. Also, we can easily observe that the number of abstracted signal infosets identified by KRWI is much higher than that identified by POI.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes]Justification: We have provided detailed information on the testbed, evaluation metrics, and experimental scenarios in Section 6. Additionally, specific experimental parameters and equipment are given in Section 7. Therefore, we have included sufficient details in the paper to reproduce the experiments.Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.• If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.• Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.• While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/ public/guides/CodeSubmissionPolicy) for more details.• While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).• The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more details.• The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.• The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.• At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).• Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
k-recall winrate abstraction with earth mover's distance: 
 Fu et al. [12] introduced potential and k-recall outcome features, referred to as outcome-based features, to distinguish different abstracted signal infosets. In the previous section, we developed potential and k-recall winrate features, termed winrate-based features, for the same purpose. In these two methods, Each unique feature corresponds to a single abstracted signal infoset. Intuitively, we can infer that feature similarity might reflect the similarity among abstracted signal infosets, enabling further abstraction and compression for application in large-scale games. However, assessing similarity with outcome-based features is challenging because the identification code indicates only the category, without reflecting the degree of similarity. In contrast, winrate-based features represent winrates, which are inherently comparable, allowing for an easy definition of distances between them.For the signal information sets ϑ, ϑ ′ of player i at phase r, we can define the distance of their k-recall winrate feature asEQUATIONAmong Equation ( 5), Emd is the operator used to calculate the earth mover's distance (EMD) [24] .The EMD calculates the distance between two histograms using optimal transport theory. Since it requires solving linear programming equations, the computational complexity of the EMD is sensitive to the dimensionality of the histograms, and approximate algorithms are usually used for larger-scale problems. However, the dimensionality of winrate-based features is small, with a dimension of 3 in a two-player scenario, so we attempt to use a fast algorithm for accurately computing the EMD [5] .w 0 , . . . , w k are hyperparameters used to control the importance of EMD at each phase r, . . . , r -k.We use the KMeans++ algorithm [3] , combined with the distance of their k-recall winrate feature, to cluster the abstracted signal infosets of KRWI. We named this algorithm KrwEmd.Although calculating EMD on small-dimensional histograms is already very fast, clustering actual Texas Hold'em still faces a significant computation. For example, for the River phase of We conducted experiments on the Nu-meral211 Hold'em [12] testbed. Nu-meral211 is a two-player three-phase Taxes Hold'em-style game with more complex hand systems than the Leduc Hold'em [26] and Rhode Island Hold'em [25] test environments, making it suitable for studying hand abstraction issues. Detailed rules are included in Appendix B.Table 3 shows the number of abstracted signal infosets recognized by KRWI and KROI, along with lossless isomorphism, in Numeral211 Hold'em.CSMCCFR algorithm [30, 22] in different abstracted signal infoset scales. We gauge the performance over exploitability. For doing that, we consider both symmetric and asymmetric abstraction scenarios.In this symmetric abstraction setting, we measure the exploitability of approximate equilibrium that is yielded when both the players in the game employ signal abstraction in the original game.However, it may lead to the abstraction pathology [28] . To avoid such problems, we illustrate the theoretical performance of the signal abstraction under evaluation through asymmetric abstraction.The approximate equilibrium in the signal abstracted games Γ(α1,Θ2) and Γ(Θ1,α2) is obtained to obtain π * ,1 and π * ,2 , respectively. Finally, we concat the two strategies to get π ′ = (π * ,1 1 , π * ,2 2 ) and check the exploitability of π ′ .7 Experiment✁ ✂✄☎✆✁ ✂✄☎✝✁ ✞✆✁ ✟✠✡☛☞✌✄✍✄✎✄✏ ✟✠✡☛☞✌✄✏✄✎✄✍ ✑ ✒ ✓ ✔ ✕ ✖ ✗ ✘ ✙ ✖ ✔ ✖ ✗ ✚ ✛ ✜ ✙ ✢ ✣ ✤ ✥ ✦✧ ✧✥ ★✧ ✩✥✥ ✩✦✧ ✪✫✬✭✮ ✯✰✱✫✬✰✭✲✮✳ ✩✥ ✴ ✩✥ ✵✶ (a) ✁ ✂✄☎✆✁ ✂✄☎✝✁ ✞✆✁ ✟✠✡☛☞✌✄✍✄✎✄✏ ✟✠✡☛☞✌✄✏✄✎✄✍ ✑ ✒ ✓ ✔ ✕ ✖ ✗ ✘ ✙ ✖ ✔ ✖ ✗ ✚ ✛ ✜ ✙ ✢ ✣ ✤ ✥ ✦✥ ✧✥ ★✥ ✩✥ ✪✫✬✭✮ ✯✰✱✫✬✰✭✲✮✳ ✦✥ ✴ ✦✥ ✵✶ (b)Figure 4 : Full abstraction setting experiment, trained for 5.5 × 10 10 iterations.Firstly, we provide an evaluation of the performance of KRWI (2-RWI) compared with KROI (2-ROI)and POI (0-ROI) approaches and lossless isomorphism. We keep the most abstracted signal infosets identified under the full abstraction setting. Note that POI is the common refinement of existing signal abstraction algorithms that only consider future information. And, since previous works cannot control the number of abstracted infoset, they cannot justify their performance in that considering historical information in signal abstraction was better than that in signal abstraction with the same number of abstracted infoset. To investigate this issue, we included KrwEmd and set the clustering scale to be consistent with POI. Note here, that 2-RWI and 2-ROI share the same capability of infoset recognition in Preflop and Flop, while POI is only a little bit worse than 2-RWI and 2-ROI in Flop.Thus, we can directly allow clustering of KrwEmd abstraction use the abstracted signal infosets identified by POI in Preflop and Flop, and only perform clustering in River. Here, we design four sets of hyper-parameters: (w 0 , w 1 , w 2 ), i.e., exponentially decreasing: (16, 4, 1), linearly decreasing:(7, 5, 3), constant: (1, 1, 1), and increasing: (3, 5, 7) in the importance of historical information. We only show the result of best-and worst-performing parameters (to make the figure neat). The full figures appear in the Appendix C. Figure 4a shows the result of symmetric abstraction, while Figure 4b shows the result of asymmetric abstraction. We observed that both symmetric and asymmetric abstractions maintained consistent abstraction performance without abstraction pathologies. As expected, overfitting was observed in the symmetric abstraction scenario while in the asymmetric scenario overfitting was significant only for POI. The performance difference between 2-RWI and 2-ROI is small, which means that under the full abstraction setting, using simple winrate-based features instead of complex outcome-based features can achieve nearly the same performance. Even with the worst parameter configuration (increasing importance), KrwEmd with the same number of abstracted signal inforsets as POI still outperforms POI. Next, we compared the performance of KrwEmd with the currently applied signal abstraction algorithms Ehs and PaEmd. It should be noted that POI is the common refinement both for Ehs and PaEmd, meaning that the maximum number of abstracted signal infosets they can recognize will not exceed that of POI. Thus, we set a compression rate that is 10 times lower than that of POI, while not performing abstraction for Preflop. The final number of abstracted infosets is set to (100, 225, 396).✁✂ ✄☎ ✆✝ ✞✟✠ ✆✝✡☛☞✡✌✡☛ ✞✟✠ ✆✝✡✍✡✎✡✏ ✞✟✠ ✆✝✡☛✡☛✡☛ ✞✟✠ ✆✝✡✏✡✎✡✍ ✑ ✒ ✓ ✔ ✕ ✖ ✗ ✘ ✙ ✖ ✔ ✖ ✗ ✚ ✛ ✜ ✙ ✢ ✣ ✤ ✥✦✦ ✧★✦ ✩✦✦ ✪✫✬✭✮ ✯✰✱✫✬✰✭✲✮✳ ✴✦ ✵ ✴✦ ✶ ✴✦ ✷✸ (a) ✁✂ ✄☎ ✆✝ ✞✟✠ ✆✝✡☛☞✡✌✡☛ ✞✟✠ ✆✝✡✍✡✎✡✏ ✞✟✠ ✆✝✡☛✡☛✡☛ ✞✟✠ ✆✝✡✏✡✎✡✍ ✑ ✒ ✓ ✔ ✕ ✖ ✗ ✘ ✙ ✖ ✔ ✖ ✗ ✚ ✛ ✜ ✙ ✢ ✣ ✤ ✥✦ ✧✦ ★✩✦ ✪✫✬✭✮ ✯✰✱✫✬✰✭✲✮✳ ★✦ ✴ ★✦ ✵✶ (b)To exclude the influence of random events on performance, we generated 3 sets of abstractions for Ehs and PaEmd each. KrwEmd used hyperparameters (w 3,0 , w 3,1 , w 3,2 ; w 2,0 , w 2,1 ) in Flop and River, which are exponentially decreasing (16, 4, 1; 4, 1), linearly decreasing (7, 5, 3; 5, 3), constant(1, 1, 1; 1, 1), and increasing (3, 5, 7; 5, 7) in the importance of historical information. Additionally, since PaEmd uses approximate EMD calculations, its approximate distance is asymmetric, making it difficult for the algorithm to converge. We truncated after 1000 iterations on a single core, with an average cost of 1427.7s, while Ehs and KrwEmd both achieved convergent clustering results, requiring an average of 12.3 and 96.7 iterations, with average time costs of 11.2s and 341.4s, respectively.Figure 5a shows the results of symmetric abstraction experiments, while Figure 5b shows the results of asymmetric abstraction experiments. We observed that both symmetric and asymmetric abstractions maintained consistent abstraction performance, similar to the full abstraction scenario, without significant abstraction pathologies. The experimental results show that KrwEmd's performance is far superior to that of Ehs and PaEmd under all parameter settings. Our experiments also confirmed that, despite PaEmd's convergence issues, it is indeed a more effective abstraction algorithm than Ehs. Additionally, we further validated that the importance of historical information decreases progressively from bottom to top, although this time the best-performing parameter was exponentially decreasing rather than linearly decreasing as in the previous experiment.These two experiments validate that considering historical information is indeed more effective than considering future information only in signal abstraction even in imperfect recall setting. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: As stated in the reproducibility statement, we have provided detailed information on the testbed, evaluation metrics, and experimental scenarios in Section 6. Additionally, specific experimental parameters and equipment are given in Section 7.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.• The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: Due to the long experimental time and limited sample size, error bars cannot be provided. In the first experiment (Figure 4 ), the baseline settings adopt fixed abstraction settings and have a large performance gap, so the performance of strategies solved by CSMCCFR is stable and not easily affected by random factors. In the second experiment (Figure 5 ), random factors may indeed affect individual experimental data. Therefore, we sampled the control group multiple times and drew the performance range, which is far lower than the performance of our algorithm under the worst parameters, which also proves the effectiveness of the algorithm.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).• If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
conclusion: 
 This research introduces the first imperfect recall signal abstraction algorithm that considers historical information. This algorithm has the ability to adjust the scale of the abstracted signal infosets. Based on this, we fully verified that the imperfect recall signal abstraction and abstraction algorithms considering historical information is superior to that only considering future information. Therefore, the KrwEmd algorithm has replaced the PaEmd algorithm and become the SOTA in this field. Based on the KrwEmd algorithm, we are expected to build a stronger Texas Hold'em AI.Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: We discuss the computational resources and time cost of the experiments in Sections 5, 7, and Appendix A.3.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.• The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.• The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss the impact of this work in Section 8. As noted, this work represents the SOTA in hand abstraction algorithms and could be used to create more powerful Texas Hold'em AI.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.• The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.• If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: Our work focuses solely on introducing a more efficient algorithm for hand abstraction and does not involve the release of data or models.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.• Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.• We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: This paper provides comprehensive citations for all comparative methods involved, and all comparison experiments were re-implemented without using existing tools or code.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL.• The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.• For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: This paper does not release any new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.• The paper should discuss whether and how consent was obtained from people whose asset is used.• At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: This paper does not involve human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. Justification: This paper does not involve human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.• For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Require: r = 1, . . . , R. Phases.Θ i = R r=1 Θ (r)i . Signal infoset space for player i. Index i (r, •) : Θ EQUATIONi , FEATUREFUNC). for ϑ ′ ∈ Θ (r+1) i , such that ∃θ ′ ∈ ϑ ′ , ∃θ ∈ ϑ: ς(θ ′ |θ) > 0 do 18: idx ← Index i (r + 1, ϑ ′ ), abs ← PD (r+1) i [idx], f eature ϑ ′ ← PC (r+1) i [abs].for j = 0 to N do 20: Algorithm A1 describes the computation process for potential winrate isomorphism. This algorithm operates in reverse, starting from the game's final phase R.f eature ϑ [j] ← f eature ϑ [j] + f eature ϑ ′ [j]P r{ϑ ′Algorithm A2 constructs the k-recall winrate isomorphism using the k-recall winrate feature. This process requires the prior construction of the potential winrate isomorphism map PD (r) i using Algorithm A1.According to Equation (5), we note that the distance calculation between a k-recall winrate isomorphism class and a centroid's k-recall winrate feature can be decomposed into k+1 pairs of potential winrate feature EMD calculations. The potential winrate feature of the hand remains unchanged, while only the potential winrate feature of the centroid changes. Decomposing the calculation into the EMDs of potential winrate features involves significantly fewer computations than directly calculating the EMD of two k-recall winrate features. Specifically, for the River phase of HUNL&HULHE, we have the compression ratio as 169+1028325+1850624+20687 529890863 = 2899805 529890863 = 0.0054725.Algorithm A3 describes how we accelerate the batch EMD computation between a centroid and all KRWI classes' k-recall winrate features. It should be noted that the K-recall winrate feature involved in the calculation of the centroid in the algorithm is in the form of Equation (3), while the K-recall winrate feature in RC (r,k) is in the form of Equation ( 4). This method reduced the computational for r = 1 to R do 3:EQUATIONi , FEATUREFUNC). return (RC(1,0) i , RD (1,0) i ), . . . , (RC (k+1,k) i , RD (k+1,k) i ), . . . , (RC (R,k) i , RD (R,k) i). 8: end procedure 9: procedure KRECALLWINRATESFEATURE(ϑ, r, k) 10: initial a empty vector f eature.for s = r to r -k do 12:ϑ ′ ← the predecessor signal infoset of ϑ in the s phase for player i.13: idx ← Index i (s, ϑ ′ ), abs ← PD (s) i [idx].Append f eature with abs. The distance batch calculation for each centroid can be processed independently and distributed across tens of multi-core computer (e.g. 96-core computers), with each computer responsible for calculating the features of some centroids in one iteration, which are then aggregated. Using this technique, we can reduce an iteration to a few hours, which is acceptable for Texas Hold'em AI training.Numeral211 Hold'em is played according to the following rule:1. Ante: Each player antes 5 chip into the pot at the start of the hand.Distances of all k-recall winrate feature with centroid. 1: procedure DISTANCEBATCH(w 0 , . . . , w k , rc, r, k)Initial phase s empty earth mover's distance vector EmdDis (s) for s = r, . . . , r -k. Initial empty output distance vector Dis.for t = 0 to k do3:for pf in PC Append EmdDis (r-t) with Emd(pf, rc[t]) for rf i in RC (r,k) i do 8:dis ← 0.for t = 0 to k do 10:dis ← dis + w t * EmdDis (r-t) [PD (r-t) i [rf i[t]]].11:end for 12:Append Dis with dis.end for return Dis. 14: end procedure 9. Showdown: If neither player folds, a showdown occurs. Players reveal their cards, aiming to form the best possible hand. The player with the highest-ranked hand wins the pot. In the case of a tie, the pot is split evenly. The Table 2 show the hand ranks of Numeral211 Hold'em.10. Betting Options: Throughout the game, players have options to fold, call, or raise. In each betting phase, the total sum of bets and raises is limited to a maximum of 4, with fixed bet sizes of 10 chips in the first phase and 20 chips in the last two betting phases.Figure 6 show all of the result in experiment 1. Guidelines:✁ ✂✄☎✆✁ ✂✄☎✝✁ ✞✆✁ ✟✠✡☛☞✌✄✍✄✍✄✍ ✟✠✡☛☞✌✄✍✎✄✏✄✍ ✟✠✡☛☞✌✄✑✄✒✄✓ ✟✠✡☛☞✌✄✓✄✒✄✑ ✔ ✕ ✖ ✗ ✘ ✙ ✚ ✛ ✜ ✙ ✗ ✙ ✚ ✢ ✣ ✤ ✜ ✥ ✦ ✧ ★ ✩✪ ✪★ ✫✪ ✬★★ ✬✩✪ ✭✮✯✰✱ ✲✳✴✮✯✳✰✵✱✶ ✬★ ✷ ✬★ ✸✹ (a) ✁ ✂✄☎✆✁ ✂✄☎✝✁ ✞✆✁ ✟✠✡☛☞✌✍✍✍ ✟✠✡☛☞✌✍✎✏✍ ✟✠✡☛☞✌✄✑✄✒✄✓ ✟✠✡☛☞✌✄✓✄✒✄✑ ✔ ✕ ✖ ✗ ✘ ✙ ✚ ✛ ✜ ✙ ✗ ✙ ✚ ✢ ✣ ✤ ✜ ✥ ✦ ✧ ★ ✩★ ✪★ ✫★ ✬★ ✭✮✯✰✱ ✲✳✴✮✯✳✰✵✱✶ ✩★ ✷ ✩★ ✸✹ (b)• The answer NA means that the abstract and introduction do not include the claims made in the paper.• The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.• The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.• It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.https://en.wikipedia.org/wiki/Earth_mover%27s_distance 


Decision: This paper proposes a new approach to hand abstraction in Texas Hold'em-style poker games, addressing the limitation of previous methods that disregard historical data. The authors present two novel methods that explore a promising direction to alleviate the existing issue. However, significant improvements are needed to take place before publication. The reviewers' low confidence score and numerous clarification questions on the setting are indication to this. That is, a substantial editing is required to take place before the paper is ready for publication.

I do encourage the authors to use the feedback to improve the work and resubmit it in the future, as it has value for the research community.

Reviews: review: paper_summary: paper_summary: The paper develops new hand abstraction techniques for Texas Hold'em-style games (in general: games with ordered signals), which fare better than previous methods in both the number of hands identified, and performance (exploitability) in a simplified version of the game. 

Hand abstraction is a technique aiding the strategy construction in a Texas Hold'em, where concrete hands, or rather concrete signal infosets (i.e. "possible words" according to the information revealed so far), are replaced by abstract infosets, represented in an abstract feature space (here $\mathbb{R}^n$).

The core idea of the paper is to use the features of hands from previous rounds in the construction of the current round feature. More precisely, the paper investigates a simple method (KRWI = *k-recall winrate isomorphism*) of maintaining, at a given round, the collection of all potential-winrate isomorphims (PWI) features from previous $k$ rounds by concatenating them all together. PWI for an $n$-player game is a categorical probability distribution over $n+1$ events of a form *"this player outperformed exactly $l-1$ other players while losing to none"* for $l = 1, 2, \ldots n$ and *"this player lost to at least one player"*. Those distributions can be computed by a dynamic programming method. To reduce the cardinality of the space, the paper later clusters KRWI features with k-means using the Wasserstein distance, naming it the KrwEmd method.

All of the methods are benchmarked against currently used techniques that do not use historical information. Experiments find that KRWI identifies similar proportion of signal infosets as the previously used KROI. Using the metric of exploitability of the equilibrium (how it deviates from Nash equilibrium) of the strategy found by an imperfect-information game solver, authors find that 2-RWI-based approach performs almost the same as 2-ROI, and that KrwEmd outpefrorms previously used Ehs and PaEmd by a relatively large margin.
strength_weakness: strength_weakness: From the perspective of someone not at all acquainted with the field of imperfect information games/games with ordered signals, the paper was quite hard to read and understand - even though (assuming that the authors agree with my summary), the contribution is a relatively straightforward idea.

The introduction was uninformative and confusing (I would recommend rewriting the whole second paragraph); the preliminaries, although presented in-depth and trying to be formal, also posed quite a few questions; sections 4 and 5 describing the main contribution lacked detail and justification (i.e. ideally I would like to see definition/theorem/proof style - otherwise the text is impossible to read for someone unfamiliar with the field), and the experimental setup is assuming a lot of background knowledge that was not explained neither in the main paper nor in the appendix (it was also difficult to gauge if the comparison between SOTA and the new approach was fair from the resources pov - the paper reports some numbers, but never an aggregated "memory/time used" for all methods).

Please see Questions below for a detailed explanation of what I found lacking or hard to understand.
questions: questions: Intro:
- What are the "clustering settings" referred to in line 66? (It's difficult to understand how important it is for the claim 'outperforms the SOTA')

Prelims:
- I think the definition of the game $\Gamma$ should include the initial distribution over signals, to start iterating the map $\varsigma
: \Theta \to \Delta(\Theta)$
- I did not understand how exactly the map $\varsigma$ interacts with the shape of the game tree given by $X, \tau$ and the order $\sqsubseteq$ on $\Theta$ - are we assuming that $\theta \sqsubseteq \theta'$ for all $\theta'$ in the codomain of $\varsigma(\theta)$ (dividing the signal space appropriately)? Are the terminal signals $\tilde{\Theta}$ take wrt to the order $\sqsubseteq$, or wrt to the "realisable" order of '$\theta$ is final if it can only ever appear as the final signal of any game trajectory'? Is the order $\sqsubseteq$ assumed to be tree-shaped? (line 12 of algorithm A2 requires a unique predecessor of $\vartheta$) 
- Why is the order $\preccurlyeq$ assumed to be partial? Shouldn't it be total on $\tilde{\Theta}$?
- Are the survival status and the $\preccurlyeq$ relevant to the paper? Is the signal abstraction refinement relevant to the paper?
- Why is $\gamma$ provided in the definition of the game and not just derived from other structures?
- What happens to observations $O$ and the map $\varsigma$ in the signal abstracted game $\Gamma^{\alpha)?
- Line 121 - "such that the union of those" should be just "such that they"
- Is the criterion of the signal perfect recall for a game $\Gamma$ equivalent to saying that "players can be Markov"? I.e. that for any non-Markov (wrt to signals) strategy profile $\pi = (\pi_i, \pi_{-i}$) there exists a Markov (wrt to signals) strategy $\pi'_i$ such that $(\pi'_i, \pi_{-i})$ is equivalent to $\pi$?
- I think the signal space $\Theta$ has to be assume to be finite - otherwise, the definition of perfect recall breaks.

Winrate Isomorphism:
- What are the outcome-based features? (never explained, but used throughout this and next sections). What is POI?
- Line 179 - "an identical Winrate-based feature uniquely determines an abstracted signal infoset" - is that a definition of the abstracted signal infosets for winrate-based features? If so, this requires proof or justification (that they partition nicely, satisfy order conditions, interact with  $\varsigma$ in the right way etc).
- The above question is even more relevant for KrwEmb, which uses a complicated clustering mechanism inside.
- Why is $\mathcal{D}$ an isomorphism?
- Line 196 - typo in "classify"
- What is the intuition behind constructing PWI in this particular way (i.e. with "lose at least to one other player"/"win with with $l-1$ players and tie with the rest")? Is this a natural choice? Why not "lose to $l$ players$" or some other metric?
- What is the River phase? What is HUNL&HULHE?
- It would be good to include at least the definition of the Wasserstein distance, instead of linking to Wikipedia

Experiments:
- What is "mb/g" in Figures 4 and 5?
- The plots do not show LI (potentially overshadowed?) and overshadow 2-RWI. I would recommend changing the style, since this is one of the few most important results in this section.
- Line 299 - "We gauge the performance over exploitability" - what does that mean?
- Line 331 - typo in "infosets"
- Line 336 - "the final number of abstracted infosets is set to" - are these numbers for Ehs, PaEmd and KwrEmd respectively? In what order? 
- Line 338 - why does KrwEmd use so many parameters here, instead of just 3?
- If the costs of 1000x1427.7s, 12x11.2s and 96.7x341.4s are total costs, wouldn't it be more fair to set the hyper-parameters of the algorithms such that their total computational budget is approximately equal? Otherwise, it's difficult to judge the performance improvement.
limitations: limitations: Yes.
paper_summary: paper_summary: This paper proposes KrwEmd, a novel hand abstraction algorithm for imperfect recall settings in Texas Hold'em poker. The algorithm leverages K-recall winrate features, incorporating historical information in addition to future information for constructing hand abstractions. The authors introduce two new isomorphism frameworks: Potential Winrate Isomorphism (PWI) and K-recall Winrate Isomorphism (KRWI). They demonstrate that KRWI outperforms existing methods like POI in identifying distinct infosets. KrwEmd, which combines KRWI with Earth Mover's Distance (EMD) for hand classification, shows superior performance compared to POI, Ehs, and PaEmd in the Numeral211 Hold'em environment.
strength_weakness: strength_weakness: I found the paper  challenging to understand, though this may be due to my limited background knowledge in poker AI and game theory.  While the authors provide a background section, the density of the technical content and the numerous specialized terms make comprehension difficult.

The description of the accelerated algorithm in Appendix A.3 could be expanded for better understanding. Additionally, a clear discussion of the limitations of the accelerated algorithm would be beneficial.

The paper provides limited information about the proposed algorithms, particularly KrwEmd. While the core concepts are presented, the details regarding implementation and specific design choices are limited. More in-depth explanation and pseudocode would enhance the paper's quality.
questions: questions: What are the limitations of the accelerated algorithm? Does it introduce any approximations or trade-offs in performance or accuracy?

Have you explored the application of KrwEmd to other poker variants beyond Numeral211? Are there any specific challenges or adaptations needed for different game settings?

How sensitive is KrwEmd to the choice of hyperparameters (w0, w1, w2)? Is the algorithm robust to different hyperparameter configurations?
limitations: limitations: The authors acknowledge the computational complexity of clustering with EMD and introduce an accelerated algorithm. However, the paper lacks a dedicated section addressing the limitations of the proposed methods and the accelerated algorithm.
paper_summary: paper_summary: This paper focuses on the problem of hand abstraction for Texas Hold-Em style poker games. Hand abstraction is the process of partitioning game histories into infosets which still contain enough information to make strategically advantageous decisions. Previous approaches have focused on abstractions that primarily focus on the future outcomes from each hand, but the authors suggest that it may instead be beneficial to also include past information. They design a hand abstraction algorithm called KwrEmd that outperforms previous work by incorporating historical information.
strength_weakness: strength_weakness: As somebody who is unfamiliar with the subfield of imperfect recall abstraction, I found the paper to be quite confusing throughout. The authors do not often provide intuition or examples for their method, and I found it difficult to tell exactly which contributions were novel compared to Fu et al. While it's reasonable for a paper to use technical language at times, well-written papers are usually understandable by a broader set of readers than just those in the specific subfield.
questions: questions: No specific questions.
limitations: limitations: Not sure.
paper_summary: paper_summary: This paper introduces a novel approach to hand abstraction in Texas Hold'em-style poker games, addressing the limitations of current methods that often disregard historical information. The authors make two primary contributions: First, they develop KRWI (K-Recall Winrate Isomorphism), a new abstraction method that incorporates historical information from previous game phases. Second, they present KrwEmd, the first hand abstraction algorithm to effectively combine K-recall win rate features with earth mover's distance for hand classification. Through experiments conducted in the Numeral211 Hold'em environment, the authors demonstrate that KrwEmd significantly outperforms state-of-the-art algorithms such as Ehs and PaEmd in terms of exploitability, while maintaining the same number of abstracted information sets. This work shows that incorporating historical information can substantially enhance the performance of hand abstraction algorithms, potentially leading to more advanced strategic computation in large-scale adversarial games and stronger poker AI systems.
strength_weakness: strength_weakness: For me, the paper's primary weakness is the presentation method. I had trouble understanding the significance and nature of the contribution from the current submission. In general, a clearer description of this area of research for people who, e.g., work on games but don't focus on poker would be quite helpful. Some specific suggestions/areas for improvement are:

* Clearer introduction of key concepts: The paper jumps into technical terms like 'imperfect recall abstraction' and 'hand abstraction' without adequately explaining them for a broader audience. A brief explanation of why these concepts are important in poker AI would be beneficial.
* More intuitive explanations of the algorithms: The descriptions of PWI, KRWI, and KrwEmd are highly technical. Including some simple examples or diagrams to illustrate how these algorithms work could greatly improve understanding.
* Better contextualization of the contribution: While the paper claims to outperform existing methods, it's not clear how significant this improvement is in the broader context of poker AI. A discussion of the practical implications of this improvement would be valuable.
* Clarification of experimental setup: The Numeral211 Hold'em environment is not well-known. A clearer explanation of how this relates to standard poker variants would help readers understand the relevance of the results.
* More accessible presentation of results: The graphs and tables are dense with information but lack clear explanations. Simplifying these visualizations or providing more guidance on how to interpret them would be helpful.
* Glossary of terms: Given the many technical terms used (e.g., 'earth mover's distance', 'K-recall winrate feature'), a glossary could be a valuable addition to help readers keep track of these concepts.
questions: questions: Please address my comments about clarity overall and address any specifics that seem appropriate.
limitations: limitations: There's limited discussion of limitations. It would be good to include an explicit limitations section. In particular, it would be good to discuss potential computational challenges in more detail, any limitations on the scope of the evaluation, and future work that might be planned.
paper_summary: paper_summary: This paper introduces the KrwEmd algorithm, a novel approach to hand abstraction in Texas Hold’em-style games. The main contribution is the integration of historical information using K-recall winrate features and earth mover’s distance, addressing the limitations of previous imperfect recall abstraction methods. The algorithm demonstrates significant performance improvements over state-of-the-art methods.
strength_weakness: strength_weakness: Scalability: While the paper demonstrates KrwEmd's effectiveness in Texas Hold’em-style games, its scalability to more complex and diverse game scenarios remains to be explored.

Comparative Analysis: The paper could benefit from a more detailed comparison with existing hand abstraction methods to better highlight KrwEmd's unique advantages and limitations.
questions: questions: 1. How well does KrwEmd scale when applied to more complex and diverse game scenarios beyond Texas Hold’em? Could the authors provide any preliminary results or insights into this?

2. While KrwEmd achieves significant performance improvements, are there scenarios or tasks where additional training data might be beneficial or required? If so, what kind of training data would be most useful?

3. Are there other game models or systems that KrwEmd could enhance similarly? What challenges might arise in such integrations?
limitations: limitations: The paper primarily focuses on Texas Hold’em-style games. While this is a significant achievement, a discussion on the model's scalability and generalization to more complex and varied game scenarios would be beneficial. Including potential strategies to address these challenges would strengthen the paper.

Comments: title: Thanks for reviewer
Response for the comment
Response
Thanks for review
Response
Thanks for the review
Answers for the questions
Answers for the questions
Answers for the questions
The explanations of some confusing terms
comment: Dear Reviewer,

We noticed that you haven't yet provided a response, and we appreciate your recognition of our work. Currently, our submission is on the borderline for acceptance. Would it be possible for you to consider increasing one more score? This would greatly helpful for the work.

We are confident in the contribution of our research. The field of hand abstraction has seen little progress in the past decade, and our work offers a seamless application to current poker AI systems, potentially improving their performance significantly. Moreover, our research introduces new ideas that could inspire a series of further advancements in hand abstraction.

Thank you for your consideration.

Best regards,
Thanks for your review. It is very helpful. : )
I thank the authors for continued discussion, and I appreciate the willingness to fix the problems. Their answers to my minor points above seem reasonable.

I haven't increased my score mostly because the number of changes (mentioned in response to this review, as well as in other comments) require a substantial re-write of the paper. This seems to be a consensus among the reviewers. It is difficult to judge how will it look like after that, and thus impossible to assign a higher score based on this.
During our discussion, we identified another point that could enhance the rigor of the paper, specifically related to your question about $\gamma$.

$\gamma$ is indispensable in the definition, as it is one of the key concepts that distinguishes imperfect information games from sequential signaling games. Our work on hand abstraction is also based on $\gamma$ in games with ordered signals. However, your point is also valid—$\gamma$ can indeed be deduced from $\tilde{\rho}$ and $\tilde{\tau}$ (in fact, its definition originates from this). The issue lies in whether $\gamma$ appears in other components of the definition of $\tilde{\Gamma}$. If it does not, then it could be excluded from the definition and treated as an extended concept. However, if $\gamma$ does appear in other components of $\tilde{\Gamma}$, then it should be included in $\tilde{\Gamma}$. Our original intention was the latter, but this may not have been clearly reflected in the current version. We are considering adjusting the definition of $\Theta$ to $\Theta = (\Theta^{(1)}, \dots, \Theta^{(\max{\mathfrak{r}})})$, so that $\gamma$ would appear in other parts of the definition of $\tilde{\Gamma}$. This adjustment would also allow us to avoid the need for an additional definition of the terminal signal set $\tilde{\Theta}$, as we could directly use $\Theta^{(\max \mathfrak{r})}$ (we have always felt that introducing $\tilde{\Theta}$ in the introduction was somewhat inelegant).

Regarding your question about the initial state, after careful consideration, we agree that your point is well-founded. Introducing an initial state does enhance the generalizability of the model, allowing it to be applied to different initial states in poker games. For Texas Hold'em, we can define $x^o = (\tilde{x}^o, \phi)$, where $\tilde{x}^o$ is the root public node, and $\phi$ indicates that no cards have been dealt.

Once again, we sincerely appreciate your thorough and detailed review. Your suggestions have prompted us to think more deeply and improve our paper. Furthermore, our paper is currently in a borderline state with a reasonable chance of acceptance. If you could give us one more score, it would be greatly helpful. We are confident in the contributions of this work, as it could significantly enhance the performance of AI in Texas Hold'em (after all, previous hand abstraction methods have resulted in significant information loss). If this paper could be accepted sooner, it would promptly impact the field (since work in hand abstraction has stagnated for the past decade). We sincerely hope you can lend us your support.

Best regards,
the authors
I thank the authors for their thorough response and addressing all of the points raised in my review. I think the paper can be significantly enhanced by incorporating those explanations into the text, as promised by the authors.

Two minor points regarding the above:
 - In the question about $\gamma$, I meant that, as far as I understand, it is possible to deduce it from $\rho$ and $\tau$ - so I am puzzled as to why it is (seemingly unnecessarily) included in the game definition.
 - Even though for poker specifically, the initial distribution is known, an abstract definition of a game with imperfect information still requires it to be specified.

I decided to keep my original rating.
We are pleased to have had the opportunity to explain our work to you. Thank you for raising your evaluation of our work, which is very important and helpful to us.

May you continue to find success in your work and happiness in your life.

Best regards,
the authors.
Thank you for the clarifications. I think that the paper's presentation could be significantly improved by incorporating some of the explanations given in your rebuttal—this will clarify the significance of the paper to a wider audience.
## Winrate Isomorphism

- Line 179 - "an identical Winrate-based feature uniquely determines an abstracted signal infoset" - is that a definition of the abstracted signal infosets for winrate-based features? If so, this requires proof or justification (that they partition nicely, satisfy order conditions, interact with $\varsigma$ in the right way etc).

  We can argue this intuitively, albeit not rigorously. Each signal infoset has a unique potential winrate feature or k-recall winrate feature (obtained by enumerating all opponent holes and then calculating the statistics, a deterministic and non-random process; note that these features are defined on signal infosets, not on individual signals). We then arbitrarily define signal infosets with the same potential winrate feature or k-recall winrate feature as equivalence classes, i.e., an abstracted signal infoset. Therefore, each signal infoset will be in one and only one abstracted signal infoset.

- The above question is even more relevant for KrwEmb, which uses a complicated clustering mechanism inside.

  You can think of it this way: KROI is an input to a K-Means problems, and the K-Means algorithm will assign each input data point to a unique class. In the previous question, we also argued that each signal infoset will definitely be assigned to one of KROI's class. Therefore, KrwEmd will also assign each signal infoset to a unique abstracted signal infoset.

- why is $\mathcal{D}$ an isomorphism  

  As mentioned above, we use isomorphism to represent the equivalence with respect to a certain feature. Signal infosets determined to be isomorphic by $\mathcal{D}$ have the same certain feature, such as potential winrate/outcome feature or k-recall winrate/outcome feature.

- What is the intuition behind constructing PWI in this particular way (i.e. with "lose at least to one other player"/"win with with $l-1$ players and tie with the rest")? Is this a natural choice? Why not "lose to $l$ players" or some other metric?

  Since Texas Hold'em is a winner-takes-all game, during the showdown, if you lose to any other player who reaches the showdown, you will lose your chips (though it may not be all your chips due to the all-in and side pot rules, but we are not discussing such complexities here. In academic discussions, the simplified problem typically assumes that all players bring the same amount of usable chips to the game. In this case, if your hand is lower than that of any other player at the showdown, you will lose all your chips). Therefore, as long as your hand is not the highest, it means you lose. A tie occurs only when a player's hand is the highest but not the sole highest hand.

## Experiments

- Line 299 - "We gauge the performance over exploitability" - what does that mean?

  We use the metric of exploitability to evaluate the performance of each signal abstraction algorithm.

- Line 336 - "the final number of abstracted infosets is set to" - are these numbers for Ehs, PaEmd and KwrEmd respectively? In what order?

  The word "final" is a typo that was mistakenly added and it introduced ambiguity. (100, 225, 396) As described in line 320, phase 1 sets 100 abstracted signal information sets, phase 2 sets 225 abstracted signal information sets, and phase 3 sets 396 abstracted signal infosets.

- Line 338 - why does KrwEmd use so many parameters here, instead of just 3?

  We apply KrwEmd in the Numeral211 Hold'em environment. Clustering is not needed in the first phase due to the small number of singal infosets, so we directly use LI. We cluster the signal infosets for the second and third phases separately. Clustering in the second phase needs to consider two phases, phase 1 and phase 2, requiring two parameters $(w_{2,0}, w_{2,1})$. Clustering in the third phase needs to consider three phases, phase 1, phase 2, and phase 3, requiring three parameters $(w_{3,0}, w_{3,1}, w_{3,2})$. A total of five parameters are needed.

- If the costs of 1000x1427.7s, 12x11.2s and 96.7x341.4s are total costs, wouldn't it be more fair to set the hyper-parameters of the algorithms such that their total computational budget is approximately equal? Otherwise, it's difficult to judge the performance improvement.

  As we mentioned in the **Weakness** section, signal abstraction is a data preprocessing process that is computed only once in the entire Poker AI construction. As long as it is computable, even if it takes months to compute high-quality hand abstraction data, it is worth the effort. These hand abstraction data can then be used for AI iterations with algorithms such as CFR, as well as for real-time strategy solving. We acknowledge that our algorithm has a very high computational cost, mainly because the input data considering historical information is several orders of magnitude larger than the input data that ignores historical information. However, this results in much higher quality abstraction data compared to previous algorithms.
- Are the survival status and the  relevant to the paper? Is the signal abstraction refinement relevant to the paper?

  Survival status is part of the definition of ordered signal games. It determines at which nodes the order of signals will affect the player's payoff, thus it is relevant to our discussion. Although it might seem less important when studying signal abstraction alone, omitting this concept might lead to misunderstandings regarding the orthogonal relationship between the signals and public tree.

  Signal abstraction refinement is a relationship between signal abstractions that describes the inclusion relationship in terms of the number of infosets (information sets) identified. In our paper, we use the conclusion that POI is a signal abstraction refinement of the mainstream abstraction algorithms Ehs and PaEmd (see line 311). Applying this conclusion, we only need to demonstrate that if KrwEmd performs better than POI while identifying the same number of information sets as POI, then KrwEmd is a stronger abstraction algorithm than Ehs and PaEmd.

- Why is $\gamma$ provided in the definition of the game and not just derived from other structures?

  In imperfect information games, there is no concept of phases. However, in games with ordered signals, we define phases as the number of times the chance player (nature) takes actions during the game. The definition of phases was first formally described by [^3], and a more detailed definition was provided by [^1]. With the definition of phases, the problem of signal abstraction gains a more mathematical description.

- What happens to observations $O$ and the map $\varsigma$ in the signal abstracted game $\Gamma^{\alpha}$?

  We can use the Table.1 to answer this question. Let's assume $\alpha = (POI, \varTheta_2)$ (i.e., player 1 uses POI abstraction while player 2 does not use any abstraction). $O_2([J,Q;K]) = \\{[J,Q;K], [Q,Q;K], [K,Q;K]\\}$ and $O_1([J,Q;K]) = \\{[J,Q;K], [J,J;K], [J,K,K]\\}\cup \\{[J,J;Q], [J,Q;Q], [J,K;Q]\\} \cup \\{[Q,J;J],[Q,Q;J], [Q,K;J]\\}$。In another representation, we can write $O_2([J,Q;K]) = \\{[-,Q;K]\\}$ and $O_1([J,Q;K]) = \\{[J,-;K]\\}\cup \\{[J,-;Q]\\} \cup \\{[Q,-;J]\\} = POI_{2-3}$. 
  
  The $\varsigma$ in $\tilde{\Gamma}$ and $\tilde{\Gamma}^{\alpha}$ is the same; signal abstraction only affects observation.

- Is the criterion of the signal perfect recall for a game $\Gamma$ equivalent to saying that "players can be Markov"? I.e. that for any non-Markov (wrt to signals) strategy profile $\pi=\left(\pi_i, \pi_{-i}\right)$ there exists a Markov (wrt to signals) strategy $\pi_i^{\prime}$ such that $(\pi^\prime_{i}, \pi_{-i}) \equiv \pi$?

  In imperfect information games, the intuitive meaning of perfect recall is remembering all the information observed by the players throughout the game. In contrast, in stochastic games, the Markov property indicates that the future outcome of the game depends only on the current state (i.e., the current state includes all necessary information). The two concepts can indeed be compared. Regarding the strategy comparison you mentioned, I don't quite understand its specific meaning.

- I think the signal space $\Theta$ has to be assume to be finite - otherwise, the definition of perfect recall breaks.

  You are right. Although we aim to discuss the signal abstraction problem more generally, for now, $\Theta$ should be restricted to a finite set.
## Intro

- What are the "clustering settings" referred to in line 66? (It's difficult to understand how important it is for the claim 'outperforms the SOTA')

  By "clustering settings," we refer to the approach of using clustering techniques for hand abstraction, with representative techniques being Ehs and PaEmd. Although clustering-based abstraction has become mainstream, there are still other approaches, such as the Lossless Isomorphism methods developed by Gilpin and Sandholm and Waugh[^3][^4], as well as rule-based abstraction methods based on human expertise[^5]. This confusion arises from my choice of words; a more accurate expression would be "In the context of clustering-based abstraction algorithms,...".
  
## Prelims

- I think the definition of the game  should include the initial distribution over signals, to start iterating the map.

  The distribution of signals is fixed and known to everyone, but you cannot have the opponent's observation of the given signal (opponent's hole), and therefore cannot determine the distribution of subsequent signals. For example, in Leduc Hold'em, if a player receives the hole [J] and knows that the opponent's hole is [Q], they can accurately determine that the probabilities of the board being [-; J], [-; Q], and [-; K] are 0.25, 0.25, and 0.5, respectively. However, since the player cannot know the opponent's hole, they can only estimate a prior probability: the board has a 0.2 probability of being [-; J], a 0.4 probability of being [-; Q], and a 0.4 probability of being [-; K]. This distribution is typically known in poker games, similar to looking up a table, so there is no need to discuss the initial distribution separately.

- I did not understand how exactly the map $\\varsigma$ interacts with the shape of the game tree given by $X, \\tau$ and the order $\\sqsubseteq$ on $\\Theta$ - are we assuming that $\\theta \\sqsubseteq \\theta^{\\prime}$ for all $\\theta^{\\prime}$ in the codomain of $\\varsigma(\\theta)$ (dividing the signal space appropriately)? Are the terminal signals $\\tilde{\\Theta}$ take wrt to the order $\\sqsubseteq$, or wrt to the "realisable" order of ' $\\theta$ is final if it can only ever appear as the final signal of any game trajectory'? Is the order $\\sqsubseteq$ assumed to be tree-shaped? (line 12 of algorithm A2 requires a unique predecessor of $\\vartheta$ )

  Signals can indeed be understood as a tree structure because we require each signal to have a unique predecessor. In Leduc Hold'em, a possible signal is [J, Q; K], which is a second-phase signal indicating that in the first phase, player 1 received the hole J and player 2 received the hole Q, and in the second phase, the board K was dealt. The unique predecessor of this signal is [J, Q], a first-phase signal indicating that player 1 received the hole J and player 2 received the hole Q. Symbolically, we have [J, Q] $\\sqsubseteq$ [J, Q; K].

  $\varsigma$ is only related to the current signal and is independent of $X$ and $\\tau$. As illustrated in the above example, $\\varsigma([J, Q]) = \\{ 0.25[J, Q; J], 0.25[J, Q; Q], 0.5[J, Q; K] \\} $.

  The description of terminal signals is indeed not very detailed, which might confuse readers unfamiliar with poker game rules. We can illustrate this with an example: in Leduc Hold'em, terminal signals are all the second-phase signals, and an order can be defined on these signals. For instance, we have $\\succcurlyeq ([J, Q; K], 1, 2) = false$ and $\\succcurlyeq ([J, Q; K], 2, 1) = true$, meaning that for the terminal signal [J, Q; K], the order of player 1 is lower than that of player 2.

  Are all signals at terminal nodes terminal signals? No. For example, if a player folds in the first phase, it will also create a terminal node, but there is no need to compare orders at this terminal node. Are all terminal signals at terminal nodes? Not necessarily; the terminal signal [J, Q; K] can also appear at internal nodes in the second phase (i.e., when player 1 and player 2 are still making decisions). If we consider signals as a tree structure, we can assert that signals without successors are terminal signals, while those with successors are not. Terminal signals are the leaf nodes of the signal tree.

- Why is the order $\\preccurlyeq$ assumed to be partial? Shouldn't it be total on $\\tilde{\\Theta}$ ?

  You are right, that was a typo. $\\succcurlyeq$ is a total order on terminal signals (i.e. $\\tilde{\\Theta}$), but what I meant is that it is a partial order on $\\Theta$. The domain of $\\succcurlyeq$ projected onto $\\Theta$ is $\\tilde{\\Theta}$ exactly.
Thank you for pointing out the typos, which will be corrected in the revised version. It seems that many of my terms and abbreviations were not clearly explained. In the revised version, I will include a glossary in the appendix. For now, I will provide preliminary explanations for the terms you found confusing.

- What are the outcome-based features? (never explained, but used throughout this and next sections). What is POI?
  
  In line 164, we expand the abbreviation POI to "potential outcome isomorphism," which originates from the paper [^1]. Outcome-based features is a general term for k-recall outcome features and potential outcome features, introduced in the same paper.
  
  To explain these concepts, we take the Leduc Hold'em environment[^2] as an example, focusing on potential outcome features and potential outcome isomorphism. Due to space constraints, we will not discuss k-recall outcome features. We denote a player's hand in the first phase as [x] and in the second phase as [x;y], where x is the hole and y is the board. Potential outcome features are equivalent to potential winrate features in the terminal phase. Before the final phase, a potential outcome feature describe a distribution of potential outcome features for the next phase (or, equivalently, a distribution of potential outcome isomorphisms). Potential outcome isomorphism refers to classifying hand categories using potential outcome features.
  
  For example, in Leduc Hold'em, there are nine possible hand combinations in the final phase (phase 2): [J;J], [J;Q], [J;K], [Q;J], [Q;Q], [Q;K], [K;J], [K;Q], and [K;K]. However, their potential winrate (outcome) features fall into only three categories. Therefore, the potential outcome isomorphism in this phase consists of three classes:
   
  Table.1
  
  | POI | hands               | (loss, draw, win) |
  |:---:|:-------------------:|:-----------------:|
  | 2-1 | [J;J], [Q;Q], [K,K] | (0, 0, 1)         |
  | 2-2 | [Q;K], [K;J], [K,Q] | (0.25, 0.25, 0.5) |
  | 2-3 | [J;Q],[J;K],[Q;J]   | (0.75, 0.25, 0)   |
  
  In the first phase, there are three possible hand combinations: [J], [Q], and [K]. Their potential outcome features also fall into three distinct categories. Therefore, the potential outcome isomorphism in this phase consists of three classes:

  Table.2
  
  | POI | hands | (2-1, 2-2, 2-3)                            |
  |:---:|:-----:|:------------------------------------------:|
  | 1-1 | [J]   | (0.2 {1[J;J]}, 0, 0.8 {2[J;Q]+2[J;K]})     |
  | 1-2 | [Q]   | (0.2 {1[Q;Q]}, 0.4 {2[Q;K]}, 0.4 {2[Q;J]}) |
  | 1-3 | [K]   | (0.2,0.8 {2[K;J]+2[K;Q]},0)                |

- Why is an isomorphism? 

  Using the term isomorphism, we aim to express a classification equivalence. For example, in Table.1, the hands [J;J] and [Q;Q] have the same potential outcome (winrate) features, and therefore, they are isomorphic in terms of potential outcome (winrate) features.

- What is the River phase? What is HUNL&HULHE?

  HUNL stands for Heads-Up No-Limit Texas Hold'em, and HULHE stands for Heads-Up Limit Hold'em (see line 225). The rules regarding hands are the same for both HUNL and HULHE, which is why I group them together. In both HUNL and HULHE, the game is divided into four phases: Preflop (phase 1), Flop (phase 2), Turn (phase 3), and River (phase 4).

- What is "mb/g" in Figures 4 and 5?

  We indeed overlooked the explanation of this unit. mb/g means milli-blind per game, which can be understood as the number of blinds won or lost per thousand games. In HUNL and HULHE, a similar unit is mbb/g (milli-big-blind per game). However, in Numeral211 Hold'em, we do not distinguish between small and big blinds.