Temperature Regret Matching for Imperfect-Information Games

Abstract: Counterfactual regret minimization (CFR) methods are effective for solving two player zero-sum extensive games with imperfect information.  Regret matching (RM) plays a crucial role in CFR and its variants to approach Nash equilibrium. In this paper, we present Temperature Regret Matching (TRM), a novel RM algorithm that adopts a different strategy. Also, we consider not only the opponent's strategy under the current strategy but also the opponent's strategies of the several last iterations for updating the external regret of each iteration. Furthermore, we theoretically demonstrate that the update of TRM converges to Nash Equilibrium.  Competitive results in imperfect-information games have verified its effectiveness and efficiency.

Reviews and author responses:

This paper presents a new algorithm for regret matching that can be dropped into existing CFR variants. The new algorithm is claimed to demonstrate quicker convergence than existing algorithms and be more efficient. Additionally, the authors present a new proof generalizing the CFR convergence proof. 

These are substantial contributions to the literature, and if validated by the rest of the paper, would make this easily worthy of publication. However, I found the paper to lack polish; there were a number of minor typos (which didn’t impact my review), and the results were hard to evaluate. For instance, the description of the algorithm in section 3) was hard to follow. Breaking it out into a separate figure would make it more clear. 

I found the graphs in section 5 confusing. It seems that the results were broadly similar to the results without the new algorithm; I did not see a large difference. It would be easier to follow the results if the different algorithms were plotted against each other, rather than having the different algorithms be plotted in separate figures, or if there was a table summarizing the results. Additionally, the number of different lines in the plots made it hard to follow. I would recommend doing a number of comparisons within the same figure, for instance plotting TRM_CFR, RTRM_CFR, and CFR in one graph; having another graph for CFR+, and a third graph for LCFR. Of course, I would be content with any setup that the authors used as long as it was easy to follow.

Additionally, I think that the empirical evaluation would be improved if non-poker comparisons were added, e.g. in Liar’s Dice and/or the imperfect info variant of Goofspiel. Results on larger games would be nice as well, such as battleship, or Goofspiel with 6 cards.

I am not familiar with the size of the “Big Leduc” game. It would be good if the authors could include the size of the games they run on as a point of comparison.

The authors have a line about “we use the network in deep CFR to save the strategies for players i, -i in iteration t0-1 to avoid consuming too many storage resources.” More elaboration on this would be good; is this a tabular algorithm or an algorithm that uses function approximation? If the latter, these results are significantly more impressive, potentially even state of the art, and this should be called out. As far as I could tell, the word “network” was only used once in the paper, so I found that line quite confusing, and that left me wondering what the actual algorithm being run is.

I would encourage the authors to run these algorithms for more iterations. Right now, it looks as if many of the algorithms are converging to similar final levels of performance (or even getting worse exploitability as the number of iterations progresses). 

In short, this paper feels like it has the potential to be a major contribution to the literature, but I think that the current form is too unpolished to recommend it for acceptance. As such, I vote to reject the paper, but I would be willing to accept it if the authors are able to make the changes suggested.

Minor typos which did not impact the score I assigned the paper:
“iteartions” is used a few times in section 5
“startegies” in Section 5
I think the "beta" in section 3 between equations 10 and 11 should be a symbol.

I should note that, while I am familiar with the CFR literature, I am not intimately familiar with the theory, so I have low confidence in my assessment of the novelty of the theoretical contributions here. I am more confident in my assessment of the empirical contributions.

Summary:

This paper proposes two variations on the CFR algorithm for solving extensive form games. The first (Temperature Regret Matching) changes Regret Matching, the method for computing a policy from accumulated regret values.  The second ("New Regret Updating Method") changes the regret update by putting extra emphasis on adjusting to the most recent opponent strategy.  By combining both changes, the authors show a slight advantage in convergence speed over baseline CFR variants.

----------

Positives:

- The two techniques proposed are each small, straightforward, and independent changes to the CFR family (TRM more so, "New Update Method" less so). The CFR family of algorithms is an active research area, and an advance could have impact across many papers in combination with other ongoing research.

----------

Negatives:

- The paper has clarity issues throughout the work.  There are frequent minor issues (typos, poor word choices, grammatical errors) but there are also key sections that I cannot follow at all, despite being very familiar with the CFR technique and literature being extended in this paper.  I'll flag both the minor and major issues in 'Issues and Suggestions' below.

- Incorrect statements. I'll flag these in 'Issues and Suggestions' below.  There are three statements in the paper that I flagged as being simply false, and are described correctly in papers that the authors cite.

- Poorly motivated changes.  For both of the proposed changes, the text doesn't motivate why the proposed change should actually help (beyond a vague statement such as "This setting can help agents have a certain chance of producing more effective strategies."), and a proof sketch that the technique should still converge.  The experiments then show particular hyperparameter settings that show a slight advantage.

- Incremental technique and results.  While acknowledging that an ICLR paper does not have to beat baselines to be accepted, and this paper does present results under some hyperparameters that exceed baseline techniques, the advantage is *very* slight, only occurs with specific hyperparameter values, and are not averaged over repeated independent experiments.  The techniques themselves are not motivated and described well enough to be of sufficient interest on their own.

- Insufficient experiments.  The technique proposes two independent changes, and while TRM has experiments on its own (Figure 2, 3), the second ("New Regret Updating Method") is only examined in combination with TRM (in Figure 4).  These techniques needed to be examined independently as well as in combination, for us to properly attribute where an advantage might be coming from.

----------

Recommendation and Justification:

I suggest rejection.  I feel that the clarity issues, lack of motivation and intuition for the changes, and insufficient empirical analysis would each independently be reason enough to reject the paper; in combination, it is just not ready.  I've flag the issues I noted in 'Issues and Suggestions' below, which is pretty extensive, and gives concrete advice.

This is unfortunate - the authors are experimenting with changes to CFR that could have broad applicability across many CFR variants (which they did present in their combination with Vanilla CFR, CFR+, LCFR, and Deep CFR), so if they have indeed found a useful set of changes and can demonstrate it through better experiments, I can imagine a future resubmission of this paper being accepted and having impact.

----------

Questions to clarify recommendation:

I don't have a question for the authors that would clarify my review, but I've given some pretty extensive comments in the section below, with concrete and actionable ways to improve the presentation and content of the paper.  If the authors feel I've misunderstood the paper from my comments below, then I would be very open to hearing any clarifications they would like to offer.

----------

Issues and Suggestions:

I flagged a mix of major and minor issues in my review.  I'm going to list them as they occurred in the paper.

- Pg1, Intro.  Incorrect statement.  The statement that CFR+ was used to solve heads-up limit hold'em (Bowling, 2015) is true.  But the sentence continues to say "...and heads-up no-limit Texas hold'em (Moravcik 2017, Brown and Sandholm, 2018), and this is *definitely* false.  Solving a game means computing a Nash equilibrium, or in the case of Bowling 2015, approximating it to such a tight tolerance that the game is "essentially solved".  Heads-up no-limit is, unquestionably, not solved.  Not only does neither of those papers claim to have solved the game, but as far as I know, no group has claimed to have done so for no-limit poker.  Further, even if a strategy was claimed to be a solution to the game, there is no known, practical technique for computing a best response in no-limit poker, which would be required to verify how close it was to an equilibrium.  Instead, the claims made in Moravcik 2017 and Brown and Sandholm 2018 is that their respective programs had defeated human pros and top human pros, respectively.  While that is an accomplishment, it is far from solving the game, and the authors have made no claim about how close they are to Nash equilibria.

- Pg1, Intro.  Awkward phrasing.  "It helps agents defeat poker professionals."  Maybe "CFR is the technique these algorithms used to defeat poker professionals."?

- Pg1, Intro.  Clarity issue, incorrect citation.  "We prove that any strategies which are inversely positively to the external regret can help the CFR algorithm converge to an e-equilibrium."  First, the clarity issue: I cannot parse this sentence, and I can't even imagine what it's referring to.  What does "inversely positively to the external regret" mean?  Further, the sentence cites Nesterov 2005 for this statement, but the first CFR paper did not come out until 2007, so it is impossible that Nesterov has made any statement about CFR.  This whole paragraph is actually unclear; the notion of a "combinatorial regret matching method" needs to be explained, and the last sentence describes weighting external regret, but isn't it counterfactual regret that is weighted in this technique?

- Pg1, Intro.  Typo.  "...which the last iteration" --> "...where the last iteration"

- Pg1, Intro.  Typo.  "convergence , we" --> "convergence, we"

- Pg1, Intro.  Typo.  "are listed bellow" --> "are listed below"

- Pg2, bullet 1.  Clarity.  "inversely positively to the external regret" again.  This sentence doesn't make sense.

- Pg2, bullet 3.  Clarity.  "opponent's strategy of the last moment".  Does this mean "last iteration"?  Can the statement about "improves the learning efficiency" be made more precise?

- Pg2.  Typo.  "in the vanilla CFR" --> "in vanilla CFR"

- Pg2.  Typo.  "...standard benchmarks Leduc Hold'em" --> "standard benchmark".  Although, I flagged this later - the version of Leduc Hold'em used in this paper is quite different from the standard one!

- Pg2, Section 2 title.  Typo.  "Perliminaries" --> "Preliminaries"

- Pg2. Typo.  "a finitset" --> "a finite set"?

- Pg2.  Typo.  "extension form game" --> "extensive form game"

- Pg2. Typo.  "...where c is the chance." --> "...where c is the chance player."?

- Pg2. Clarity.  "u_1 + u_2 = 0" --> "u_1(z) + u_2(z) = 0 for all z \in Z"?

- Pg2, Notations.  Clarity.  The sentence "The probability of a particular action a..." describes a behavioral form strategy, which is standard in the CFR literature (after all, we use regrets to compute action probabilities at infosets).  The following sentence "We define \sigma_i to be a probability vector for player i over all available strategies in the game" describes a normal form (e.g., mixed) strategy, defined at the root as a probability distribution over pure strategies.  While these notions are equivalent in perfect recall games (you can convert from one to the other) it's confusing to define both back-to-back (especially without differentiating them!) and the latter definition doesn't appear to be needed in the rest of the paper.

- Pg2, Notations.  Clarity.  "Hence, \pi^\sigma_i(h) is the probability..." is an unclear sentence.  Maybe "...is the probability that h is reached if player i plays according to \sigma, and chance and the other players choose actions leading to h."

- Pg3, before equations 2 and 3.  Typo.  "strategy profile \sigma satisfies" --> "...that satisfies"

- Pg3, Exploitability.  Clarity.  In other CFR and poker work, it's standard to divide the two best response values by 2, giving an average exploitability across the game positions.  For example, see papers like "Accelerating Best Response Calculation", IJCAI 2011, which give the exploitability of Always Fold as 0.75 and not 1.5 (exploitable for 1.0 in one position, 0.5 in the other).  Without the /2, a player can be exploitable for more than the max amount that can be lost in a single game, which while technically correct (it's only different by a constant) is quite non-intuitive.

- Pg3, Section 2.3.  Clarity.  The sentence "Let \Sigma_a be the set of valid actions."  The previous page described \Sigma_i as the set of whole-game strategies for player i, and A(h) as the set of valid actions at history h.  So what is \Sigma_a?  The previous sentence said "consider repeatedly playing an extensive game", which suggests we really are picking actions, and not picking strategies from \Sigma_i as we would in a normal form game.

- Pg3, Section 2.3  Clarity.  "At each iteration, the player selects an action a_t and get a reward CFR makes frequent use..."  Is there a missing period after 'reward'?  And 'gets a reward'.

- Pg3, above equation 7.  Typo.  "infoaet" --> "infoset"

- Pg3, above equation 8.  "The whole regret R^T".  By "whole regret" do you mean "accumulated regret"?  That's the name I'm most familiar with from the CFR literature, and I haven't seen "whole regret" used in previous works.

- Pg3, below equation 9.  Typo.  "interation" --> "iteration".  I believe I saw this typo in several places in the paper.

- Pg3, describing CFR. Maybe move the Zinkevich 2007 cite one sentence later, since both statements (regret bound, average strategy) are citing that work and not just the first.

- Pg3, describing CFR+.  Correction.  This states CFR+ has two small changes from CFR, but it's actually three: the two mentioned here, plus the use of alternating updates, where on each iteration we update P1 and then P2 (using P1's updated strategy), instead of updating P1 and P2 simultaneously or precomputing the P1 and P2 current strategies before computing the regret changes.  This makes a surprisingly large difference in convergence speed; CFR+ performs much worse without it.

- Pg3, describing CFR+.  Clarity.  "CFR+ uses strategy on iteration T according to Regret Matching +".  Two things.  First, the previous sentence should use the name "RM+" where "regret-like value" is currently, so that the reader knows what RM+ is in this sentence.  Second, this is pretty unclear...  Maybe "CFR+ uses RM+ instead of RM to compute its strategy on iteration T.

- Pg3, describing CFR and CFR+.  Clarity.  CFR has two strategies for each player: the strategy computed from the regrets (often called the Current strategy in the CFR literature) and the average strategy (the part that converges to Nash).  Throughout this paper, a collection of other names are used, like "output strategy", and it's unclear which of these two strategies those names are referring to.  The authors should choose standard names (ideally 'current' and 'average') and use them throughout the paper, and also define them here, so that the reader knows what they are.

- Pg3, CFR.  "Optimistic CFR counts the regret..." --> "...accumulates the regret..."?

- Pg3, CFR.  "It means the iteration t regret has a weight 2t/(T^2+T)..." This sentence is unclear, and seems to disagree with the previous sentence which said the regret is multiplied by t/(t+1).  Please clarify this.

- Pg3, CFR.  In the 3rd last sentence, the phrase 'external regret' is used.  This term was used in the intro, but was never defined or described.  Further, throughout the paper, the authors seem to use it interchangeably with "counterfactual regret", but where counterfactual regret is the correct term.  The CFR theory uses counterfactual regret to bound external regret, but the regrets that are accumulated, weighted, etc, are counterfactual regrets.  Further, the next page mentiones "swap regret" without definining it.  Both of these terms, "external regret" and "swap regret" should be defined and described here in the Notations section.

- Pg4, Swap Regret.  This section seems to be connecting the TRM technique to Swap Regret (in Fig 1, for example) but I cannot follow most of this section because of clarity issues, and I can't see a place in the text that actually makes this connection.  Please clarify this entire section and make the connection to swap regret clear.

- Pg4, first sentence of TRM.  "...efficient RM method that adopts a different strategy..."  The word 'strategy' already has a meaning in this sentence (an agent's policy).  Maybe use a different term like 'a different approach' to make this easier for the reader.

 - Pg4, TRM section.  "output strategy".  I mentioned this term above, but this is the first use.  What is an 'output strategy'?  The current strategy, the average strategy, or something else?  The term suggests the average strategy (since that is the output of running CFR - the current strategy isn't used to play actual games and doesn't converge!), but Regret Matching is used to produce the current strategy (which is never "output" by the program).  Please use consistent terms, and define them in the Notations section.  And ideally, use the standard terms in the CFR literature.

 - Pg4, TRM section.  "perliminary" --> "hyper-parameter"?

- Pg4, TRM section.  Incorrect statement.  "In all past variants of CFR, each iteration strategy is given by regret matching using external regret".  This is incorrect.  Prior work has used Hedge instead of RM (see "Dynamic Thresholding and Pruning for Regret Minimization" by Brown et al, or "Solving Imperfect Information Games via Discounted Regret Matching" by Brown & Sandholm, or "Lazy-CFR" by Zhou et al, which describes CFR as using RM or AdaHedge).  And should "external regret" be "counterfactual regret" here?

- Pg4, TRM section.  There are grammatical and clarity issues throughout this section.  

- Pg4, TRM section.  Clarity.  What does it mean for an *action* to have its own set of algorithms?  An algorithm like RM takes regrets and produces a probability distribution over actions; I can imagine using several algorithms to produce different probability distributions for one infoset, but I don't understand how an action can itself have a set of algorithms.  This needs to be presented much more clearly.

- Pg4, TRM section.  Clarity.  What is a "counterfactual vector"?  A vector of counterfactual values?  Is this a vector over actions (i.e., at one infoset), a vector over private states (as in vector-form CFR, as used in CFR+), or something else?  This term was used without defining it.

- Pg5, below equation 16.  "It means we can first as m_1 algorithm for a recommended strategy, then use the regret R^T ask m_2 algorithm for a recommended strategy".  Grammar issues aside, the 'm' algorithms were previously described as indexed using two values (m11, m12, ..., m1k, m21, m22, etc), below Equation 10.  I didn't understand what these algorithms were from the previous section, but how can we index with one value here?

- Pg5, below equation 16.  "When an action with a higher regret" --> "action has a higher regret"

- Pg5, equation 17.  Note the Beta term here is 'alpha plus gamma t'.  Later, in Figure 3, Beta is described instead as 'x + t(y-x)/T', and the text describes x=alpha and y=gamma.  Why aren't the equation and the later use stated in the same way?

- Pg5, end of section 3.  The phrase "This setting can help agents have a certain chance of producing more effective strategies".  I didn't follow much of the section, but it appeared to be describing a convergence proof for using B terms in TRM.  But I didn't see any intuition described for why this might converge faster - just that it should converge.  What is the motivation behind this sentence, saying that it should have a chance of producing more effective strategies?  Is there a *proof* of faster convergence?  If not, what is the intuition behind why this might help?  As it stands, this seems like an unsupported claim.  Even saying "In our later experiments, we will show that some choices of alpha, gamma appear to converge faster than CFR" would be better than this claim.  Further - what is the intuition for alpha and gamma?  Gamma is described as affecting noise...  How about alpha?  Is there some intuition for how we should tune them, other than through a parameter sweep to try many values and see what works?

- Pg5, "A New Regret Updating Method".  This section describes a new approach for updating regret, which seems most similar to Optimistic CFR.  It looks like it can be used independently of TRM.  However, this section does not appear to give it a name!  This makes it difficult to describe later in the paper.  How would we describe using it without TRM?  What should we call it?

- Pg5, "New Regret Updating Method".  A few grammar issues: "leads to faster converge", "Linear CFR calculate the t iteration regret has a weight".

- Pg5, above equation 18.  "redefine two regret" --> "define two regret"?  Nothing is being redefined.

- Pg6, first line.  "The regret \sigma^t_i r_i^...".  Is that leading "\sigma^t_i" a typo that should be removed?  It doesn't appear in equation 19, which just defines r_i^...

- Pg6, "New Regret Updating".  More confusing uses of "output strategy".  It seems like "current strategy" from context, but it's really important to be clear about this!

- Pg6, "New Regret Updating".  "Second, when the CFR+ / Linear CFR algorithm iterates many times, according to experience, each iteration of its output strategy is close to the Nash equilibrium".  At least for CFR+, while this has been found in practice for specific  games like heads-up limit hold'em, I am not aware of any theoretical proof that the current strategy converges.  Further, it's clearly not true for most of the computation (or else we would stop, since our goal is to get close to Nash equilibrium!).

- Pg6, "New Regret Updating".  Similar to the TRM section, this section ends with an unsupported claim that this will help the algorithm converge faster.  This appears to be based on intuition, and not from proof or prior experiments, so it should be described as a hypothesis, instead of a fact.

- Pg6, Experiments.  "To verify ... TRM algorithm and regret matching method" --> "regret updating method"?  TRM was changing regret matching, and "new regret updating" was changing the update.

- Pg6, Experiments.  Incorrect statement.  Leduc Hold'em is described here, and Southey 2005 is (correctly) cited.  Leduc Hold'em is indeed a standard benchmark game.  However, what the authors describe here is not the standard version of Leduc Hold'em!  Leduc Hold'em is a limit poker game, where bets must be of a specific size (See Southey 2005, pg2, under "Leduc Hold'em").  This limit-betting game is the standard version used in other CFR work.  What the authors describe here, in the sentence "In Leduc Hold'em, the player may wager any amount of chips up to a maximum of that player's remaining stack", is a no-limit poker game.  This is a significant difference, and means that the results in Figures 3 and 4 cannot be compared against Leduc Hold'em results in other papers!  Further, the authors don't describe the stack size they use for Leduc or Big Leduc, so we can't even tell how big the game is, or replicate the experiments.  Ideally, the authors would use the standard limit game, so that the results can be compared against other work.  If they want to stay with no-limit, they should describe it as no-limit in the text, clarify that it is NOT the standard benchmark game, and also state the stack size.

- Pg6, Experiments.  Frequent grammatical errors throughout this section.

- Pg6, Experiments.  Describing Leduc poker.  "If a player's private card is paired with the community card, that player wins the game; otherwise, the player with the highest private card wins the game."  This is not true!  The player with the highest-ranked hand wins a *showdown*, if one occurs.  But if the higher-ranked player folds later in the game, they will still lose.

- Pg7, Figure 2.  Why use 'distance to Nash' on the y-axis here, instead of 'exploitability' for consistency with later results?  That would be clearer for the reader.  Also, why use a log-linear plot, when a log-log would likely be easier to read, and also consistent with the remaining figures?  Also, typo: 'equilibriumin' --> 'equilibrium'

- Pg7, Ablation Studies.  This is not an ablation study - it is a hyperparameter sweep.  An ablation study is when components of a technique are removed, to show whether the technique's success comes from one particular component, or a set of components in combination.  This section is just sweeping over B and stochastic B values, and is not an ablation study in any sense.  However, note that this paper really does need an ablation study: examining the performance of CFR, CFR+TRM, CFR+"New Update Method", and CFR+TRM+"New Update".  But CFR+"New Update", without TRM, is not examined in any of the figures.  That would have been a very important ablation study to have in this paper.

- Pg7, end of Ablation Studies.  "These results valudate that RM converges to..."  Should RM be TRM?

- Pg7, start of Poker Games.  "Since we do not know what Nash equilibrium strategies are..."  First, a clarity issue - the text means that we don't know what they are for these games...  Except that we do: we can compute them to an extremely tight tolerance (far closer than CFR gets) using a Linear Program, and Leduc Hold'em is small enough to do that.  The *real* problem, however, is that there can be multiple equilibria in these games, so measuring "Distance to Nash" is not straightforward: distance to which Nash?  Distance to any Nash?  Do we have algorithms for computing all Nash for a game?  And so on.  That is why exploitability is the better measurement than distance here.

- Pg7, typo.  "iteartions" --> "iterations"

- Pg7, poker games.  "This is because in the experiments of CFR on Leduc Hold'em, there are a large portion of histories with n average."  This sentence makes no sense to me - please clarify.

- Pg8, Figure 3.  Several issues here.  What is the unit on the y-axis - exploitability in chips/game?  Why use log10 on the y-axis and log2 on the x?  This would be clearer with log10 on both axes.  And clarify - is this the exploitability of the average strategy?  Even without the "output strategy" confusion described earlier, it should still be stated to make this clear.

- Pg8, Fig 3.  As noted earlier, the definition of B here differs from the one given in Equation 17 - why?  Also, in Figure 2, an 'a' value like 'a0.1' meant randomly sampling a Beta value in (0, 2a).  Is that random sampling of 'a' values in Figure 3, like 'a1.005-0.995' also true, or are these deterministic B values as given by the equation in the Figure 3 caption?  If deterministic, maybe avoid using 'a' in the name to avoid this confusion.

- Pg8, Figure 4.  "The exploitability (y-axis) are reported in T iterations".  What does this mean?  The y-axis measures exploitability (after small t iterations), and the x-axis measures iterations.  Also, 'itearations' --> 'iterations'.  Also, what does "RTRM" stand for?  I know it's the new regret updating method, and TRM is TRM...  so what's the R?  Once again: the "new regret updating method" really needs a name that we can refer to it by.  And this figure needs to demonstrate that technique without TRM.

- Pg8, under Figure 4.  "We use the network in deep CFR to save the startegies..." First, typo: 'startegies' --> 'strategies'.  But further - wow, Deep CFR is a lot of complication to add to this!  The memory cost of CFR usually involves two equal-sized arrays: one for regrets per infoset, the other for the accumulated average strategy.  The "New Update" method would require another array (so a 50% increase in memory) to store the previous iteration's current strategy.  That's a bit annoying, but not bad: if we can afford the memory to solve Big Leduc, then a 50% increase should not be an issue.  However - Deep CFR adds A LOT of complexity, and approximation error!  Leduc and Big Leduc are toy games (*tiny* compared to the full-scale games used in CFR research like Limit and No-Limit Hold'em) and this paper is proposing and justifying the new techniques.  That is best done with a clear experiment focusing on your contributions, without adding in extra complications like Deep CFR in order to "avoid consuming too many storage resources".  

The authors study two-player zero-sum imperfect-information (dynamic) games with multiple players and consider algorithms to find Nash Equilibrium (NE). The main contribution is a novel approach referred to as Temperature Regret Matching (TRM) for counterfactual regret minimization: this method helps the average strategy converges faster, obtaining NE.

The paper looks well written and structured. The proposed approach has both theoretical groundings and a large experimental evaluation. I have doubts that the experimental evaluation is done properly given stochastic nature of evaluation (see the 3rd question)

Main questions:

-	In Sec.4, the authors write “.. we propose Temperature Regret Matching (TRM), an efficient RM method ..”. Does efficiency be proved for TRM? In fact, I read the paper and did not find a proof of this claim. Is it possible that the term "efficient" is used informally? If yes, it would be better to rephrase this place, because “efficient” is usually a formal notion in game theory.
-	It would be interesting to know possible applications of the proposed methods in some industrial problems (not poker games). Maybe give an overview. 
-	In Sec.5: Does a statistical significance level be measured for differences between performance metrics of evaluated methods?



Minor issues:
-	I’m not sure that the 3-rd contribution (in Sec.1) has best presentation in the current writing. Possibly, presentation here should be improved
-	Remove space between “CFR+ (Tammelin, 2014)” and a comma in the last paragraph in Sec.1



Meta-review: This paper consider an important problem CounterFactualRegret (CFR) minimization, and proposes a new algorithm to solve this problem.

Reviewers raised many questions and concerns that the authors chose not to answer.

We can only recommend rejection

Decision: withdrawn-rejected-submissions